Abstract:
Most P2P systems that provide a DHT abstraction %(such as CAN, Chord) distribute objects randomly among ``peer nodes'' in a way that results in some nodes O(log N) times as many objects as the average node. Further imbalance may result due to non-uniform distribution of objects in the identifier space and a high degree of heterogeneity in object loads and node capacities. Additionally, a node's load may vary greatly over time since the system can be expected to experience continuous insertions and deletions of objects, and skewed object arrival patterns. %, and continuous arrival and departure of nodes.
In this paper, we propose and study an algorithm for load balancing in such heterogeneous, dynamic P2P systems. We divide our study into two parts. In the first, we simulate a storage application and study the tradeoffs achieved by the tradeoffs. In the second, we implement a simpler version of the load balancing algorithm over the Bamboo DHT and run a parallel synthetic pattern search application on top of it to show the feasibility of our approach in practice.
Our simulation results show that in the face of rapid arrivals and departures of objects of widely varying load, our algorithm achieves load balancing for system utilizations as high as 90% while moving only about 8% of the load that arrives into the system. We also show that our distributed algorithm performs only negligibly worse than a similar centralized algorithm, and that node heterogeneity helps, not hurts, the scalability of our algorithm. Our implementation shows we are able to balance unpredictable load in practice and that checkpointing state into the DHT to avoid redoing work is a feasible approach.