A Go library implementing consistent hashing with bounded loads for uniform key distribution and load balancing.
Consistent is a Go library that implements consistent hashing with bounded loads, a distributed algorithm for mapping keys to nodes in a cluster. It solves the problem of uneven load distribution and excessive data movement when nodes are added or removed, making it ideal for scalable load balancers, caches, and distributed databases.
Go developers building distributed systems such as caches, databases, load balancers, or any service requiring scalable and efficient key-to-node mapping.
Developers choose Consistent for its strict adherence to the bounded loads algorithm, which guarantees no node is overloaded, and its production-ready performance with minimal relocation overhead during cluster changes.
Consistent hashing with bounded loads in Golang
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Guarantees no node exceeds the calculated average load, preventing hotspots as per the research-backed algorithm from Google, ensuring predictable performance in distributed systems.
Locates key owners in constant time (252 ns/op in benchmarks) with no memory allocation during key location, making it efficient for high-throughput applications.
Limits partition movement when nodes are added or removed, as demonstrated in the example where only 6% of partitions relocated, reducing data transfer and system churn.
Supports concurrent member additions, removals, and key lookups, making it suitable for production environments with multiple goroutines.
The number of partitions cannot be changed after creation, limiting scalability adjustments without reinitializing the entire consistent hashing ring, which can be disruptive.
Requires implementing a custom Hasher interface, adding setup complexity and potential for errors compared to libraries that provide built-in hash functions out of the box.
If all members are at maximum load, attempting to add a new member causes a panic, which may crash the application rather than offering graceful error handling or queuing mechanisms.