How to use doublejump for load balancing in Go microservices?

Import the doublejump package, create a hash instance with NewHash(), add backend nodes using Add(), and distribute requests by calling Get() with a key like a request ID. When nodes fail or are added, use Remove() and Add() to update the ring dynamically, ensuring minimal key redistribution.

Is doublejump faster than StathatConsistent for consistent hashing in Go?

Yes, benchmarks show doublejump runs at 22-46 ns/op with zero allocations, while StathatConsistent averages 273-367 ns/op with allocations, making doublejump 10-20 times faster and more memory-efficient for high-performance scenarios.

What changes when upgrading from doublejump V1 to V2?

V2 requires Go 1.18+ and introduces a generic interface, so you need to update imports to '/v2' and use type parameters like NewHash[string](). The Get() method also returns an additional boolean for success, improving error handling in type-safe code.

How does doublejump handle key redistribution when nodes are removed?

It uses a revamped jump consistent hash algorithm that minimizes reshuffling; only keys mapped to the removed node are reassigned to other nodes, maintaining high consistency and performance without rehashing the entire key space.

Doublejump or original jump hash: which is better for dynamic clusters?

Choose doublejump if you need to add or remove nodes dynamically, as the original jump hash doesn't support removal. For static node sets, both are fast, but doublejump's flexibility makes it superior for evolving systems.

Can I benchmark doublejump against other Go hashing libraries?

Yes, use Go's testing framework to write benchmark functions that simulate your workload. The README provides example benchmarks comparing doublejump to StathatConsistent and SerialxHashring, so you can replicate those tests or adapt them for your specific nodes and keys.

Open-Awesome

doublejump

BSD-3-ClauseGov1.0.1

A revamped jump consistent hash algorithm for Go that supports node removal with zero memory allocations.

GitHub

111 stars14 forks0 contributors

What is doublejump?

Doublejump is a Go implementation of a revamped jump consistent hash algorithm that extends Google's original design to support node removal. It solves the problem of distributing keys across nodes in distributed systems while allowing dynamic cluster changes without the limitation of fixed node sets. The library provides high-performance hashing with zero memory allocations.

Target Audience

Go developers building distributed systems, load balancers, or data partitioning solutions that require consistent hashing with dynamic node management.

Value Proposition

Developers choose Doublejump because it offers the speed and consistency of Google's jump hash while adding critical node removal functionality, outperforming alternative consistent hash implementations by 10-20x in benchmarks with zero allocation overhead.

Overview

A revamped Google's jump consistent hash

Use Cases

Best For

Implementing consistent hashing in distributed databases
Building load balancers that need dynamic backend management
Creating sharding systems with node addition/removal capabilities
Developing caching layers with consistent key distribution
Systems requiring high-performance hashing with minimal memory overhead
Go microservices that need to distribute work across worker nodes

Not Ideal For

Projects built in languages other than Go that require consistent hashing
Systems needing weighted node distribution or virtual node support for finer-grained control
Applications where simple round-robin or static hashing suffices without dynamic node changes
Teams relying on extensive third-party integrations or plugins not available in the Go ecosystem

Pros & Cons

Pros

Node Removal Support

Overcomes the key limitation of Google's original jump hash by allowing dynamic node removal while maintaining consistency, essential for real-world distributed systems.

Zero Allocation Efficiency

Operations perform with zero memory allocations, as shown in benchmarks with 0 B/op, making it highly memory-efficient and suitable for high-throughput systems.

High Performance Benchmarks

Benchmarks demonstrate 22-46 ns/op performance, significantly faster than alternatives like StathatConsistent and SerialxHashring, enabling low-latency key distribution.

Type-Safe Generic API

V2 offers a generic interface for Go 1.18+, providing type safety and modern Go features without sacrificing the minimalist design.

Cons

Go-Only Limitation

It's specific to Go, so projects in other languages cannot leverage its benefits, forcing polyglot teams to seek alternative implementations.

Basic Feature Set

Focuses solely on core hashing without built-in support for weighted nodes, replication, or fault tolerance, requiring additional logic for advanced use cases.

Version Split Complexity

The separation between V1 for Go <=1.17 and V2 for Go >=1.18 may confuse users and necessitate careful migration, especially in legacy codebases.

Frequently Asked Questions

Related Projects

Go-zero

A cloud-native Go microservices framework with cli tool for productivity.

A standard library for microservices.

Stars27,431

Forks2,444

Last commit2 years ago

Kratos

Your ultimate Go microservices framework for the cloud-native era.

Stars25,803

Forks4,168

Last commit23 days ago

grpc-go

The Go language implementation of gRPC. HTTP/2 based RPC

Stars23,013

Forks4,725

Last commit22 hours ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub