Automated orchestration tool for benchmarking message queues in scaled-up, distributed configurations.
Flotilla is an automated orchestration tool for benchmarking message queues in scaled-up, distributed configurations. It solves the problem of unrealistic single-machine benchmarks by allowing tests across multiple machines with arbitrary numbers of producers and consumers. The tool provides statistically meaningful results using HDR Histogram to analyze performance at low latencies.
Developers, engineers, and architects who need to evaluate and compare message queue performance in realistic distributed environments, particularly those designing microservices or event-driven systems.
Flotilla offers a unique approach by automating complex distributed benchmarks with a single command, supporting multiple brokers out-of-the-box, and focusing on statistical accuracy. It saves time and resources compared to ad hoc benchmarking solutions.
Automated message queue orchestration for scaled-up benchmarking.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Allows benchmarking with arbitrarily many producers and consumers across multiple machines using a single command, as shown in the example, simplifying complex setups.
Supports out-of-the-box testing for popular brokers like Kafka, RabbitMQ, NATS, and Google Cloud Pub/Sub, enabling easy comparisons without custom integration.
Uses HDR Histogram for low-latency data recording and analysis, providing more statistically meaningful results than basic benchmarks, as emphasized in the README.
Leverages Docker containers to run brokers consistently across different environments, including local and distributed setups, ensuring reproducible tests.
The project is explicitly marked as a work-in-progress with several TODOs and caveats, such as lack of security and incomplete features like clustered configurations.
In distributed tests, unsynchronized clocks between sender and receiver can skew latency measurements, as admitted in the caveats, limiting accuracy.
Requires Docker installation and running flotilla-server daemons on multiple hosts, which adds overhead and may not be feasible in all environments.
Some brokers support message batching for throughput while others don't, affecting fair comparisons, as noted in the caveats about benchmarking disparities.