Apache Iggy
A persistent message streaming platform built in Rust for ultra-low latency and high throughput, supporting multiple transport protocols.
What is Apache Iggy?
Apache Iggy is a high-performance, persistent message streaming platform written in Rust, designed to process millions of messages per second with sub-millisecond latency. It solves the need for a dedicated streaming log built from the ground up for maximum efficiency and speed, utilizing a thread-per-core shared-nothing architecture and modern I/O like io_uring. Unlike extensions on existing infrastructure, it is a standalone system prioritizing raw performance and resource efficiency.
Developers and architects building real-time data pipelines, event-driven microservices, or high-throughput streaming applications that require predictable low latency and minimal resource usage. It is suited for teams needing a performant alternative to traditional message brokers like Kafka, especially in Rust or polyglot environments.
Developers choose Apache Iggy for its exceptional throughput and ultra-low latency achieved through a Rust-based, thread-per-core shared-nothing design with io_uring, avoiding garbage collection overhead. Its unique selling point is being a purpose-built streaming log from scratch, offering multiple transport protocols (QUIC, WebSocket, TCP, HTTP), persistent logs with configurable retention, and extensibility via Rust plugins and an MCP server for LLM integration.
Overview
Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed
Use Cases
Best For
- Building real-time applications requiring sub-millisecond latency and high message throughput, such as financial trading platforms or gaming backends.
- Implementing event-driven architectures in microservices where persistent, ordered message streaming with consumer groups is essential.
- Developing data ingestion pipelines that need to handle millions of messages per second with efficient resource utilization on modern Linux systems.
- Creating extensible streaming platforms with custom data connectors using Rust plugins for sources, sinks, and transformations.
- Integrating LLM context in real-time via the Model Context Protocol (MCP) for AI-powered applications.
- Deploying a single-binary, dependency-free message streaming server with built-in security, TLS support, and management tools like CLI and Web UI.
Not Ideal For
- Systems requiring immediate, out-of-the-box high-availability clustering with automatic failover
- Deployments on non-Linux operating systems or older Linux kernels without io_uring support
- Teams needing a vast ecosystem of pre-built connectors and third-party integrations for legacy systems
Pros & Cons
Pros
Built in Rust with a thread-per-core shared-nothing architecture and io_uring, enabling processing of millions of messages per second at sub-millisecond latency, as demonstrated in its benchmarking platform.
Supports QUIC, WebSocket, TCP, and HTTP (including REST API), allowing clients to choose the best protocol for their use case without sacrificing performance.
Provides append-only logs with consumer groups, partitions, and configurable retention policies, ensuring reliable message streaming and horizontal scaling for high-throughput applications.
Features a connector runtime for custom Rust plugins (sources/sinks) and an MCP server for LLM context, enabling tailored data pipelines and real-time AI enhancements.
Cons
Heavily relies on io_uring and thread-per-core design, which are optimized for modern Linux kernels, leading to degraded performance or compatibility issues on other platforms like Windows or macOS.
Clustering and data replication based on Viewstamped Replication are still on the roadmap, making it unsuitable for distributed high-availability deployments until this feature is implemented.
As an incubating project, it lacks the extensive community-driven connectors, tools, and integrations found in more established platforms like Apache Kafka, requiring more custom development.
Open Source Alternative To
Apache Iggy is an open-source alternative to the following products:
Frequently Asked Questions
Related Projects
candle-wasm-examplesMinimalist ML framework for Rust
fluvio🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
ArroyoDistributed stream processing engine in Rust
webapp.rsA web application completely written in Rust. 🌍
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.