A high-performance Java library for asynchronous data ingestion to Fluentd, Fluent Bit, AWS S3, and Treasure Data.
Fluency is a high-performance Java library for asynchronous data ingestion to logging and data platforms like Fluentd, AWS S3, and Treasure Data. It solves the problem of efficiently sending large volumes of application logs and metrics with configurable buffering, failover, and backup mechanisms to prevent data loss.
Java developers and engineers building data-intensive applications, microservices, or monitoring systems that require reliable, high-throughput logging to external data pipelines or storage.
Developers choose Fluency for its superior performance over alternatives, flexible configuration for different destinations, and built-in reliability features like failover and local backup, making it ideal for production environments with strict data delivery requirements.
High throughput data ingestion logger to Fluentd, AWS S3 and Treasure Data
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Benchmarks show it's up to 4 times faster than fluent-logger-java, thanks to optimized buffering and asynchronous flushing, as highlighted in the README's performance comparison.
Includes failover with multiple servers, TLS/SSL encryption, ACK response mode, and local disk backup for unsent data, ensuring data delivery even during network issues.
Supports Fluentd/Fluent Bit, AWS S3 (JSONL, CSV, MessagePack), and Treasure Data with dedicated builders, allowing tailored ingestion pipelines.
Offers TCP/UDP heartbeats, UNIX domain sockets for Java 16+, and custom error handlers, providing fine-grained control over log transport.
Requires tuning multiple buffer parameters, flush policies, and failover settings, which can be overwhelming and error-prone without deep expertise.
Advanced features like UNIX domain socket support are only available for Java 16 or later, restricting compatibility with older environments.
Users must manually handle BufferFullException and configure max buffer sizes carefully, adding complexity to error handling and risking data loss if misconfigured.