A one-stop, full-scenario integration framework for massive data, supporting data ingestion, synchronization, and subscription.
Apache InLong is a one-stop, full-scenario integration framework for massive data that supports data ingestion, synchronization, and subscription. It provides automatic, secure, and reliable data transmission capabilities, enabling both batch and stream data processing for real-time applications. The framework is designed to handle ultra-large-scale data reporting environments, simplifying data flow from source to target clusters.
Data engineers and developers building real-time data applications, especially those needing to manage massive data streams in big data scenarios. It suits environments requiring rapid data reporting platform setup and automated data sorting.
Developers choose InLong for its proven stability in production, handling trillions of data pieces daily, and its comprehensive feature set including real-time ETL and pluggable architecture. It offers a unified platform for data integration, reducing complexity in managing diverse data sources and sinks.
Apache InLong - a one-stop, full-scenario integration framework for massive data
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Handles over 80 trillion data pieces daily with high reliability for 100 billion-level streams, as derived from Tencent's production environment.
Supports numerous extract and load nodes including Kafka, MySQL, HDFS, and more, covering common big data technologies for seamless integration.
Provides built-in extract, transform, load (ETL) and rule-based sorting capabilities, enabling complex data processing pipelines without external tools.
Offers a SaaS-based dashboard with fine-grained metrics and configurable alert services for easy visualization and error handling.
Requires Java JDK 8, Maven, and Docker for building, and deployment involves multiple components (Ingestion, Convergence, Caching, Sorting, Management), making setup cumbersome.
New users must understand various configurations and modular architecture, which can be challenging given the sparse beginner-friendly documentation in the README.
The supported data nodes list is marked as 'Updating,' indicating potential gaps or instability in integrations compared to more established frameworks like Apache Flink.