camus
LinkedIn's previous generation Kafka to HDFS pipeline for batch data ingestion.
What is camus?
Camus is LinkedIn's previous generation data pipeline that transfers data from Kafka message queues to Hadoop HDFS in batch mode. It solves the problem of making streaming data available for batch analytics by providing a reliable, scalable ingestion pipeline between these two distributed systems.
Data engineers and infrastructure teams who need to move data from Kafka to Hadoop for batch processing, analytics, or data warehousing purposes.
Developers chose Camus for its proven reliability at LinkedIn's scale, straightforward integration with existing Kafka and Hadoop ecosystems, and its specialized focus on batch-oriented data transfer between these specific technologies.
Overview
LinkedIn's previous generation Kafka to HDFS pipeline.
Use Cases
Best For
- Moving Kafka data to HDFS for batch analytics
- Building ETL pipelines between Kafka and Hadoop ecosystems
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
