A MySQL change data capture daemon that streams database changes as JSON to Kafka, Kinesis, and other platforms.
Maxwell's daemon is a change data capture application that reads MySQL binary logs and streams database changes as JSON to platforms like Kafka and Kinesis. It solves the problem of real-time data synchronization by capturing inserts, updates, and deletes without modifying application code.
Data engineers, DevOps teams, and backend developers building real-time data pipelines, ETL processes, or event-driven architectures that involve MySQL databases.
Developers choose Maxwell's daemon for its simplicity, reliability, and compatibility with modern streaming platforms. It provides a lightweight, non-intrusive way to capture database changes in real-time without complex middleware or application rewrites.
Maxwell's daemon, a mysql-to-json kafka producer
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Operates by reading MySQL binlogs without requiring any changes to application code, making it easy to deploy and maintain, as emphasized in the key features.
Outputs database changes as clean, schema-aware JSON with database, table, operation type, and data values, facilitating easy integration with modern data pipelines.
Supports writing events to Kafka, Kinesis, and other streaming systems, enabling flexible integration with various data infrastructure, as highlighted in the features.
Continuously monitors binlogs to capture inserts, updates, and deletes in real-time, supporting immediate data synchronization for ETL and audit logs.
Limited to MySQL databases, making it unsuitable for projects using other database systems, which restricts its use in heterogeneous environments.
Requires external streaming platforms like Kafka for output, adding setup and maintenance complexity, and it doesn't natively support simpler outputs like files.
Reading binlogs can impose performance costs on the MySQL server, potentially affecting high-throughput applications, a trade-off not explicitly addressed in the README.