A source-agnostic distributed change data capture system for reliably capturing and streaming primary data changes.
Databus is a source-agnostic distributed change data capture system built by LinkedIn to reliably capture and stream primary data changes from databases like Oracle and MySQL. It solves the problem of keeping secondary data stores, caches, and indexes consistent with primary data mutations by mining database transaction logs, avoiding the consistency issues of application-driven dual writes.
Data engineers and architects at technology companies building complex data ecosystems with multiple specialized data stores (e.g., relational databases, NoSQL, caches, search indexes) that need reliable, low-latency data change propagation.
Developers choose Databus for its source-agnostic approach that prevents technology lock-in, its strong consistency guarantees by using database logs as a single source of truth, and its ability to handle high throughput with low latency while supporting infinite data lookback.
Source-agnostic distributed change data capture system
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Extracts changes from database transaction logs like Oracle and MySQL without relying on proprietary replication, avoiding technology lock-in, as highlighted in the Key Features.
Solves consistency issues by mining database logs as a single source of truth, unlike error-prone application-driven dual writes, per the Philosophy and Use-cases sections.
Delivers transport layer latencies in low milliseconds and handles thousands of events per second per server, ensuring scalable data flow for high-throughput environments.
Supports replaying historical data changes from any point in time, enabling flexible subscription and recovery capabilities for downstream applications.
Requires downloading Oracle JDBC driver with specific licensing and version, and uses gradle for build, making initial setup non-trivial and prone to errors, as warned in the How to build? section.
Primarily designed for Oracle and MySQL sources; adapting to other databases requires custom log mining implementations, which may not be well-documented or supported.
Based on a 2012 paper and older technology, the project might have less active development and community support compared to modern CDC tools like Debezium, risking obsolescence.