A Rust-based data transfer suite for ultra-fast replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka, and ClickHouse.
ape-dts is a high-performance data transfer suite written in Rust that facilitates ultra-fast data replication between various databases and systems like MySQL, PostgreSQL, Redis, MongoDB, Kafka, and ClickHouse. It solves the problem of efficient and reliable data migration, synchronization, and disaster recovery across heterogeneous data stores. The tool supports both snapshot and real-time change data capture (CDC) operations with features for data validation and custom processing.
Database administrators, DevOps engineers, and platform teams who need to perform database migrations, set up disaster recovery solutions, or implement real-time data synchronization between different data systems.
Developers choose ape-dts for its exceptional performance, lightweight footprint, and standalone architecture that eliminates dependencies on third-party components. It offers superior speed compared to alternatives like Debezium, as demonstrated in benchmarks, and provides flexible data processing capabilities through Lua scripting.
ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka and ClickHouse, ideal for disaster recovery (DR) and migration scenarios.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Benchmarks show ape-dts achieves over 126,582 rows per second in MySQL snapshot migration, significantly outperforming Debezium in throughput.
The Docker image is only 71.4 MB compared to Debezium's 1.38 GB, making it ideal for lightweight, cloud-native deployments.
Requires no third-party components or external storage, simplifying deployment in stateless infrastructures as emphasized in the philosophy.
Supports user-defined Lua scripts for on-the-fly ETL operations, allowing custom data modifications during migration.
Data check/revise/review features are missing for some target combinations like Redis to Redis, limiting thorough data integrity checks in heterogeneous migrations.
Structure migration is not supported for MongoDB and Redis, which can hinder migrations involving these databases.
Enabling advanced features like metrics requires Cargo feature flags and Rust knowledge, posing a barrier for teams unfamiliar with the ecosystem.