walex
A simple and reliable Elixir library for capturing Postgres change events (CDC) via logical replication.
What is walex?
WalEx is an Elixir library that implements Change Data Capture (CDC) for PostgreSQL, allowing developers to listen to insert, update, and delete events on database tables in real-time. It solves the problem of reacting to database changes without polling or modifying application logic, enabling use cases like streaming data to external services, sending notifications, or updating caches automatically.
Elixir developers building applications with PostgreSQL who need to react to database changes in real-time, such as for event-driven architectures, data synchronization, or external integrations.
Developers choose WalEx for its Elixir-native design, simplicity in setup, and reliability in handling PostgreSQL logical replication, offering a clean DSL for event handling without the overhead of external CDC services.
Overview
Postgres change events (CDC) in Elixir [WAL]
Use Cases
Best For
- Streaming PostgreSQL changes to external services like analytics or CRMs
- Sending welcome emails or notifications after user account creation
- Augmenting existing Postgres-backed applications with event-driven business logic
- Updating search indexes or invalidating caches on database changes
- Building real-time data pipelines within Elixir applications
- Handling database change events with custom filtering and transformation logic
Not Ideal For
- Projects using databases other than PostgreSQL, as WalEx is exclusive to PostgreSQL and lacks support for other DBMS.
- Teams seeking a fully managed, external CDC service without embedding logic in Elixir; tools like Sequin might be better for streaming to Kafka or S3.
- Applications with extremely high-throughput change events requiring out-of-the-box back-pressure handling, as WalEx's implementation is advanced and requires custom setup.
- Environments where PostgreSQL server configuration cannot be modified (e.g., shared hosting or locked-down production DBs), due to the need for wal_level changes and publication creation.
Pros & Cons
Pros
Leverages PostgreSQL's native logical replication slots for reliable, low-latency change capture without polling, as detailed in the configuration steps for wal_level and publications.
Provides a clean callback interface with on_insert, on_update, and on_delete, allowing intuitive event handling with pattern matching and filtering, as shown in the Event Module examples.
Supports runtime configuration updates and subscriptions to specific tables or all tables, enabling dynamic control over monitored changes without restarting.
Offers both temporary and durable replication slots, balancing event loss prevention with disk safety, though durable slots require careful management to avoid issues.
Cons
Exclusively tied to PostgreSQL, making it unsuitable for multi-database setups or applications using other databases, as admitted in the README's alternatives section.
Requires non-trivial PostgreSQL server adjustments like setting wal_level, creating publications, and configuring replica identity, which can be a barrier for teams with limited DBA access or in restricted environments.
Features like durable slots and back-pressure handling are marked as advanced with warnings about gotchas and incomplete documentation (e.g., 'a detailed guide is planned'), potentially leading to operational challenges.
Frequently Asked Questions
Related Projects
ectoA toolkit for data mapping and language integrated query.
postgrexPostgreSQL driver for Elixir
eventstoreEvent store using PostgreSQL for persistence
redixFast, pipelined, resilient Redis driver for Elixir. 🛍
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.