A Redis-backed job processing library for Elixir with Resque/Sidekiq compatibility.
Exq is a background job processing library for Elixir that enables reliable, asynchronous task execution using Redis for storage. It solves the need for durable job queues with retries, concurrency control, and cross-node processing, especially in applications where jobs must persist through restarts. Its compatibility with Resque and Sidekiq allows easy integration with existing Ruby-based systems.
Elixir developers building applications that require robust background job processing, such as web apps with async tasks, data pipelines, or integrations with existing Sidekiq/Resque setups.
Developers choose Exq for its proven Resque/Sidekiq compatibility, ensuring seamless migration from Ruby ecosystems, combined with Elixir's concurrency model and reliability features like atomic enqueuing and unique job handling.
Job processing library for Elixir - compatible with Resque / Sidekiq
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Uses the same job format as Resque/Sidekiq, enabling seamless integration with existing Ruby/Rails projects and the Sidekiq UI, as detailed in the compatibility section of the README.
Jobs are stored in Redis with backup queues using RPOPLPUSH, ensuring they survive worker restarts and node failures, providing reliability for long-running tasks.
Supports automatic job retries with exponential backoff and a configurable max retries setting, making it robust for handling transient failures in distributed systems.
Prevents duplicate job enqueuing with lock-based strategies for idempotency and debouncing, though it's noted as best-effort in the Unique Jobs section.
Allows enqueuing multiple jobs atomically with enqueue_all, ensuring consistency in batch operations, as described in the Enqueuing Many Jobs Atomically section.
Requires a dedicated Redis instance, adding infrastructure complexity, serialization latency, and operational costs that may be overkill for simple use cases.
Multiple settings for queues, concurrency, modes, middleware, and recovery options can be overwhelming, with misconfiguration risks highlighted in the README's setup guides.
The unique jobs feature is implemented via middleware and explicitly described as 'best effort' in the README, not a strict guarantee against duplicate execution.
Node recovery via heartbeat is not enabled by default, requiring manual configuration for auto-scaling environments, which could lead to orphaned jobs if overlooked.
exq is an open-source alternative to the following products: