oban
A robust, database-backed job processing library for Elixir with PostgreSQL, MySQL, and SQLite3 support.
What is oban?
Oban is a robust background job processing library for Elixir that uses PostgreSQL, MySQL, or SQLite3 as its backend. It provides reliable, observable job execution with enterprise-grade features like unique jobs, scheduled jobs, and historic metrics retention. It solves the problem of managing asynchronous tasks in Elixir applications with strong consistency and minimal external dependencies.
Elixir developers building applications that require reliable background job processing, especially those already using PostgreSQL, MySQL, or SQLite3 as their primary database.
Developers choose Oban for its tight integration with SQL databases, which reduces system dependencies and simplifies backups. Its focus on reliability, observability, and retention of job data for metrics sets it apart from other job processing tools that delete jobs after completion.
Overview
💎 Robust job processing in Elixir, backed by modern PostgreSQL, SQLite3, and MySQL
Use Cases
Best For
- Processing background jobs in Elixir web applications with transactional integrity
- Building systems that require guaranteed job execution and no data loss
- Applications needing historic job metrics and inspection capabilities
- Teams wanting to avoid additional infrastructure like Redis for job queues
- Implementing cron-like periodic tasks across distributed nodes
- Enforcing uniqueness constraints to prevent duplicate background work
Not Ideal For
- Applications that don't use a SQL database (PostgreSQL, MySQL, SQLite3) as their primary data store
- Systems requiring sub-millisecond job latency, where database overhead may be prohibitive compared to in-memory queues like Redis
- Teams seeking a fully managed, serverless job queue without self-hosted infrastructure
- Projects needing advanced features like global rate limiting or job workflows without purchasing Oban Pro
Pros & Cons
Pros
Leverages existing SQL databases for job persistence, reducing external dependencies and simplifying backups, as highlighted in the 'Fewer Dependencies' advantage.
Enables atomic enqueuing of jobs alongside other database operations, ensuring data integrity and consistency, a key feature mentioned under 'Transactional Control'.
Jobs are retained indefinitely after processing for inspection and analytics, preventing data loss and providing deep observability, core to Oban's philosophy.
Prevents duplicate jobs with fine-grained uniqueness across arguments, queues, and workers, as described in the 'Unique Jobs' section.
Emits job life-cycle events via Telemetry for easy logging and monitoring, enhancing observability without additional plugins.
Cons
Ties the job queue to a SQL database, making it unsuitable for NoSQL-based applications or those preferring Redis for faster, in-memory processing.
Key enterprise features like workflows, batches, and global rate limiting are only available in Oban Pro, a paid extension, limiting the core library's capabilities.
Requires configuring database connections and Ecto migrations, which can be challenging for teams not already using Elixir's Ecto ecosystem.
Database-backed queueing can introduce latency compared to dedicated message brokers, especially under extreme concurrency, as noted in performance trade-offs.
Frequently Asked Questions
Related Projects
broadwayConcurrent and multi-stage data ingestion and data processing with Elixir
exqJob processing library for Elixir - compatible with Resque / Sidekiq
verkA job processing system that just verks! 🧛
honeydewJob Queue for Elixir. Clustered or Local. Straight BEAM. Optional Ecto. 💪🍈
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.