A simple, in-memory FIFO queue for Elixir with worker pooling and rate limiting.
OPQ is an Elixir library that provides a simple, in-memory FIFO queue with built-in worker pooling and rate limiting. It solves the problem of managing background tasks and concurrent job processing in Elixir applications efficiently. The library leverages Erlang's queue module and Elixir's GenStage for reliable performance.
Elixir developers who need a lightweight, in-memory queue for background job processing, task management, or rate-limited operations in their applications.
Developers choose OPQ for its simplicity, performance, and essential features like worker pooling and rate limiting without the overhead of more complex job processing systems. It integrates seamlessly with Elixir's concurrency model and supervision trees.
Elixir queue! A simple, in-memory queue with worker pooling and rate limiting in Elixir.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Leverages Erlang's queue module for efficient FIFO operations, ensuring low-latency processing as emphasized in the README's focus on speed.
Easily starts as part of Elixir supervision trees with minimal configuration, demonstrated in the usage examples for seamless application setup.
Supports configurable worker counts and custom worker modules, allowing tailored job processing as shown in the CustomWorker example.
Offers configurable intervals to control processing speed, useful for scenarios like rate-limited API calls, with bypass option when not needed.
Jobs are stored only in memory, so they are lost on application crashes or restarts, limiting reliability for critical workloads without external backups.
Designed for in-memory use on a single node, lacking built-in support for distributed queues or cluster synchronization, which restricts scalability.
While timeouts are configurable, there's no inherent retry logic, dead-letter queues, or advanced error recovery mechanisms, requiring manual implementation.