A high-performance, storage-agnostic message queue and worker pool system for Go with type safety and extensible adapters.
VarMQ is a high-performance message queue and worker pool system for Go that simplifies concurrent task processing. It provides type-safe queues, multiple worker abstractions, and supports persistence and distribution through adapter interfaces, enabling efficient asynchronous job handling in Go applications.
Go developers building concurrent applications that require efficient task processing, message queuing, or worker pool management, especially those needing type safety, persistence, or distributed processing capabilities.
Developers choose VarMQ for its clean API, storage-agnostic design with extensible adapters, support for multiple queue types (standard, priority, persistent, distributed), and fine-grained concurrency control, all while maintaining high performance and type safety through Go generics.
A Simplest Storage-Agnostic and Zero-dep Message Queue for Your Concurrent Go Program
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Benchmarks show low overhead with ~580ns per job for batch adds and minimal memory allocations, making it suitable for high-throughput scenarios.
Uses Go generics to ensure compile-time type safety for jobs, reducing runtime errors and providing a clean API that feels natural to Go developers.
Supports custom storage via IPersistentQueue and IDistributedQueue interfaces, with built-in adapters for Redis, SQLite, and DuckDB, enabling flexible persistence and distribution.
Allows binding multiple queues to a single worker with configurable strategies like Priority, RoundRobin, MaxLen, and MinLen, optimizing job processing from diverse sources.
The PostgreSQL adapter is listed as 'upcoming,' and other storage backends require custom implementation, limiting out-of-the-box options for production use.
Function helpers like Func() and ErrFunc() cannot be used with persistent or distributed queues, as functions are not serializable, restricting durable task workflows.
Implementing custom adapters or setting up distributed queues requires additional boilerplate and dependency management, adding complexity compared to drop-in solutions.