A Go library for asynchronous task queues and job processing with distributed message passing.
Machinery is a Go library that provides an asynchronous task queue system based on distributed message passing. It allows developers to offload background jobs, schedule periodic tasks, and orchestrate complex workflows across multiple workers. The library solves the problem of handling long-running or resource-intensive operations without blocking main application threads.
Go developers building scalable microservices or applications that require background job processing, workflow automation, or distributed task execution.
Developers choose Machinery for its simplicity, extensive broker and backend support, and powerful workflow features like chains and chords, all within a lightweight, idiomatic Go API.
Machinery is an asynchronous task queue/job queue based on distributed message passing.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Supports RabbitMQ, Redis, AWS SQS, and GCP Pub/Sub, allowing flexibility to integrate with existing infrastructure, as detailed in the Configuration section for each broker.
Offers groups for parallel tasks, chains for sequential execution, and chords for callbacks, enabling complex task dependencies, with examples provided in the Workflows section.
Includes exponential backoff for retries and custom error callbacks, ensuring robustness for failed tasks, explained in the Retry Tasks and Error Handling sections.
Allows scheduling recurring tasks using cron expressions for tasks, groups, chains, and chords, as demonstrated in the Periodic Tasks & Workflows section with code snippets.
Requires setting up and maintaining external brokers and backends like RabbitMQ or Redis, adding operational overhead, as noted in the Requirements and Development sections.
Only supports basic JSON-serializable types (e.g., primitives and arrays), restricting the passing of complex Go structs without custom marshaling, as listed in the Supported Types section.
Coexistence of V1 and V2 can cause confusion; V2 is recommended to avoid unnecessary dependencies but requires manual injection of brokers and backends, as described in the V2 section.