pgmq
A lightweight message queue built on PostgreSQL, offering SQS-like functionality with transactional guarantees.
What is pgmq?
pgmq is a lightweight message queue that runs directly on PostgreSQL, providing reliable message queuing with transactional semantics. It offers an alternative to cloud-based queues like AWS SQS by leveraging PostgreSQL's durability and consistency guarantees.
Developers and teams already using PostgreSQL who need a simple, reliable message queue without adding separate infrastructure, such as those building applications requiring asynchronous task processing or event-driven architectures within a PostgreSQL environment.
Developers choose pgmq because it eliminates the need for separate messaging infrastructure by leveraging PostgreSQL's built-in reliability, replication, and transactional guarantees, offering a familiar SQS-like API with minimal dependencies.
Overview
A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.
Use Cases
Best For
- Implementing reliable message queuing within existing PostgreSQL deployments without external dependencies.
- Applications requiring strong transactional semantics for message sending and receiving to ensure data consistency.
- Teams seeking an alternative to cloud-based queues like AWS SQS with on-premise or self-hosted PostgreSQL.
- Scenarios needing configurable visibility timeouts to prevent duplicate message processing in distributed systems.
- Lightweight integration of message queuing in microservices or event-driven architectures using PostgreSQL as the backbone.
- Projects that prioritize leveraging PostgreSQL's durability, replication, and backup capabilities for messaging needs.
Not Ideal For
- Systems requiring ultra-high message throughput with sub-millisecond latency, where dedicated in-memory queues like Redis or Kafka would outperform.
- Projects needing advanced messaging features such as topic-based routing, message prioritization, or built-in dead-letter queues not provided by pgmq.
- Teams not using PostgreSQL and unwilling to adopt it solely for messaging, as it introduces an unnecessary database dependency.
Pros & Cons
Pros
Messages are sent and received within PostgreSQL transactions, ensuring atomic operations and data consistency, as highlighted in the Key Features.
Provides an interface similar to AWS SQS with simple operations, reducing the learning curve for developers already accustomed to cloud queues.
Leverages PostgreSQL's built-in durability, replication, and backup capabilities, eliminating the need for separate messaging infrastructure.
Minimal dependencies and easy setup allow quick deployment within existing PostgreSQL environments, as noted in the Lightweight Design feature.
Supports visibility timeouts to prevent duplicate message processing, essential for reliable distributed systems, as mentioned in the Key Features.
Cons
Lacks features like message routing, prioritization, or comprehensive monitoring tools compared to full-fledged message brokers such as RabbitMQ.
Using PostgreSQL for queuing can introduce latency and resource contention under high load, as it's not optimized solely for messaging workloads.
Requires PostgreSQL installation and management, which may not suit teams preferring database-agnostic or lightweight alternatives.
Open Source Alternative To
pgmq is an open-source alternative to the following products:
RSMQ (Redis Simple Message Queue) is a lightweight message queue system built on Redis, providing simple queuing functionality for distributed applications.
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
Frequently Asked Questions
Related Projects
riverFast and reliable background jobs in Go
pgBossQueueing jobs in Postgres from Node.js like a boss
BeanQueueBeanQueue, a lightweight Python task queue framework based on SQLAlchemy, PostgreSQL SKIP LOCKED queries and NOTIFY / LISTEN statements.
@andyrmitchell/pg-queueThe 'No Maintenance' Postgres Queue for Node.js
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.