How do I set up Solid Queue with PostgreSQL?

Add Solid Queue to your Gemfile, run the install command, and configure a separate 'queue' database in database.yml with PostgreSQL details. Then, run db:prepare to load the schema, as outlined in the installation section.

Solid Queue vs Sidekiq: which is better for Rails?

Solid Queue excels in durability and Rails integration by using your SQL database, avoiding external Redis. However, Sidekiq typically offers higher throughput with in-memory processing, so choose based on whether you prioritize reliability over raw speed.

Can Solid Queue handle high-volume job processing?

Yes, but with caveats: it's designed for horizontal scaling and uses SKIP LOCKED for efficiency, but performance can suffer with older databases or complex concurrency controls, so benchmark for your workload.

How to retry failed jobs in Solid Queue?

Relies on Active Job's retry mechanisms; manually, find failed executions in the solid_queue_failed_executions table and use retry or discard methods, as described in the 'Failed jobs and retries' section.

What databases does Solid Queue support?

Officially supports PostgreSQL, MySQL, and SQLite, but for best performance, use PostgreSQL 9.5+ or MySQL 8+ with SKIP LOCKED. SQLite is suitable for smaller applications.

How to schedule a recurring task with Solid Queue?

Define tasks in config/recurring.yml with a class or command and a Fugit-based schedule (e.g., 'every day at 9am'), then ensure the scheduler process is running via bin/jobs or the Puma plugin.

Is Solid Queue good for microservices?

Not ideally, as it's tightly integrated with Ruby on Rails and Active Job. For non-Rails services or polyglot environments, external queuing systems like RabbitMQ might be more flexible.

Open-Awesome

solid_queue

MITRubyv1.5.0

A database-backed queuing backend for Active Job in Ruby on Rails, designed for simplicity and performance.

GitHub

2.5k stars242 forks0 contributors

What is solid_queue?

Solid Queue is a database-based queuing backend for Ruby on Rails' Active Job framework. It enables reliable, persistent background job processing by storing jobs in SQL databases like PostgreSQL, MySQL, or SQLite, eliminating the need for external queue services. It solves the problem of job durability and scalability while maintaining tight integration with Rails' conventions and tooling.

Target Audience

Ruby on Rails developers who need a reliable, database-backed background job system without external dependencies, particularly those deploying on platforms where external Redis or RabbitMQ services are undesirable or complex to manage.

Value Proposition

Developers choose Solid Queue for its simplicity, durability, and seamless Rails integration. It reduces infrastructure complexity by leveraging the existing database, provides transactional integrity for job enqueuing, and offers features like concurrency controls and recurring tasks out of the box.

Overview

Database-backed Active Job backend

Use Cases

Best For

Rails applications needing durable, database-backed job queues without external services
Teams wanting to avoid Redis or RabbitMQ for background job processing
Implementing concurrency-limited job execution (e.g., rate-limiting per account)
Scheduling recurring tasks (cron jobs) within a Rails application
Applications requiring transactional integrity between database writes and job enqueuing
Scaling background job processing horizontally across multiple servers

Not Ideal For

Applications requiring ultra-low latency job processing where in-memory queues like Redis are critical
Projects not using Ruby on Rails or Active Job, as it's tightly coupled to Rails conventions
Teams already invested in external queuing services (e.g., RabbitMQ, AWS SQS) with existing tooling and monitoring
Environments with limited database resources where adding queue tables could impact primary app performance

Pros & Cons

Pros

Database Durability & Integrity

Jobs are stored in SQL tables with ACID compliance, ensuring persistence through crashes and allowing transactional enqueuing within database writes, as noted in the 'Jobs and transactional integrity' section.

Seamless Rails Integration

Configured by default in Rails 8 and works as an Active Job backend, reducing setup overhead and leveraging Rails conventions like migrations and configuration files.

Advanced Concurrency Controls

Allows limiting simultaneous job executions by key, group, or arguments, with configurable conflict handling (block or discard), useful for scenarios like rate-limiting per account.

Built-in Recurring Tasks

Supports cron-like scheduling via a YAML file or dynamic runtime scheduling, eliminating the need for external cron jobs or separate schedulers.

Cons

Performance Overhead

Concurrency controls introduce significant overhead, and the README advises against using them for simple throttling, recommending limited workers per queue instead, which can complicate scaling.

Database Version Dependencies

Optimal performance requires databases with SKIP LOCKED support (e.g., MySQL 8+, PostgreSQL 9.5+); without it, lock contention can degrade performance, limiting older setups.

Complex Multi-Database Setup

Installation involves configuring separate database connections and schemas (e.g., queue database in database.yml), which adds complexity and potential for errors compared to drop-in solutions.

Frequently Asked Questions

Related Projects

devise

Flexible authentication solution for Rails with Warden.

Stars24,343

Forks5,480

Last commit1 month ago

spree

Open Source eCommerce Platform for B2B, Marketplace, and Enterprise. REST API, TypeScript SDK, and production-ready Next.js storefront. Self-host it. Own your stack. No vendor lock-in. Zero platform fees.

Stars15,570

Forks5,288

Last commit14 hours ago

activeadmin

The administration framework for Ruby on Rails applications.

Stars9,704

Forks3,324

Last commit17 hours ago

Carrierwave

Classier solution for file uploads for Rails, Sinatra and other Ruby web frameworks

Stars8,783

Forks1,650