Question 1

How does Durable Queue compare to Kafka for a simple Clojure app?

Accepted Answer

Durable Queue is better for lightweight, in-process persistence without Kafka's operational overhead, but it lacks Kafka's distributed scalability and cross-language support. Use it when you need simplicity and embeddability within a single Clojure process.

Question 2

How to configure Durable Queue for high performance?

Accepted Answer

Disable :fsync-put? and :fsync-take? to achieve up to 100k tasks/sec on SSDs, and use :fsync-threshold or :fsync-interval for batching. Be aware that this increases the risk of data loss on process crashes.

Question 3

Can Durable Queue handle task retries with backoff?

Accepted Answer

No, it only provides basic retry! functionality without advanced features like exponential backoff or dead-letter queues. You'll need to implement custom logic for sophisticated retry mechanisms.

Question 4

What happens if the disk gets corrupted while using Durable Queue?

Accepted Answer

Task dereferencing may throw IOExceptions, and the library doesn't automatically recover corrupted data. You should use the :complete? predicate on initialization to filter out bad tasks and implement manual cleanup.

Question 5

Is Durable Queue good for real-time event processing?

Accepted Answer

Not ideal, as disk I/O and fsync can introduce latency. It's better suited for background jobs or batch processing where milliseconds of delay are acceptable and durability is prioritized.

Question 6

How to monitor queue health in production with Durable Queue?

Accepted Answer

Use the stats function to get counts like :enqueued and :in-progress, and integrate these metrics into your monitoring system. There's no built-in alerting or GUI, so you'll need to build custom dashboards.

Durable Queue

What is Durable Queue?

Overview

Use Cases

Best For

Related Projects

Found a gem we're missing?

Not Ideal For

Pros & Cons

Pros

Cons

Open Source Alternative To

Frequently Asked Questions