How does Scheduler compare to Quartz for distributed scheduling?

Scheduler is specifically built for distributed, multi-datacenter environments with Kafka and Cassandra, emphasizing fault tolerance and task ordering, whereas Quartz is more general-purpose and may require additional configuration for similar reliability in distributed setups.

How to configure Scheduler for multi-datacenter deployment?

Refer to the Operations Guide in the doc/ folder for detailed setup; it involves deploying on multiple servers across data centers, configuring Kafka and Cassandra for redundancy, and ensuring task consistency as per the design documentation.

What are the Kafka and Cassandra requirements for Scheduler?

Scheduler relies on Kafka for task buffering and Cassandra for task persistence; you need running instances of both, with appropriate clustering for fault tolerance, as mentioned in the README and design docs.

Is Scheduler suitable for real-time task execution?

It's designed for scheduled tasks at arbitrary times, not necessarily real-time; distributed buffering with Kafka might introduce latency, so it's better for reliable, time-based scheduling rather than low-latency real-time processing.

How does Scheduler handle task failures and retries?

Built with fault tolerance in mind, it likely includes retry mechanisms via Cassandra persistence and Kafka buffering, but specific details should be checked in the user and operations guides for error handling configurations.

Can I use Scheduler without Kafka or Cassandra?

No, it's tightly coupled with Kafka and Cassandra as core dependencies for buffering and persistence, so alternative backends aren't supported without significant modification, as per the README.

Open-Awesome

Scheduler

BSD-3-ClauseScala

A fault-tolerant Scala library for scheduling arbitrary code to run at arbitrary times across distributed systems.

GitHub

213 stars37 forks0 contributors

What is Scheduler?

Scheduler is a Scala library that enables scheduling arbitrary code to run at arbitrary times in distributed systems. It solves the problem of reliable, fault-tolerant task scheduling across multiple servers and data centers, maintaining task ordering despite infrastructure changes.

Target Audience

Scala developers building distributed services that require reliable, production-grade task scheduling across redundant, scalable deployments.

Value Proposition

Developers choose Scheduler for its proven production reliability at PagerDuty, built-in fault tolerance for multi-datacenter environments, and ability to maintain task ordering in distributed setups.

Overview

A Scala library for scheduling arbitrary code to run at an arbitrary time.

Use Cases

Best For

Scheduling tasks in Scala microservices with high reliability requirements
Building distributed cron-like systems across multiple data centers
Ensuring task ordering in redundant, fault-tolerant architectures
Managing job execution in scalable, multi-server deployments
Implementing production-grade scheduling with Kafka and Cassandra backends
Handling infrastructure changes while maintaining scheduled task consistency

Not Ideal For

Teams running simple, single-server applications without distributed infrastructure needs
Projects not already using Kafka and Cassandra, as it imposes these heavy dependencies
Developers seeking a lightweight, embeddable scheduler with minimal setup complexity

Pros & Cons

Pros

Production-Proven Reliability

Used in production at PagerDuty and designed for fault tolerance in multi-datacenter environments, as emphasized in the README.

Distributed and Scalable

Runs across many servers for redundancy and scaling, built from the ground up for multi-datacenter deployments.

Maintains Task Ordering

Ensures a sense of task ordering despite distributed execution, a key feature highlighted in the introduction.

Flexible Code Scheduling

Allows scheduling arbitrary Scala code to run at arbitrary, user-defined times, providing high customization.

Cons

Complex Setup and Dependencies

Requires Kafka for task buffering and Cassandra for persistence, adding infrastructure overhead, and setup is more complicated than simple library inclusion, as noted in the README.

Scala and JVM Lock-in

Specifically designed for Scala services, limiting adoption in polyglot environments or non-JVM language projects.

Documentation Scattered Across Guides

Most documentation is in specific guides in doc/, which can make initial onboarding harder compared to integrated, centralized documentation.

Frequently Asked Questions

Related Projects

CCM

Apache Cassandra® Cluster Manager (CCM) – easily create and destroy clusters on localhost

Stars1,234

Forks309

Last commit4 months ago

Netflix-Priam

Co-Process for backup/recovery, Token Management, and Centralized Configuration management for Cassandra.

Stars1,039

Forks293

Last commit2 months ago

Twissandra

Twissandra is an example project, created to learn and demonstrate how to use Cassandra. Running the project will present a website that has similar functionality to Twitter.

Stars798

Forks161

Last commit6 years ago

cmb

This project is no longer actively supported. It is made available as read-only. A highly available, horizontally scalable queuing and notification service compatible with AWS SQS and SNS

Stars274

Forks49

Last commit7 years ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub