Dagu
A local-first, single-binary workflow orchestration engine that runs declarative DAGs from laptop to distributed cluster.
What is Dagu?
Dagu is a local-first workflow orchestration engine that executes tasks defined as Directed Acyclic Graphs (DAGs) in YAML. It solves the problem of managing complex, dependent automation scripts by providing a declarative, file-based system that scales from a single machine to a distributed cluster. Unlike heavier orchestration tools, it requires no external databases or brokers, storing all state locally.
Developers, DevOps engineers, and data engineers who need to automate and orchestrate scripts, data pipelines, infrastructure tasks, or scheduled jobs without the overhead of complex distributed systems.
Developers choose Dagu for its extreme simplicity and portability—it's a single binary that works anywhere, supports a wide range of executors out-of-the-box, and offers both local and distributed execution modes without requiring additional infrastructure.
Overview
Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.
Use Cases
Best For
- Orchestrating ETL/ELT data pipelines with parallel and sequential steps
- Replacing fragile crontab setups with robust, observable scheduled jobs
- Automating infrastructure tasks across remote machines via SSH or containers
- Running batch processing workloads across a pool of labeled workers
- Managing and wrapping legacy scripts into reusable, monitored workflows
- Building air-gapped automation systems that require no external services
Not Ideal For
- Teams requiring real-time, event-driven workflow triggers from systems like Kafka or webhooks
- Organizations that need deep integration with external database-backed audit and compliance systems
- Projects where workflow logic must be dynamically generated or stored in a versioned database rather than static YAML files
Pros & Cons
Pros
Runs as a standalone executable with no external databases or brokers, making it incredibly portable and easy to deploy in any environment, including air-gapped systems, as highlighted in the philosophy section.
Includes 17 executors for common tasks like Docker, Kubernetes, SSH, HTTP, and S3, eliminating the need for plugins and reducing setup time, as detailed in the executors table.
Supports coordinator/worker architecture over gRPC with label-based routing, allowing horizontal scaling across machines without complex infrastructure, as explained in the distributed execution documentation.
Provides a web UI, Prometheus metrics, structured logging, and notifications, offering full visibility into workflow execution out-of-the-box, with examples like Slack and email integrations.
Cons
Relies on local files for state storage by default, which can limit performance in high-concurrency scenarios and complicates high-availability deployments without shared storage systems, as noted in the architecture.
No plugin system for adding custom executors; teams must rely on built-in types or modify source code, which can hinder adaptation to unique use cases not covered by the 17 provided executors.
Setting up the coordinator/worker mode involves multiple environment variables and gRPC setup with TLS options, which can be daunting and error-prone for users unfamiliar with distributed systems.
Frequently Asked Questions
Related Projects
go-taskflowA pure go General-purpose Task-parallel Programming Framework
Cadence-clientFramework for authoring workflows and activities running on top of the Cadence orchestration engine.
workflowThe type-safe, event-driven workflow orchestration library that scales with your business. Build robust, distributed workflows in Go with compile-time safety, automatic retries, and horizontal scaling out of the box. Integrate with your existing tech stack using adapters 🚀
go-dagA Go-based framework has been developed to oversee the execution of workflows delineated by directed acyclic graphs (DAGs).
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.