Airflow
A platform to programmatically author, schedule, and monitor workflows as code.
What is Airflow?
Apache Airflow is an open-source workflow orchestration platform that allows users to programmatically author, schedule, and monitor complex data pipelines and computational workflows. It solves the problem of managing dependencies, scheduling, and execution of tasks in a scalable and maintainable way by defining workflows as code.
Data engineers, DevOps professionals, and developers who need to build, schedule, and monitor batch-oriented data pipelines, ETL processes, and other automated workflows.
Developers choose Airflow for its code-based workflow definition, rich extensibility, and powerful UI for monitoring, which together provide greater control, maintainability, and collaboration compared to static configuration tools.
Overview
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Use Cases
Best For
- Orchestrating ETL (Extract, Transform, Load) data pipelines
- Scheduling and monitoring batch data processing jobs
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
