prefect
A workflow orchestration framework for building resilient data pipelines in Python.
What is prefect?
Prefect is a workflow orchestration framework for building resilient data pipelines in Python. It transforms scripts into production-ready workflows with features like scheduling, caching, retries, and event-based automations, helping data teams automate complex processes confidently.
Data engineers, data scientists, and developers who need to automate, monitor, and manage data pipelines and workflows in Python.
Prefect offers a simple Python-native API with built-in resilience features, making it easy to build and deploy dynamic data pipelines that recover from failures and adapt to changing conditions.
Overview
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Use Cases
Best For
- Automating ETL (Extract, Transform, Load) pipelines with error handling
- Scheduling recurring data processing jobs with retry logic
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
