Hamilton
A lightweight Python library for creating portable, expressive, and testable data transformation DAGs with built-in lineage and metadata.
What is Hamilton?
Apache Hamilton is a Python library that helps data scientists and engineers define, manage, and execute data transformation workflows as directed acyclic graphs (DAGs). It structures code into modular functions that automatically build a DAG, providing built-in lineage tracking, metadata capture, and data validation to improve collaboration and maintainability.
Data scientists, data engineers, and ML engineers building ETL pipelines, ML workflows, LLM applications, or any data transformation system in Python who need structure, testability, and portability.
Developers choose Apache Hamilton for its unique combination of portability (runs anywhere Python does), expressiveness through function modifiers, and built-in software engineering practices like modularity and validation, which reduce technical debt and ease the transition from development to production.
Overview
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
Use Cases
Best For
- Building modular and testable ETL pipelines in Python
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
