Hamilton
A Python library for defining portable, modular, and testable data transformation DAGs with built-in lineage and metadata.
What is Hamilton?
Apache Hamilton is a Python library that helps data scientists and engineers define, manage, and execute data transformation workflows as directed acyclic graphs (DAGs). It structures code into modular, testable functions that automatically build a DAG, solving problems of code maintainability, collaboration, and portability across environments. The library includes built-in features for lineage tracking, metadata capture, and data validation.
Data scientists, data engineers, and ML engineers building ETL pipelines, ML workflows, LLM applications, or BI dashboards in Python who need scalable, maintainable data transformation code.
Developers choose Apache Hamilton for its unique combination of portability (runs anywhere Python does), modularity (encourages clean, testable code), and built-in observability (lineage and metadata tracking). Its function modifiers and separation of definition from execution reduce redundancy and ease the transition from development to production.
Overview
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
Use Cases
Best For
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.