Pathway
A Python ETL framework for stream processing, real-time analytics, and building live LLM/RAG pipelines, powered by a scalable Rust engine.
What is Pathway?
Pathway is a Python ETL and stream processing framework designed for building real-time data pipelines, analytics, and live AI applications like LLM and RAG workflows. It solves the challenge of unifying batch and streaming computation with a simple Python API, while delivering high performance through a scalable Rust engine that handles incremental processing and distributed workloads.
Data engineers and ML engineers building real-time ETL pipelines, streaming analytics, or live AI/LLM applications who want Python simplicity with production-scale performance.
Developers choose Pathway for its unique combination of a Python-friendly API and a high-performance Rust backend, enabling them to write once and run anywhere—from local development to distributed cloud deployments—without sacrificing speed or scalability.
Overview
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Use Cases
Best For
- Building real-time ETL pipelines from Kafka or other streaming sources
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
