Data Pipelines
Showing 11 of 83 projects
A Node.js Transform stream that processes data chunks concurrently with configurable max concurrency.
A GitHub Action that executes parameterized Jupyter notebooks using Papermill and uploads outputs as artifacts.
A distributed batch data processing framework that handles scalability and intermediate storage, letting users focus on transforms and quality control.
A Node.js library that pipes queued streams sequentially to preserve content order.
Lightweight DataFrame validation decorators for pandas, Polars, Modin, and PyArrow with no custom types required.
A Python reactive stream programming framework for concurrent asynchronous data transformations.
A dynamic framework for processing high-volume data streams with subsecond pipeline instantiation and modification latency.
A Go library providing generic functions for building concurrent processing pipelines with fan-out/fan-in patterns.
Example code for large-scale metrics analysis using Ruby with AWS services like EMR and Redshift.
Store, version, edit, and execute Jupyter notebooks via REST APIs for integration into data pipelines.
A C++ reactive stream programming framework for live data computation with concurrent asynchronous operations.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.