Disco
A distributed map-reduce framework for parallel computations over large datasets on unreliable computer clusters.
What is Disco?
Disco is a distributed map-reduce framework for parallel computations over large datasets on unreliable computer clusters. It abstracts away technical complexities like communication protocols, load balancing, and fault tolerance, allowing developers to process massive data with minimal code.
Data engineers and scientists who need to analyze large datasets across distributed systems without managing low-level cluster infrastructure.
Developers choose Disco for its simplicity in writing distributed jobs, strong fault tolerance, and seamless integration with Python data science tools, making it easier to focus on data analysis rather than distributed systems engineering.
Overview
a Map/Reduce framework for distributed computing
Use Cases
Best For
- Processing and analyzing terabyte-scale datasets across distributed clusters
- Implementing fault-tolerant map-reduce jobs without managing infrastructure
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
