glow
A distributed computation system written in Go for parallel and cluster processing, similar to Hadoop MapReduce and Spark.
What is glow?
Glow is a distributed computation system written in pure Go that enables parallel and distributed data processing. It provides a library for building data pipelines that can run across multiple threads or scale to clusters of machines, solving the problem of efficiently processing large datasets without complex infrastructure setup.
Go developers and data engineers who need to process large datasets in parallel or across distributed systems, particularly those looking for Hadoop/Spark alternatives in the Go ecosystem.
Developers choose Glow for its simplicity, minimal resource footprint, and pure Go implementation that eliminates external dependencies while providing familiar MapReduce-style operations with easy cluster deployment.
Overview
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
Use Cases
Best For
- Processing large text files or log data in parallel
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
