Distributed Processing

4 projects

Showing 4 of 4 projects

A high-performance PHP import framework for distributed data processing with optimized memory consumption.

Java-based web archive deduplication tool that identifies duplicates and converts them to reference records in WARC files.

A splitable Hadoop InputFormat for processing concatenated GZIP files and web archive (*.warc.gz) data efficiently in distributed systems.

A web application that transcodes video files into multiple formats using the Golem decentralized computing network.

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.