Web Data Commons

0 stars0 forks0 contributors

Overview

Structured data extracted from Common Crawl. (Stable)

Related Projects

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub

Created

Links & Resources

Website

Included in

Web Archiving2.5k

ArchiveSpark

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

Stars160

Forks19

Last commit7 months ago

Archives Unleashed Toolkit

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Stars157

Forks34

Last commit5 months ago

Common Crawl Columnar Index

SQL-queryable index, with CDX info plus language classification. (Stable)

Stars0

Forks0

Last commit

Common Crawl Web Graph

A host or domain-level graph of the web, with ranking information. (Stable)

Stars0

Forks0

Last commit