Webarchive
3 projects
Showing 3 of 3 projects
Go Get CrawlGo
A Go tool and library for downloading URLs and files from Common Crawl and Wayback Machine web archives.
#crawler#go-library#wayback-machine
Stars179
Forks17
Last commit1 year ago
ArchiveSparkScala
An Apache Spark framework for efficient data processing, extraction, and derivation from web archives and archival collections.
#data-lineage#apache-spark#web-archives
Stars160
Forks19
Last commit7 months ago
FastWARCRust
A collection of robust and fast Python tools for parsing, extracting, and analyzing web archive data, including a high-performance WARC parser.
#cython#batch-processing#content-extraction
Stars140
Forks18
Last commit11 hours ago
Community-curated · Updated weekly · 100% open source
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.