Warc Files
3 projects
Showing 3 of 3 projects
node-warcJavaScript
A Node.js library for parsing and creating Web ARChive (WARC) files with support for Chrome, Puppeteer, and Electron.
#stream-processing#web-archives#warc-files
Stars104
Forks22
Last commit1 year ago
Common Crawl Jupyter notebooksJupyter Notebook
A collection of Jupyter notebooks for analyzing Common Crawl web archive data using columnar indexes and webgraph datasets.
#warc-files#data-science#web-archive-analysis
Stars66
Forks11
Last commit
WarcworkerPython
A dockerized, queued web archiver using Chrome headless to create high-fidelity WARC files from URLs.
#squidwarc#high-fidelity-preservation#warc-files
Stars62
Forks9
Last commit1 year ago
Related Tags
Community-curated · Updated weekly · 100% open source
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.