Web Archiving
Showing 36 of 80 projects
Open-source self-hosted web archiving tool that saves websites in multiple durable formats like HTML, PDF, and WARC.
CLI tool and library for saving complete web pages as a single, self-contained HTML file.
A simple, self-hosted bookmark manager with offline archiving, built as a Pocket alternative.
Offline full-text search and archiving tool for Chromium-based browsers that saves and indexes every page you visit.
Offline full-text search and archiving tool for Chromium-based browsers that saves and indexes every page you visit.
An open-source, extensible, web-scale, archival-quality web crawler from the Internet Archive.
An open-source, extensible, web-scale, archival-quality web crawler from the Internet Archive.
A curated list of resources, tools, and services for web archiving, from acquisition and replay to analysis and community.
A curated list of resources, tools, and services for web archiving, from acquisition and replay to analysis and community.
A privacy-focused web archiving tool with an IM-style interface that captures pages to multiple archival services.
Core Python Web Archiving Toolkit for replay and recording of web archives
A preconfigured web crawler for backing up websites, producing WARC files with a live dashboard and dynamic ignore patterns.
A browser extension and desktop app for interactive, high-fidelity web archiving directly in the browser.
A standalone Docker container for high-fidelity, browser-based web archiving crawls using Puppeteer and Brave.
A Python tool to automatically archive web content (videos, images, social media) from Google Sheets and other sources in a secure, verifiable way.
A set of Python tools for downloading and preserving wikis, including MediaWiki wikis and Wikimedia projects.
A distributed and persistent web archive replay system that uses IPFS to store and serve WARC files.
A Python package and CLI tool for interacting with the Wayback Machine's Save, CDX, and Availability APIs.
Legacy web archive replay engine for accessing historical web content from WARC files.
A comprehensive curated list of open-source and hosted tools for monitoring and detecting changes on websites.
Streaming WARC/ARC library for fast web archive IO
A Tool To Push Web Resources Into Web Archives
WarcDB is an SQLite-based file format that makes web crawl data easier to share and query.
A graphical desktop application that simplifies web archiving by providing a one-click interface to preserve and replay web pages using Heritrix and OpenWayback.
A Go package and CLI tool that saves web pages as single HTML files with all assets embedded.
A high-fidelity, browser-based web archiving library and CLI for capturing single web pages with provenance.
A Go tool and library for downloading URLs and files from Common Crawl and Wayback Machine web archives.
Python command-line tools and libraries for handling, validating, and converting WARC and ARC web archive files.
A high-fidelity, user-scriptable archival web crawler using Chrome/Chromium to preserve JavaScript-rendered content.
An Apache Spark framework for efficient data processing, extraction, and derivation from web archives and archival collections.
A web application for searching, browsing, and analyzing archived web content (ARC/WARC files) with a Solr backend.
A toolkit for indexing and exploring web archive content from ARC and WARC files using OpenSearch/Elasticsearch.
A toolkit for indexing and exploring web archive content from ARC and WARC files using OpenSearch/Elasticsearch.
A curated list of software, literature, and resources for the Memento protocol (RFC7089) enabling time-based access to archived web content.
A Node.js library for parsing and creating Web ARChive (WARC) files with support for Chrome, Puppeteer, and Electron.
A compilation of research materials on data resilience, interactivity, and related topics for the Data Together community.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.