Browsertrix Crawler
A standalone Docker container for high-fidelity, browser-based web archiving crawls using Puppeteer and Brave.
What is Browsertrix Crawler?
Browsertrix Crawler is a standalone browser-based high-fidelity crawling system that runs in a single Docker container. It automates web archiving by controlling Brave Browser windows with Puppeteer to capture web content exactly as rendered, solving the problem of inaccurate or incomplete archival crawls. It is designed for complex, customizable archiving tasks where fidelity to the original web experience is critical.
Digital archivists, researchers, librarians, and developers who need to preserve web content with high accuracy for projects like digital libraries, compliance, or historical records.
Developers choose Browsertrix Crawler because it offers containerized simplicity with browser-based precision, avoiding the limitations of traditional crawlers. Its use of real browsers via Puppeteer ensures superior fidelity, making it a reliable open-source tool for professional-grade web archiving.
Overview
Run a high-fidelity browser-based web archiving crawler in a single Docker container
Use Cases
Best For
- Creating high-fidelity archives of dynamic websites with JavaScript-heavy content
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
