Heritrix Walkthrough
MITShell
A virtual machine and walkthrough for setting up and using the Heritrix web crawler for web archiving.
10 stars1 forks0 contributors
Overview
Heritrix Walkthrough provides a pre-configured virtual machine and step-by-step guide for running Heritrix, the open-source, extensible web crawler designed for web archiving. This project simplifies the setup process, allowing researchers and archivists to quickly begin capturing web content.
Key Features
- Pre-configured Virtual Machine — Includes Ubuntu 14.04, Java 7, and Heritrix 3.2.0 for immediate use.
- Step-by-Step Walkthrough — Detailed instructions in the
walkthroughsubdirectory guide users from setup to operation. - Vagrant-Based Deployment — Uses Vagrant and VirtualBox for easy, reproducible environment provisioning.
- Minimal Host Requirements — The VM uses 2GB of RAM, making it accessible on most modern machines.
Philosophy
Heritrix Walkthrough aims to lower the barrier to entry for web archiving by providing a straightforward, reproducible setup that lets users focus on capturing content rather than configuring software.
Related Projects
Community-curated · Updated weekly · 100% open source
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
