Web2Warc

MITScalav1.1

A customizable Scala crawler for creating personal web archives in WARC/CDX format.

GitHub

26 stars3 forks0 contributors

Overview

An easy-to-use and highly customizable crawler that enables you to create your own little Web archives (WARC/CDX)

Related Projects

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub

Last commit

8 years ago

CreatedSince 2016

Built With

s

sbt

S

Scala

Included in

Web Archiving2.5k

Auto-fetched 1 day ago

ArchiveBox

🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

Stars27,821

Forks1,546

Last commit6 days ago

monolith

⬛️ CLI tool and library for saving complete web pages as a single HTML file

Stars15,308

Forks463

Last commit1 month ago

DiskerNet

💾 dn - offline full-text search and archiving for your Chromium-based browser.

Stars3,904

Forks147

Last commit3 months ago

Heritrix Q&A

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

Stars3,249

Forks789

Last commit12 days ago