gowarcserver
Apache-2.0Gov0.4.4
A BadgerDB-based server for indexing and serving WARC file contents with CDX capture index support.
17 stars2 forks0 contributors
Overview
gowarcserver is a specialized tool for indexing and serving the contents of WARC (Web ARChive) files. It provides a capture index (CDX) server that enables efficient querying and retrieval of archived web resources, making it valuable for digital preservation and web archiving workflows.
Key Features
- WARC Indexing — Creates searchable indexes from WARC files for fast record retrieval
- CDX Server — Implements the CDX (Capture Index) format for standardized access to archived web captures
- BadgerDB Storage — Uses BadgerDB as the underlying key-value store for efficient index persistence
- TiKV Support — Optional TiKV backend for distributed storage scenarios
- Web Archive Serving — Serves individual WARC records and resources from archived web content
Philosophy
The project focuses on providing a reliable, performant indexing solution for web archives with support for both embedded and distributed storage backends.
Related Projects
Community-curated · Updated weekly · 100% open source
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
