A simple, persistent, disk-backed key-value store for Go with performance-enhancing abstractions.
Diskv is a disk-backed key-value store written in Go that provides a simple API for storing arbitrary data on a filesystem. It solves the need for persistent, efficient local storage by adding performance-enhancing layers like caching, ordering, and compression while keeping data directly accessible via standard filesystem tools.
Go developers who need a lightweight, persistent key-value store for applications requiring local data storage without external dependencies like databases.
Developers choose Diskv for its simplicity, transparency, and performance—it offers a clean API, ensures data is always directly accessible on disk, and includes optional optimizations like caching and compression without complicating the core design.
A disk-backed key-value store.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
With just Write, Read, and Erase methods, diskv offers a straightforward interface for storing arbitrary byte data, as demonstrated in the minimal setup example in the README.
TransformFunc and AdvancedTransform allow keys to be mapped to nested directories, enabling customizable file layouts without complicating the core API, shown in the advanced examples.
Layers like in-memory caching, ordering via indexes, and compression using stdlib algorithms can be added as needed, improving efficiency while maintaining simplicity.
ReadStream and WriteStream methods support efficient handling of very large files, preventing memory issues in data-intensive applications, as highlighted in the README.
The README admits that TransformFunc must avoid key subsets that resolve to directories, which can cause errors and is acknowledged as an issue to be fixed in a future version.
Beyond a basic mutex for the cache, diskv lacks built-in transactions or advanced concurrency mechanisms, which could lead to race conditions in write-heavy environments.
As a disk-backed store, operations are tied to filesystem speed, and even with caching, latency may be higher than in-memory alternatives for certain workloads, with benchmarking noted as a future plan.