archives
A Go library for creating/extracting archives, compressing/decompressing files, and walking virtual file systems across many formats.
What is archives?
mholt/archives is a cross-platform Go library that provides a unified API for working with a wide variety of archive and compression formats. It enables developers to handle archives, compressed files, and virtual file systems seamlessly, with deep integration into Go's standard io/fs interface.
Go developers who need to programmatically create, extract, or traverse archives and compressed files in their applications, such as those building file management tools, backup systems, or data processing pipelines.
Developers choose this library for its unified API that abstracts format-specific details, extensive format support including .zip, .tar, .rar, and .7z, and its ability to treat archives and directories uniformly as io/fs file systems for consistent, idiomatic Go code.
Overview
Cross-platform library to create & extract archives, compress & decompress files, and walk virtual file systems across various formats
Use Cases
Best For
- Building Go applications that need to handle multiple archive and compression formats with a single, consistent API.
- Creating virtual file systems in Go that treat directories, archives, and compressed files uniformly using the io/fs interface.
- Developing command-line tools or services that require streaming extraction or creation of archives without loading entire files into memory.
- Implementing web servers that serve or browse archive contents directly via http.FileServer.
- Writing data processing pipelines that need to decompress or compress streams on-the-fly with formats like gzip, zstd, or snappy.
- Adding archive manipulation features to Go applications, such as appending files to existing .tar or .zip archives without full recreation.
Not Ideal For
- Applications requiring creation or modification of .rar or .7z archives, as the library only supports reading these formats.
- High-performance servers needing efficient random access to large, compressed tar archives for HTTP range requests.
- Projects where minimal memory footprint is critical when walking tar archives, due to optimizations that increase memory usage.
- Simple use cases involving only a single archive format, where Go's standard library packages might be more straightforward.
Pros & Cons
Pros
Provides a single interface for creating, extracting, and walking archives across formats like .zip, .tar, .rar, and .7z, abstracting away format-specific details as highlighted in the philosophy.
Treats directories, archives, and compressed files uniformly as fs.FS file systems, enabling consistent file operations and compatibility with standard Go tools, as demonstrated with FileFS and DeepFS.
Supports a wide range of compression formats including gzip, zstd, and snappy, with features like multithreaded gzip and adjustable levels, as listed in the supported formats.
APIs are designed for streaming without loading entire files into memory, ideal for large archives or network operations, as emphasized in the features.
Allows registering additional archive or compression formats easily, making the library adaptable to custom needs, as noted in the extensibility feature.
Cons
Tar files suffer from slow random access due to their sequential nature, and optimizations for file system semantics increase memory usage, as explicitly warned in the README's important note.
.rar and .7z archives are supported only for reading, not writing, limiting use cases that require creating or modifying these formats, as stated in the supported archive formats.
The library does not support seeking in archives, which breaks HTTP range requests and complicates integration with http.FileServer without workarounds, as admitted in the HTTP server example.
Frequently Asked Questions
Related Projects
gopsutilpsutil for golang
gatusAutomated developer-oriented status page with alerting and incident support
gofakeitRandom fake data generator written in go
puregoA library for calling C functions from Go without Cgo
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.