A fast, cross-platform, multi-threaded compression and decompression CLI tool written in Rust.
Crabz is a command-line compression and decompression tool written in Rust that provides parallel processing capabilities for handling large files efficiently. It serves as a faster, cross-platform alternative to pigz, supporting multiple compression formats including Gzip, BGZF, and Mgzip. The tool addresses the need for high-speed data compression in workflows where performance and format flexibility are critical.
System administrators, data engineers, and developers working with large datasets who need efficient compression and decompression tools for data processing pipelines or storage optimization.
Developers choose Crabz for its superior multi-threaded performance, cross-platform support, and additional format options compared to pigz. Its Rust implementation offers memory safety advantages while maintaining competitive speed through optimized backends like zlib-ng.
Like pigz, but rust
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Benchmarks show crabz with zlib-ng backend compresses up to 50% faster than pigz, making it ideal for large datasets like logs or genomic data.
Explicitly designed to run on Linux, macOS, and Windows, providing consistent performance across different operating systems without recompilation.
Supports zlib, zlib-ng, and a pure Rust backend, allowing users to choose between speed, compatibility, and memory safety as evidenced in benchmark tables.
Handles BGZF and Mgzip formats with optimized decompression, outperforming pigz in block data benchmarks, useful for bioinformatics workflows.
The README lists this as a TODO, requiring users to manually specify formats with -f flag, which can be inconvenient for mixed-format batches.
Thread pinning requires specifying physical cores with -P flag, which can conflict with other processes and needs careful management in multi-instance environments.
The -I flag deletes the input file upon completion, posing a data loss risk if not used cautiously, with no built-in backup option.
The -d flag is noted to 'may change in future releases', indicating potential breaking changes that could disrupt existing scripts.
crabz is an open-source alternative to the following products: