Deduplication
Showing 19 of 19 projects
A fast, secure, and efficient cross-platform backup program with deduplication and multiple storage backends.
Deduplicating archiver with compression and authenticated encryption.
A cross-platform GUI tool to find duplicate files on your computer, written in Python with a Qt interface.
A lock-free deduplication cloud backup tool enabling multiple computers to back up concurrently to the same storage.
A fast, encrypted, and deduplicated backup tool written in Rust, compatible with restic repositories.
A self-hostable multi-tenant Nix Binary Cache server with global deduplication and S3-compatible storage.
A Go library of probabilistic data structures for processing continuous, unbounded data streams.
A zero-details, privacy-focused in-app file system that encrypts files and directories into a secure repository.
A CLI tool that applies Git-like version control to cloud storage, enabling distributed, decentralized, and deduplicated data repositories.
A fast, memory-optimized C tool to remove duplicates from massive wordlists while preserving order, designed for password cracking.
A JavaScript library for normalizing URLs by adding protocols, removing duplicates, sorting parameters, and stripping unnecessary components.
A Rust wrapper for restic that simplifies configuration and automation of secure, deduplicated backups.
A faster, multi-threaded, feature-rich alternative to the rli utility for line removal, deduplication, and frequency analysis on large text files.
A GIMP plugin for optimizing tile maps and tile sets by deduplicating tiles and providing in-workflow feedback.
A lightweight JavaScript utility for removing duplicate values from arrays, supporting both primitive and complex types.
A modern, universal HTTP client built on native fetch with zero dependencies, featuring request orchestration, retries, rate limiting, and hedging.
A pure Go toolkit for data engineering and classic machine learning with zero external dependencies.
Java-based web archive deduplication tool that identifies duplicates and converts them to reference records in WARC files.
A Docker container that automates encrypted backups to S3-compatible storage using Duplicacy CLI on a cron schedule.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.