s5cmd
A parallel S3 and local filesystem execution tool optimized for high-speed object storage operations.
What is s5cmd?
s5cmd is a parallel S3 and local filesystem execution tool built for high-performance object storage operations. It enables fast uploads, downloads, deletions, and synchronization of files between local storage and cloud services like Amazon S3 and Google Cloud Storage. The tool solves the problem of slow bulk file transfers by leveraging concurrent processing and efficient batching.
Developers, data engineers, and system administrators who regularly work with large volumes of data in S3 or compatible object storage and need fast, scriptable command-line tools for data management.
Developers choose s5cmd for its exceptional speed—up to 32x faster than s3cmd for uploads—and its parallel execution model that saturates high-bandwidth links. It offers a rich feature set including wildcard operations, batch command execution, and sync capabilities, all while being lightweight and easy to integrate into automated workflows.
Overview
Parallel S3 and local filesystem execution tool.
Use Cases
Best For
- Bulk uploading or downloading thousands of files to/from S3
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
