smart_open

MITPythonv8.0.1

3.5k stars389 forks0 contributors

Overview

Utils for streaming large files (S3, HDFS, gzip, bz2...)

Related Projects

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub

8 days ago

CreatedSince 2015

Included in

Data Engineering8.5k

Auto-fetched 1 day ago

JuiceFS

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

a full featured file system for online data storage

Stars1,262

Forks116

Last commit15 days ago

Snakebite

A pure python HDFS client

Stars857

Forks213

Last commit4 years ago

SeaweedFS

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.

Stars30

Forks7

Last commit2 days ago