A FUSE-based file system that mounts Amazon S3 buckets locally with distributed cache synchronization via SNS notifications.
YAS3FS is a FUSE-based file system that mounts Amazon S3 buckets as local directories, enabling applications to interact with cloud storage as if it were a local file system. It solves the problem of accessing S3 data with low latency by caching files locally and synchronizing changes across multiple nodes using Amazon SNS notifications.
Developers and system administrators working in AWS environments who need to access S3 data as a mounted file system, especially those deploying multi-node clusters requiring synchronized cache invalidation.
YAS3FS offers a performant, distributed alternative to basic S3 mounting tools by implementing a smart cache with SNS-based synchronization, parallel transfers for large files, and support for streaming—all while maintaining compatibility with standard file operations.
YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Leverages Amazon SNS for real-time cache invalidation across multiple nodes, enabling efficient shared access to S3 data in clusters, as detailed in the README's architecture.
Uses multi-part uploads and downloads to maximize throughput for large files, with configurable settings for part size and parallelism.
Buffering allows file access during downloads, enabling streaming use cases like media playback directly from S3 without full file retrieval.
Provides signed URLs via xattr for secure, time-limited access to files, simplifying integration with applications that need temporary S3 object access.
Relies on Python 2.6 and older libraries like boto, with the README warning that pip installation is no longer supported, indicating potential compatibility and security issues.
Requires FUSE installation, AWS service setup (SNS, SQS), and OS-specific steps, making deployment non-trivial and error-prone for beginners or small teams.
Last major update was in 2014, suggesting it may lack support for modern AWS features, security patches, or Python 3, risking obsolescence in evolving cloud environments.