Alluxio
A distributed caching platform that bridges computation frameworks and storage systems for large-scale analytics and ML workloads.
What is Alluxio?
Alluxio is a distributed caching platform that orchestrates data access between computation frameworks and storage systems. It solves the problem of slow data access in large-scale analytics and machine learning workloads by providing a virtual distributed file system with intelligent caching. Originally developed as Tachyon at UC Berkeley's AMPLab, it accelerates data processing for frameworks like Spark, Presto, and Trino.
Data engineers and platform teams building large-scale analytics or machine learning pipelines in cloud environments, particularly those using computation frameworks like Spark, Presto, or Trino with multiple storage backends.
Developers choose Alluxio because it provides a unified interface to diverse storage systems while dramatically accelerating data access through distributed in-memory caching. Its architecture separates compute from storage, enabling consistent high performance across hybrid and multi-cloud environments.
Overview
Alluxio, data orchestration for analytics and machine learning in the cloud
Use Cases
Best For
- Accelerating Spark or Presto queries on cloud object storage
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
