A serverless proxy for Spark clusters that provides a functional programming framework and deployment model for Spark applications.
Hydrosphere Mist is a serverless proxy for Apache Spark clusters that provides a functional programming framework for deploying Spark applications. It enables Spark Function as a Service by abstracting cluster management and offering a unified API layer for building enterprise solutions and microservices on top of Spark functions.
Data engineers and developers building scalable data processing pipelines and microservices using Apache Spark who want to simplify deployment and management.
Developers choose Mist for its typesafe programming model, multi-protocol API support, and ability to manage Spark sessions across various cluster backends (EMR, Hortonworks, Cloudera, etc.) without dealing with infrastructure complexity.
Serverless proxy for Spark cluster
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Clearly defines inputs and outputs for every Spark job, reducing runtime errors and improving code reliability, as emphasized in the README's features list.
Offers REST HTTP and messaging APIs (MQTT, Kafka) for Scala and Python Spark jobs, enabling flexible integration into microservices and event-driven architectures.
Supports on-demand Spark sessions across various backends like EMR, Hortonworks, and Cloudera, abstracting infrastructure management complexities.
Enables Spark Function as a Service, allowing deployment of Spark applications without manual cluster lifecycle management, aligning with the core value proposition.
Multi-cluster mode for seamless on-demand provisioning and autoscaling is marked as pending in the README, limiting dynamic scalability in production.
Introduces a proxy layer that can increase latency and setup complexity compared to direct Spark usage, especially for simple deployments.
Requires configuration with multiple cluster types and external documentation, which may demand significant expertise for initial setup and integration.