Livy
A REST interface for interacting with Apache Spark from anywhere, enabling remote code execution and job submissions.
What is Livy?
Apache Livy is an open-source REST service that provides a web interface for interacting with Apache Spark clusters remotely. It allows users to execute Spark code snippets, submit batch jobs, and manage Spark sessions via HTTP calls, eliminating the need for direct cluster access. Livy acts as a gateway that simplifies Spark integration into applications, scripts, or tools that can communicate over REST.
Data engineers, data scientists, and developers who need to programmatically interact with Apache Spark clusters from external systems, web applications, or automated pipelines.
Livy offers a standardized, language-agnostic REST API for Spark, enabling remote job submissions and interactive sessions without modifying existing Spark code. Its multi-user support and impersonation features make it suitable for shared, production environments where secure, scalable Spark access is required.
Overview
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
Use Cases
Best For
- Submitting Spark batch jobs from web applications or microservices
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
