A distributed, multi-tenant gateway providing serverless SQL on data warehouses and lakehouses.
Apache Kyuubi is a distributed and multi-tenant gateway that provides serverless SQL on data warehouses and lakehouses. It offers a pure SQL interface through Thrift JDBC/ODBC, allowing users to manipulate large-scale data with Spark SQL engines without needing deep Spark expertise. The project solves the challenges of resource isolation, data security, and high client concurrency in big data environments.
System administrators (Spark experts) responsible for deployment and tuning, and end-users (e.g., data analysts, BI professionals) who focus on business data and use SQL or tools like Apache Superset for visualization.
Developers choose Kyuubi for its multi-tenant architecture that enables resource isolation and high concurrency, its compatibility with HiveServer2 for easy workload migration, and its vision as a unified data lake management platform supporting both ETL and analytics.
Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Decouples server and engine for resource isolation and high client concurrency, solving the limitations of Spark Thrift Server's single-tenant model.
Provides seamless migration from Hive to Spark SQL via Thrift JDBC/ODBC, enabling better performance with minimal changes to existing workloads.
Supports flexible engine deployment on Hadoop YARN and Kubernetes, allowing serverless SQL with dynamic resource management.
Acts as a single interface for ETL and BI analytics on data lakes, reducing data duplication and simplifying access through logical views and multiple catalogs.
Currently limited to Spark SQL engines, with other engines mentioned as future possibilities, reducing flexibility for heterogeneous workloads.
Requires Spark experts for deployment, configuration, and tuning, as highlighted in the target users, increasing operational overhead and barrier to entry.
Some advertised features like SQL Standard Authorization for DataLake are listed as 'coming', indicating gaps in current security and functionality.
Apache Kyuubi is an open-source alternative to the following products: