A free, open-source alternative to Spark UI and Spark History Server with enhanced CPU and memory metrics visualizations.
Delight is an open-source monitoring dashboard for Apache Spark applications that provides enhanced CPU and memory metrics visualizations. It serves as a free alternative to the standard Spark UI and Spark History Server, offering deeper insights into executor performance and resource utilization to help identify inefficiencies like slow shuffles or memory waste.
Data engineers and Spark developers who need to monitor and optimize the performance of their Spark applications across various platforms, including cloud, on-premise, or managed services like Databricks and EMR.
Developers choose Delight because it offers advanced, actionable metrics beyond the standard Spark UI for free, with easy setup and cross-platform compatibility, helping them tune configurations and reduce compute costs without maintaining their own Spark History Server.
A Spark UI and Spark History Server alternative with CPU and Memory metrics! Delight is free, cross-platform, and open-source.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Visualizes executor core usage broken down by categories and aligned with Spark job timelines, making it easy to correlate performance issues like slow shuffles, as shown in the before/after comparison images.
Reports peak memory usage per executor segmented into JVM, Python, and other processes, helping tune container memory sizes to avoid waste, demonstrated in the memory graph screenshot.
Entirely free of charge and works on any Spark platform—open-source, commercial, cloud, or on-premise—without vendor lock-in, as stated in the compatibility section.
Provides access to the Spark UI without setting up and maintaining a separate Spark History Server, simplifying deployment for users.
Delight was shut down on May 31st, 2024, with all functionalities integrated into NetApp's Ocean for Apache Spark, rendering it obsolete for new implementations and leaving existing users to migrate.
As per the FAQ, Delight is only accessible after app completion, making it unsuitable for monitoring long-running applications like interactive clusters or streaming jobs.
Logs are automatically deleted 30 days after collection, with a cap of 10,000 apps per customer, which can hinder long-term performance analysis and debugging.
Requires a Google account for sign-in with no alternative login methods, potentially excluding teams in organizations without Google integration.
Delight is an open-source alternative to the following products: