TD-Spark
A Java project that reads from and writes to TDengine using Apache Spark for data processing.
Overview
TD-Spark is a Java-based integration project that enables bidirectional data flow between TDengine and Apache Spark. It allows users to leverage Spark's distributed computing capabilities to process data stored in TDengine, making it valuable for big data analytics and ETL workflows involving time-series data.
Key Features
- TDengine Integration — Provides connectors to read data from TDengine into Spark DataFrames
- Spark Processing — Enables distributed data processing using Apache Spark's computational engine
- Bidirectional Data Flow — Supports both reading from and writing back to TDengine databases
- Java Implementation — Built as a Java project with Maven for dependency management
Philosophy
The project follows a straightforward integration approach, focusing on creating reliable connectors between TDengine's time-series database and Spark's distributed processing framework without unnecessary abstraction layers.
Related Projects
tsarchiveConsume data streams from a Kafka topic, archive the data packets into the TDEngine time series database.
YoMo x TDengineStreaming high frequency logs to TDEngine.
taostdtaostd is a simple sql executor for TDengine.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.