Data Ingestion
Showing 29 of 65 projects
A high-performance Java library for asynchronous data ingestion to Fluentd, Fluent Bit, AWS S3, and Treasure Data.
Operator and codec library for building real-time streaming applications on Apache Apex.
A blazing fast specialized time-series database optimized for IoT, real-time connected devices, and AI analytics.
An asynchronous Scala client library for interacting with InfluxDB, providing non-blocking operations for data writing, querying, and database management.
A Storm spout that fetches data records from Amazon Kinesis and emits them as tuples for real-time stream processing.
A Logstash input plugin that reads data from DynamoDB tables via table scans and DynamoDB Streams for near real-time data processing.
A Java adapter that enables Amazon Kinesis Client Library (KCL) to consume and process data from DynamoDB Streams.
A Rust client library for InfluxDB 1.0+ providing HTTP and UDP support with an easy-to-use API.
A lightweight Java library for mapping Excel data to POJOs using annotations or programmatic APIs with minimal memory overhead.
A Flume NG sink that writes event data to MongoDB with flexible configuration for collection mapping and data transformation.
A multi-threaded Python example demonstrating how to produce and consume records from Amazon Kinesis streams.
A Clojure client library for interacting with InfluxDB time-series databases.
A Haskell client library for interacting with InfluxDB 1.8 time-series databases.
A demo application that stores and indexes NASA JPL Mars images in DynamoDB with a Java ingester and JavaScript frontend.
A modular framework for ingesting and processing Algorand blockchain data into external applications.
A Rust driver for InfluxDB, enabling data writing and querying from Rust applications.
A Python tool that connects to ham radio APRS-IS servers and saves packet data into an InfluxDB database.
An Elixir client library for interacting with the Keen.io analytics API.
OpenResty client library for writing data points to InfluxDB via UDP and HTTP interfaces.
A configurable GenStage-based bulk processor for efficiently inserting data into Elasticsearch from Elixir applications.
An Erlang TCP client for DalmatinerDB, enabling data reading, writing, and connection management.
A MicroPython driver for connecting to CrateDB databases, enabling SQL queries and data operations from microcontrollers.
An Apache Flume source plugin for ingesting UDP messages directly into Flume data pipelines.
Rust bindings for TDengine, providing native Rust access to the time-series database's C API.
A HiveMQ extension that writes MQTT publish data (timestamp, topic, payload) to TDengine in real-time via HTTP or SDK.
A Kafka consumer that archives data streams into TDEngine time-series database for efficient storage and querying.
Protocol definitions for DalmatinerDB, a high-performance time-series database.
Import CSV files from AWS S3 into Cassandra using Apache Spark with a simple configuration-based approach.
A HiveMQ extension that stores MQTT publish data into TDengine using customizable JSON templates and storage policies.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.