Data Processing
Showing 36 of 361 projects
A high-performance, streaming JSON parser for C applications requiring maximum speed with large JSON documents.
A command-line tool that joins a stream of GeoJSON features with an external dataset based on matching field values.
A header-only CSV parser for Modern C++ supporting both compile-time and runtime parsing with RFC 4180 compliance.
Converts any GeoJSON object into a standardized GeoJSON FeatureCollection.
A tool for collecting, validating, enhancing, and merging GTFS feeds for public transport data, with a focus on German agencies.
A CSV reading and writing library for Elixir that operates on files or strings.
Rust bindings and safe wrapper APIs for Hadoop's libhdfs, enabling HDFS access from Rust applications.
A pure Swift CSV parser and serializer with Codable support for encoding and decoding.
A Node.js stream that buffers and transforms the first n bytes of a stream before passing it through.
A Python framework for distributed processing and automation of neuroimaging data on XNAT platforms.
A simple Elixir library for lazy CSV parsing, optimized for handling large exported databases.
A collection of libraries and tools for working with JSON-stat, a lightweight JSON format for statistical data exchange.
A Common Lisp feed parser that sanitizes HTML content and provides a Python feedparser-like API.
Global flight schedule datasets extracted from ADS-B position transmissions, published quarterly from 2024 onwards.
Compiles and analyzes bus and light rail speed, reliability, and ridership data from multiple sources for transportation insights.
An Elixir MapReduce framework with Hadoop Streaming integration, simplifying distributed data processing.
A lightweight JavaScript utility for removing duplicate values from arrays, supporting both primitive and complex types.
A Node-RED node for chainable string manipulation using the string.js library with a visual editor.
A high-performance, cache-oriented JSON library for D, optimized for transforming large volumes of JSON dataframes.
An Elixir library for parsing Excel 2000 .xlsx files into lists of cell values.
A lightweight XML parser for Deno, ported from the segmentio/xml-parser library.
A Scala sample application demonstrating Spark Streaming integration with Kafka and Cassandra for data processing.
A pure Crystal implementation of the Snappy compression format with support for both block and framing formats.
A Groovy/Java library for processing tabular data (CSV, SQL, JSON) with fuzzy column matching, transformations, merging, and querying.
A job server for distributed compute analysis of web archive (WARC) collections.
Generates simplified geoJSON route shapes from GTFS transit data by removing overlaps and downsampling points.
A lazy iterator implementation for Go providing map, reduce, and other functional programming operations.
A simple wrapper around cascading.hbase for seamless HBase integration in Cascalog workflows.
A header-only C++17 library for parsing CSV files with strict semantic versioning and cross-compiler support.
A tool for checking, sanitizing, and minimizing GTFS feeds while preserving semantic equivalency.
A Common Lisp library for reading and writing Microsoft Excel .xlsx and LibreOffice .ods files with a unified API.
A Scala/Spark library for efficient processing, extraction, and derivation of web archive data (CDX/WARC).
A JavaScript utility to split arrays and TypedArrays into chunks of a specified size.
A Common Lisp library for automatic detection of text file encoding, delimiters, end-of-line characters, and column consistency.
A Go library for reading and parsing nfdump files containing netflow and sflow data.
A collection of Python scripts for editing and modifying GeoJSON data with a focus on indoor mapping features.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.