Data Processing
Showing 36 of 361 projects
An OCaml library for parsing RSS and Atom feeds with a focus on type safety and reliability.
A command-line tool that joins a stream of GeoJSON features with an external dataset based on matching field values.
A header-only CSV parser for Modern C++ supporting both compile-time and runtime parsing with RFC 4180 compliance.
A tool for collecting, validating, enhancing, and merging GTFS feeds for public transport data, with a focus on German agencies.
Converts any GeoJSON object into a standardized GeoJSON FeatureCollection.
A CSV reading and writing library for Elixir that operates on files or strings.
Rust bindings and safe wrapper APIs for Hadoop's libhdfs, enabling HDFS access from Rust applications.
A pure Swift CSV parser and serializer with Codable support for encoding and decoding.
A Python framework for distributed processing and automation of neuroimaging data on XNAT platforms.
A Node.js stream that buffers and transforms the first n bytes of a stream before passing it through.
A simple Elixir library for lazy CSV parsing, optimized for handling large exported databases.
Global flight schedule datasets extracted from ADS-B position transmissions, published quarterly from 2024 onwards.
A Common Lisp feed parser that sanitizes HTML content and provides a Python feedparser-like API.
A collection of libraries and tools for working with JSON-stat, a lightweight JSON format for statistical data exchange.
An Elixir MapReduce framework with Hadoop Streaming integration, simplifying distributed data processing.
A lightweight JavaScript utility for removing duplicate values from arrays, supporting both primitive and complex types.
Compiles and analyzes bus and light rail speed, reliability, and ridership data from multiple sources for transportation insights.
A Node-RED node for chainable string manipulation using the string.js library with a visual editor.
A high-performance, cache-oriented JSON library for D, optimized for transforming large volumes of JSON dataframes.
A Scala sample application demonstrating Spark Streaming integration with Kafka and Cassandra for data processing.
An Elixir library for parsing Excel 2000 .xlsx files into lists of cell values.
A pure Crystal implementation of the Snappy compression format with support for both block and framing formats.
A lightweight XML parser for Deno, ported from the segmentio/xml-parser library.
A Groovy/Java library for processing tabular data (CSV, SQL, JSON) with fuzzy column matching, transformations, merging, and querying.
A job server for distributed compute analysis of web archive (WARC) collections.
A Common Lisp library for reading and writing Microsoft Excel .xlsx and LibreOffice .ods files with a unified API.
A lazy iterator implementation for Go providing map, reduce, and other functional programming operations.
Generates simplified geoJSON route shapes from GTFS transit data by removing overlaps and downsampling points.
A header-only C++17 library for parsing CSV files with strict semantic versioning and cross-compiler support.
A simple wrapper around cascading.hbase for seamless HBase integration in Cascalog workflows.
A tool for checking, sanitizing, and minimizing GTFS feeds while preserving semantic equivalency.
A Scala/Spark library for efficient processing, extraction, and derivation of web archive data (CDX/WARC).
A fast, memory-efficient CLI tool for splitting large SQL dump files into individual table files and converting between SQL dialects.
A JavaScript utility to split arrays and TypedArrays into chunks of a specified size.
A Go library for reading and parsing nfdump files containing netflow and sflow data.
A Common Lisp library for automatic detection of text file encoding, delimiters, end-of-line characters, and column consistency.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.