Data Processing
Showing 36 of 361 projects
A command-line tool that joins a stream of GeoJSON features with an external dataset based on matching field values.
A tool for collecting, validating, enhancing, and merging GTFS feeds for public transport data, with a focus on German agencies.
A header-only CSV parser for Modern C++ supporting both compile-time and runtime parsing with RFC 4180 compliance.
Converts any GeoJSON object into a standardized GeoJSON FeatureCollection.
A CSV reading and writing library for Elixir that operates on files or strings.
Global flight schedule datasets extracted from ADS-B position transmissions, published quarterly from 2024 onwards.
A pure Swift CSV parser and serializer with Codable support for encoding and decoding.
Rust bindings and safe wrapper APIs for Hadoop's libhdfs, enabling HDFS access from Rust applications.
A Python framework for distributed processing and automation of neuroimaging data on XNAT platforms.
A Node.js stream that buffers and transforms the first n bytes of a stream before passing it through.
A simple Elixir library for lazy CSV parsing, optimized for handling large exported databases.
A Common Lisp feed parser that sanitizes HTML content and provides a Python feedparser-like API.
A collection of libraries and tools for working with JSON-stat, a lightweight JSON format for statistical data exchange.
Compiles and analyzes bus and light rail speed, reliability, and ridership data from multiple sources for transportation insights.
An Elixir MapReduce framework with Hadoop Streaming integration, simplifying distributed data processing.
A lightweight JavaScript utility for removing duplicate values from arrays, supporting both primitive and complex types.
A Node-RED node for chainable string manipulation using the string.js library with a visual editor.
A high-performance, cache-oriented JSON library for D, optimized for transforming large volumes of JSON dataframes.
A pure Crystal implementation of the Snappy compression format with support for both block and framing formats.
A Scala sample application demonstrating Spark Streaming integration with Kafka and Cassandra for data processing.
An Elixir library for parsing Excel 2000 .xlsx files into lists of cell values.
A lightweight XML parser for Deno, ported from the segmentio/xml-parser library.
A Groovy/Java library for processing tabular data (CSV, SQL, JSON) with fuzzy column matching, transformations, merging, and querying.
A job server for distributed compute analysis of web archive (WARC) collections.
A lazy iterator implementation for Go providing map, reduce, and other functional programming operations.
A tool for checking, sanitizing, and minimizing GTFS feeds while preserving semantic equivalency.
A Common Lisp library for reading and writing Microsoft Excel .xlsx and LibreOffice .ods files with a unified API.
Generates simplified geoJSON route shapes from GTFS transit data by removing overlaps and downsampling points.
A header-only C++17 library for parsing CSV files with strict semantic versioning and cross-compiler support.
A simple wrapper around cascading.hbase for seamless HBase integration in Cascalog workflows.
A fast, memory-efficient CLI tool for splitting large SQL dump files into individual table files and converting between SQL dialects.
A Scala/Spark library for efficient processing, extraction, and derivation of web archive data (CDX/WARC).
A JavaScript utility to split arrays and TypedArrays into chunks of a specified size.
A tool that merges multiple GTFS transit datasets into a single unified feed for regional transit analysis.
A Common Lisp library for automatic detection of text file encoding, delimiters, end-of-line characters, and column consistency.
A collection of Python scripts for editing and modifying GeoJSON data with a focus on indoor mapping features.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.