csv
RFC 4180 compliant, composable CSV parsing and encoding library for Elixir.
What is csv?
CSV is an Elixir library for decoding and encoding CSV files, designed to handle the format's notorious inconsistencies gracefully. It focuses on extracting all correctly formatted rows from data pipelines while providing clear error diagnostics for problematic lines. The library is RFC 4180 compliant and follows a composable, stream-based design.
Elixir developers building data processing pipelines that need to parse or generate CSV files, especially when dealing with imperfect or inconsistently formatted data sources.
Developers choose this library for its robust error handling in normal mode, which returns all valid rows with descriptive errors, and its seamless integration with Elixir streams for composable data processing. Its performance-oriented design, using fast binary matching, ensures it rarely becomes a bottleneck.
Overview
CSV Decoding and Encoding for Elixir
Use Cases
Best For
- Processing large CSV files in Elixir with streaming to avoid memory issues.
- Extracting all correctly formatted rows from inconsistent CSV data while logging errors.
- Building data pipelines where CSV parsing is one step among many composable operations.
- Strict validation of CSV files where raising an exception on the first error is required.
- Handling CSV files with custom separators, escape characters, or formula escaping needs.
- Encoding Elixir data structures to CSV with polymorphic support for custom types via protocols.
Not Ideal For
- Projects outside the Elixir ecosystem requiring CSV processing in other languages like Python or JavaScript.
- Applications needing built-in CSV visualization, GUI editing tools, or interactive dashboards.
- Use cases where CSV files are always perfectly formatted and a minimal, zero-dependency parser (e.g., simple string splitting) is sufficient.
- Real-time data pipelines requiring low-latency, stateful stream processing with complex event-driven logic beyond batch-oriented CSV parsing.
Pros & Cons
Pros
In normal mode, decode returns a stream of ok/error tuples, extracting all valid rows even with malformed lines, as emphasized in the README for handling fickle CSV formats.
Uses fast binary matching to parse ~500k rows per second on modest hardware, ensuring it rarely bottlenecks data pipelines, per performance benchmarks.
Designed to work with Elixir streams and enumerables, fitting composably into data processing pipelines without extra overhead, following UNIX philosophy.
Supports custom separators, escape characters, field transformations, and formula unescaping, allowing adaptation to various CSV dialects as shown in decode options.
Allows custom data type encoding via the CSV.Encode protocol, enabling polymorphic output for complex structures, detailed in the encoding section.
Cons
Version 3.x eliminated parallelism features (num_workers, worker_work_ratio), which could reduce throughput on multi-core systems for large files compared to earlier versions.
Optimal byte streaming requires fine-tuning chunk sizes (e.g., default 1000 bytes), adding complexity for developers to achieve best performance, as noted in the performance tips.
For efficient encoding, protocols must be consolidated (consolidate_protocols: true), an extra setup step that can be overlooked and lead to performance degradation.
Frequently Asked Questions
Related Projects
nimble_csvA simple and fast CSV parsing and dumping library for Elixir
csv2sqlA blazing fast fully-automated CSV to database importer
csvlixirA CSV reading/writing application for Elixir.
cessoCSV handling library for Elixir.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.