How does simdjson compare to RapidJSON in real-world usage?

simdjson parses JSON 4x faster than RapidJSON with full validation, as shown in performance benchmarks. It also offers automatic CPU optimization, whereas RapidJSON requires manual tuning for similar speeds.

How to integrate simdjson into a C++ project quickly?

Download the single-header files simdjson.h and simdjson.cpp, include them in your code, and compile with a C++17-compliant compiler. The quick start guide provides a simple example using a sample JSON file.

Can simdjson handle JSON streams or NDJSON?

Yes, simdjson efficiently processes NDJSON streams at up to 3.5 GB/s using multithreaded parsing functions. Documentation includes details on parse_many for handling newline-delimited JSON.

Is simdjson suitable for use with Python or Node.js?

Yes, via bindings like pysimdjson for Python and simdjson_nodejs for Node.js, but these may have limitations compared to the native C++ API and require additional installation steps.

What are the memory usage implications of simdjson?

simdjson is designed to avoid surprises in memory allocation, but its high-speed parsing can involve upfront memory for padded strings and SIMD buffers, which might be higher than simpler parsers for small data.

How to validate UTF-8 with simdjson at high speeds?

Use the built-in UTF-8 validation utility that operates at 13 GB/s, as mentioned in the record-breaking features. It's integrated into the parsing process, ensuring compliance without extra steps.

Open-Awesome

simdjson

Apache-2.0C++v5.0.5

A high-performance C++ JSON parser that uses SIMD instructions to parse gigabytes of JSON per second.

Visit Website GitHub

24.4k stars1.3k forks0 contributors

What is simdjson?

simdjson is a high-performance C++ JSON parsing library that uses SIMD instructions and microparallel algorithms to parse JSON at gigabytes per second. It solves the performance bottleneck of JSON parsing in servers and data-intensive applications by offering full validation with unprecedented speed.

Target Audience

Developers and engineers working on high-performance servers, data processing pipelines, databases, and applications where JSON parsing is a critical performance factor, such as in Node.js, ClickHouse, and other data-intensive systems.

Value Proposition

Developers choose simdjson for its unmatched parsing speed—4x faster than RapidJSON—while maintaining full JSON and UTF-8 validation, automatic CPU optimization, and easy integration with no configuration needed.

Overview

Parsing gigabytes of JSON per second : used by Facebook/Meta Velox, the Node.js runtime, ClickHouse, WatermelonDB, Apache Doris, Milvus, StarRocks

Use Cases

Best For

Accelerating JSON parsing in high-throughput server applications
Improving performance in databases like ClickHouse or Apache Doris that handle JSON data
Validating and minifying large JSON datasets at multi-gigabyte speeds
Processing NDJSON (Newline-Delimited JSON) streams efficiently
Integrating a fast, reliable JSON parser into C++ projects without sacrificing compliance
Reducing server CPU usage and latency in JSON-heavy microservices

Not Ideal For

Projects in non-C++ ecosystems where integrating a native C++ library adds undue complexity
Legacy systems with outdated compilers lacking C++17 or SIMD instruction support
Applications handling only small, infrequent JSON payloads where performance overhead isn't justified

Pros & Cons

Pros

Unmatched Parsing Speed

Parses JSON at gigabytes per second—4x faster than RapidJSON and 25x faster than JSON for Modern C++—as validated by benchmarks on Intel Skylake processors.

Full Compliance and Validation

Provides complete JSON and UTF-8 validation with lossless parsing, ensuring data integrity without compromising speed, as emphasized in its design philosophy.

Automatic Hardware Optimization

Dynamically selects CPU-specific parsers at runtime with no configuration, leveraging SIMD instructions for optimal performance across different processors.

Comprehensive High-Speed Utilities

Includes record-breaking utilities like JSON minification at 6 GB/s, UTF-8 validation at 13 GB/s, and NDJSON processing at 3.5 GB/s, as listed in key features.

Cons

C++ and Compiler Dependencies

Requires modern C++ compilers (g++ 7+ or clang++ 6+) and a 64-bit system, limiting portability to older or non-standard environments, as noted in the quick start prerequisites.

Limited Native Language Support

Core functionality is C++-centric; while bindings exist for Python, Rust, and others, they add layers of complexity and may not offer full feature parity.

Overhead for Small-Scale Use

Optimized for large datasets, so the setup and SIMD-driven architecture might be excessive for applications parsing only tiny, occasional JSON files.

Frequently Asked Questions

Related Projects

FASTJSON 2.0.x has been released, faster and more secure, recommend you upgrade.

Stars25,569

Forks6,381

Last commit2 months ago

Gson

A Java serialization/deserialization library to convert Java Objects into JSON and back

Stars24,238

Forks4,491

Last commit11 days ago

SwiftyJSON

The better way to deal with JSON data in Swift.

Stars22,927

Forks3,417