Question 1

How to parse NDJSON files in Python?

Accepted Answer

Use the built-in json module to parse each line individually, or libraries like ndjson or ijson for efficient streaming. Read the file line by line, apply json.loads to non-empty lines, and handle errors for malformed JSON.

Question 2

NDJSON vs JSON Lines: what's the difference?

Accepted Answer

NDJSON and JSON Lines are functionally identical—both use newlines to separate JSON objects. NDJSON is a formal specification with a defined media type (application/x-ndjson), while JSON Lines is a more informal term often used interchangeably in tools and documentation.

Question 3

Is NDJSON suitable for HTTP streaming?

Accepted Answer

Yes, NDJSON works well with HTTP streaming by setting the Content-Type header to application/x-ndjson and sending data in chunks. Clients must parse each line incrementally, but it lacks built-in support for features like compression or backpressure management.

Question 4

How to handle errors in NDJSON streams?

Accepted Answer

Implement custom error handling by configuring parsers to skip or log malformed lines, as the spec only suggests raising errors. Use libraries with configurable behavior or build recovery logic to resume parsing after invalid entries.

Question 5

What are best practices for large NDJSON datasets?

Accepted Answer

Process data incrementally using streaming parsers to avoid memory overload. Tools like jq or custom scripts can filter and transform lines on-the-fly, and consider compressing files for storage efficiency since NDJSON doesn't include built-in compression.

Question 6

NDJSON or JSON arrays for API responses?

Accepted Answer

For streaming large datasets where clients need to process data as it arrives, NDJSON is superior due to incremental parsing. For small, complete payloads, JSON arrays are simpler and more widely supported in web APIs.

NDJSON

What is NDJSON?

Overview

Use Cases

Best For

Related Projects

Found a gem we're missing?

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions