How to handle nested unknown fields in Marshmallow?

You must implement the JSONDataErrorHandler interface on your structs, as mentioned in the README. This allows capturing dynamic data within nested structures, but it requires extra code compared to top-level fields.

Marshmallow vs json.Unmarshal: which is better for known schemas?

For fully known schemas, json.Unmarshal is slightly more memory-efficient (304 B/op vs. 608 B/op) and simpler, as Marshmallow's mixed-data features add overhead. Use Marshmallow only when you need to preserve unknown fields.

How to skip populating struct in Marshmallow for performance?

Use the WithSkipPopulateStruct option when calling Unmarshal. This boosts speed by omitting struct population, useful when you only need the dynamic map, as shown in benchmarks where it reduces time to 1751 ns/op.

Does Marshmallow support JSON schema validation?

No, Marshmallow focuses on unmarshalling with data retention, not validation. You would need to pair it with a separate validation library if schema enforcement is required for your JSON data.

Can I use Marshmallow with web frameworks like Gin or Echo?

Yes, Marshmallow works seamlessly with any Go web framework. You can use it in request handlers to unmarshal JSON payloads with mixed data, similar to how you'd use json.Unmarshal, but with added flexibility for unknown fields.

Is Marshmallow production-ready for high-performance applications?

Yes, with benchmarks showing up to 3x faster performance than alternatives for mixed data and 92.6% test coverage, it's suitable for production. However, monitor for updates due to its external dependency nature.

Open-Awesome

marshmallow

MITGov1.1.5

A high-performance Go library for JSON unmarshalling that handles both known and unknown fields without data loss.

GitHub

391 stars12 forks0 contributors

What is marshmallow?

Marshmallow is a Go library for JSON unmarshalling that excels at handling mixed structured and unstructured data. It allows developers to unmarshal JSON into a typed struct while preserving unknown fields in a dynamic map, solving the problem of data loss when working with partial schemas.

Target Audience

Go developers working with JSON APIs, microservices, or data pipelines where input data may have both known and unknown fields.

Value Proposition

Developers choose Marshmallow for its performance—it's up to 3x faster than alternatives for mixed data—and its simplicity, requiring no extra code to retain full JSON data while using typed structs.

Overview

Marshmallow provides a flexible and performant JSON unmarshalling in Go. It specializes in dealing with unstructured struct - when some fields are known and some aren't, with zero performance overhead nor extra coding needed.

Use Cases

Best For

Unmarshalling JSON with partially known schemas
Building APIs that need to preserve all incoming JSON fields
Processing configuration files with optional or dynamic fields
Implementing data validation with access to raw input
Optimizing performance in JSON-heavy Go applications
Handling webhook payloads with variable structures

Not Ideal For

Projects where the entire JSON schema is static and known upfront, as standard json.Unmarshal suffices
Applications requiring advanced JSON features like schema validation, custom unmarshalers, or streaming parsing
Systems with strict zero-dependency policies that prefer using only the Go standard library

Pros & Cons

Pros

Near-Native Performance

Benchmarks show it performs as fast as native json.Unmarshal (1845 ns/op vs. 1853 ns/op) while retaining all data, with optimizations like reflection caching.

Mixed Data Handling

Efficiently unmarshals JSON with both known and unknown fields into typed structs and a dynamic map simultaneously, eliminating boilerplate code for partial schemas.

Full Data Retention

Prevents data loss by returning a map with all original JSON fields alongside the typed struct, unlike standard unmarshal which drops unknown fields.

Flexible Configuration

Offers options like WithSkipPopulateStruct for speed boosts and WithMode for error handling, providing fine-grained control over unmarshalling behavior.

Cons

Nested Field Complexity

To capture unknown nested fields, structs must implement JSONDataErrorHandler, adding boilerplate and complexity compared to flat structures.

Memory Overhead

For known fields, it allocates more memory (608 B/op) than native unmarshal (304 B/op), as per benchmarks, which might impact memory-sensitive applications.

Third-Party Dependency

Introduces reliance on external maintenance; while active, future breaking changes or abandonment could affect long-term projects compared to the standard library.

Frequently Asked Questions

Related Projects

GJSON

Get JSON values quickly - JSON parser for Go

Stars15,560

Forks913

Last commit11 days ago

gabs

For parsing, creating and editing unknown or dynamic JSON in Go

Stars3,526

Forks200

Last commit2 years ago

SJSON

Set JSON values very quickly in Go

Stars2,738

Forks187

Last commit3 months ago

gojson

Automatically generate Go (golang) struct definitions from example JSON

Stars2,689

Forks205

Last commit5 years ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub