Datanymizer vs pg_anonymizer – which should I use?

Datanymizer stands out with Rust-based speed, over 70 built-in transformers, and advanced features like SQL assertions and cross-row references, while pg_anonymizer is simpler but may lack such depth for complex anonymization needs.

How to generate unique fake emails with Datanymizer?

Use the 'email' transformer in your YAML config with 'uniq: true' and optional 'try_count' for retries. For example, specify 'email: {kind: Safe, uniq: {required: true}}' under the column rules to ensure no duplicates.

Can Datanymizer handle large databases without slowing down?

Thanks to Rust, it's optimized for performance, but enforcing uniqueness and complex rules like templates with multiple steps can add overhead. Use selective filtering and limit clauses to manage very large tables efficiently.

Does Datanymizer work with Docker in a CI/CD pipeline?

Yes, Docker images are available for different PostgreSQL versions, and the CLI can be integrated into pipelines. Mount config files and output directories to automate anonymized dump generation for testing environments.

How to preserve foreign key relationships when anonymizing data?

Leverage the built-in key-value store with 'store_write' and 'store_read' functions in template rules to share anonymized values between tables, ensuring referential integrity is maintained in the dump.

What happens if Datanymizer fails a SQL assertion?

By default, assertions with 'severity: error' stop the dump process if conditions aren't met, while 'severity: warn' logs a warning and continues, allowing you to validate data quality before anonymization.

Open-Awesome

datanymizer

MITRustv0.7.4

A powerful, Rust-based database anonymizer that generates native SQL dumps with flexible transformation rules.

Visit Website GitHub

570 stars30 forks0 contributors

What is datanymizer?

Datanymizer is a database anonymization tool written in Rust that generates sanitized SQL dumps from production databases. It replaces sensitive data like names, emails, and phone numbers with realistic fake data using a flexible rule-based configuration, enabling safe use of production-like data in development and testing environments. The tool focuses on PostgreSQL support and produces native dumps that can be imported directly without extra tooling.

Target Audience

Developers, DevOps engineers, and QA teams who need to work with realistic but anonymized database copies for development, testing, or analytics without exposing sensitive user information.

Value Proposition

Developers choose Datanymizer for its high performance due to being Rust-based, its extensive and flexible rule system for precise data transformations, and its ability to generate directly usable PostgreSQL dumps that maintain database integrity and relationships.

Overview

Powerful database anonymizer with flexible rules. Written in Rust.

Use Cases

Best For

Creating anonymized database clones for development and staging environments
Generating compliant datasets for software testing that contain no real PII
Preparing production-like data for performance benchmarking and load testing
Migrating sensitive databases to less secure environments safely
Automating data sanitization in CI/CD pipelines for integration tests
Producing realistic fake datasets for demo applications or training

Not Ideal For

Teams using MySQL, MariaDB, or other non-PostgreSQL databases
Projects requiring real-time or incremental data anonymization instead of full dumps
Simple masking tasks where native SQL functions or basic scripts would suffice
Environments with minimal configuration tolerance due to complex YAML rule definitions

Pros & Cons

Pros

Rust-Powered Performance

Written in Rust, Datanymizer delivers fast and reliable processing for large databases, minimizing overhead during dump generation.

Extensive Transformer Library

Offers over 70 built-in rules like emails, phones, and dates via YAML config, enabling precise, column-level anonymization with realistic fake data.

Native PostgreSQL Dumps

Generates database-native SQL dumps that import directly with tools like psql, eliminating third-party dependencies and ensuring compatibility.

Advanced Referential Integrity

Maintains data relationships through cross-row references and a built-in key-value store, preserving consistency across tables in anonymized output.

Cons

Single-Database Limitation

Only supports PostgreSQL currently, with MySQL/MariaDB listed as TODO, making it ineffective for multi-database or heterogeneous environments.

Steep Configuration Learning Curve

The YAML rule system requires deep understanding of schema and transformers, with complex setups for templates, uniqueness, and conditions that can be error-prone.

No Incremental or Live Anonymization

Generates full SQL dumps only, lacking support for streaming changes or delta updates, which can be inefficient for frequently changing databases.

Frequently Asked Questions

Related Projects

goose

an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM

Stars51,561

Forks5,677

Last commit15 hours ago

just

🤖 Just a command runner

Stars34,913

Forks825

Last commit20 hours ago

mise

dev tools, env vars, task runner

Stars31,057

Forks1,283

Last commit14 hours ago

A Git-compatible VCS that is both simple and powerful

Stars30,567

Forks1,146