Data Profiler
A Python library that automatically extracts schema, statistics, and sensitive entities (PII/NPI) from datasets.
What is Data Profiler?
DataProfiler is a Python library that automates data analysis, monitoring, and sensitive data detection. It automatically loads various data formats, extracts detailed statistics and schema, and identifies personally identifiable information (PII) using a pre-trained deep learning model. The library generates comprehensive profiles for structured, unstructured, and graph data to support downstream applications and improve data governance.
Data engineers, data scientists, and analysts who need to quickly profile datasets, monitor data quality, and detect sensitive information across CSV, JSON, Avro, Parquet, text, and other formats. It is also suitable for teams requiring automated data insights for compliance and governance.
Developers choose DataProfiler for its automated, all-in-one profiling that combines statistical analysis with built-in sensitive data detection, reducing manual effort. Its ability to update, merge, and diff profiles supports incremental and distributed analysis, making it versatile for ongoing data monitoring.
Overview
What's in your data? Extract schema, statistics and entities from datasets
Use Cases
Best For
- Automatically profiling datasets to extract schema, statistics, and data types without manual configuration.
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
