How to integrate Grai with dbt for data lineage?

Install the dbt connector via `pip install grai-source-dbt`, then configure it to point to your dbt project. Grai will automatically extract lineage from dbt models and sync it to the server for visualization and testing.

Grai vs OpenMetadata: which is better for data lineage?

Grai focuses on automated column-level lineage and CI/CD-driven data testing, making it ideal for proactive quality checks. OpenMetadata offers broader metadata management and discovery features, so choose Grai if testing and lineage depth are priorities.

Can Grai run data tests in GitLab CI instead of GitHub Actions?

While Grai has built-in GitHub Actions integrations, you can adapt its Python client and testing library to other CI/CD systems like GitLab CI, but it requires custom scripting as native support isn't provided.

What data sources does Grai support out of the box?

Grai supports pre-built connectors for common sources like Snowflake, BigQuery, Postgres, dbt, Fivetran, and Metabase, with installation via pip. Check the connectors table in the README for the full list.

How to deploy Grai on Kubernetes?

Use the provided Helm charts by adding the Grai repo with `helm repo add grai https://charts.grai.io` and installing, or refer to the Kubernetes examples in the deployment folder for custom configurations.

Does Grai support streaming data lineage for Kafka?

Currently, Grai's pre-built connectors focus on batch data sources like databases and warehouses. Support for streaming platforms like Kafka is not listed, so you might need to build a custom integration.

Grai — Open Source Data Catalog

What is Grai?

Grai is an open-source data catalog and lineage tool that automatically maps how data flows across databases, warehouses, APIs, and BI dashboards. It integrates into CI/CD systems to run downstream impact tests on data changes, preventing modifications that could break data pipelines or business intelligence reports from reaching production.

Target Audience

Data engineers, analytics engineers, and data platform teams who need to maintain data quality, understand dependencies, and automate testing of data changes across complex data stacks.

Value Proposition

Developers choose Grai because it provides automated, column-level data lineage with pre-built connectors, integrates directly into existing CI/CD workflows for proactive testing, and is fully self-hostable, giving teams complete control over their metadata and deployment environment.

Overview

Grai is an open-source data lineage and catalog tool designed to help data teams understand and test how data flows across their entire stack. It automatically builds column-level lineage from databases, warehouses, BI tools, and transformation layers, then integrates this metadata into CI/CD workflows to validate changes before they reach production.

Key Features

Pre-built Connectors — Automatically synchronizes lineage from popular data sources like Snowflake, BigQuery, dbt, Fivetran, and Metabase.
Centralized Data Tests — Write validation tests that run whenever upstream data sources change, ensuring data quality.
CI/CD Integration — Runs data validation tasks as part of GitHub Actions to test changes everywhere data is used.
Self-Hosted Deployment — Fully open-source and deployable on your own infrastructure with Docker, Kubernetes, or Helm.
Column-Level Lineage — Provides detailed visibility into how data columns relate across systems, not just table-level dependencies.

Philosophy

Grai believes data teams should have full control over their metadata and be able to test data changes with the same rigor as code changes, preventing broken pipelines and dashboards before they impact production.

Grai

What is Grai?

Overview

Key Features

Philosophy

Related Projects

Found a gem we're missing?

Use Cases

Best For

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions