CloudQuery
Open-source data pipelines to sync cloud infrastructure metadata from AWS, Azure, GCP, and 70+ sources into your data warehouse.
What is CloudQuery?
CloudQuery is an open-source cloud asset inventory platform that extracts configuration, security, and billing data from cloud providers and SaaS tools. It syncs this metadata into a data warehouse, enabling teams to build custom solutions for cloud security posture management (CSPM), FinOps, and asset inventory. The platform normalizes data across sources and makes it queryable with SQL, replacing fragmented API scripts.
Platform teams, cloud engineers, and security professionals who need to unify and analyze cloud infrastructure data across multiple providers and SaaS tools.
Developers choose CloudQuery for its privacy-first, self-hosted architecture, extensive plugin ecosystem covering 70+ sources, and flexibility to integrate with existing data stacks. It avoids vendor lock-in by providing composable, code-first tools that fit into any workflow.
Overview
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Use Cases
Best For
- Building a unified cloud asset inventory across AWS, Azure, GCP, and SaaS platforms
- Implementing custom Cloud Security Posture Management (CSPM) solutions
- Centralizing cloud billing data for FinOps cost analysis and optimization
- Replacing custom API scripts with SQL-queryable cloud infrastructure data
- Self-hosting cloud data pipelines for regulated or security-sensitive environments
- Extending cloud data collection with custom plugins for niche or internal tools
Not Ideal For
- Teams wanting a fully managed, out-of-the-box CSPM or FinOps solution with no setup or maintenance
- Organizations with only a single cloud provider and simple monitoring needs that native tools can address
- Projects lacking SQL or data engineering skills for pipeline configuration and query development
Pros & Cons
Pros
Syncs data from AWS, Azure, GCP, and over 70 SaaS sources like Wiz and GitHub, enabling comprehensive asset inventory across diverse environments.
Transforms cloud APIs into SQL-queryable tables, eliminating custom API scripts and allowing easy integration with BI tools for analysis.
Uses Apache Arrow for fast data ingestion, efficiently handling large-scale cloud data volumes with optimized pipelines.
Runs entirely on your infrastructure, ensuring data never touches external servers, which is critical for regulated or security-sensitive settings.
Cons
Requires configuring data destinations, orchestrating syncs, and managing plugins, which can be daunting without data engineering expertise.
Focuses on data extraction; teams must build their own dashboards and alerts using external tools like Grafana, adding development overhead.
With many community-contributed plugins, some may have inconsistent maintenance, documentation, or feature completeness compared to official integrations.
Frequently Asked Questions
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.