Quickwit-oss/quickwit
A cloud-native search engine optimized for observability data like logs and traces, offering sub-second search on cloud storage.
What is Quickwit-oss/quickwit?
Quickwit is an open-source, cloud-native search engine specifically built for observability data like logs, traces, and future metrics. It solves the problem of expensive and complex log management by enabling fast, sub-second search directly on affordable cloud object storage, decoupling compute from storage for better scalability.
DevOps engineers, SREs, and platform teams managing large-scale observability data who need a cost-effective, scalable alternative to Elasticsearch or commercial solutions like Datadog.
Developers choose Quickwit for its significant cost reduction (up to 10x cheaper than Elastic), cloud-native stateless architecture, and full compatibility with Elasticsearch APIs, making migration straightforward while offering superior scalability on object storage.
Overview
Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.
Use Cases
Best For
- Building a cost-effective log management system on AWS S3 or similar cloud storage
- Migrating from Elasticsearch or OpenSearch to a more cloud-optimized search engine
- Implementing distributed tracing storage for Jaeger or OpenTelemetry
- Creating a scalable observability backend with decoupled compute and storage
- Handling petabyte-scale observability data with retention policies
- Deploying a Kubernetes-native search engine for logs and traces
Not Ideal For
- Real-time indexing applications requiring high availability without Kafka integration
- General-purpose search engines for e-commerce or document retrieval outside observability
- Organizations needing a mature, battle-tested solution with extensive community plugins and integrations
- Projects that rely heavily on local or block storage instead of cloud object storage
Pros & Cons
Pros
Delivers sub-second search directly on affordable object storage like S3, with claims of being up to 10x cheaper than Elasticsearch, as highlighted in their blog post on cost comparisons.
Supports a large subset of Elasticsearch APIs, including ingest and search endpoints, making migration easier and allowing use with existing clients and tools like Vector or Fluent Bit.
Built-in native support for OpenTelemetry logs/traces and Jaeger integration, optimizing it specifically for log management and distributed tracing use cases.
Decouples compute and storage with stateless indexers and searchers, enabling independent scaling and high availability for search, which is ideal for cloud-native deployments.
Offers both schemaless and strict schema indexing, accommodating unstructured observability data while allowing for structured analytics as needed.
Cons
As admitted in the FAQ, high availability for indexing is only supported with Kafka sources, which restricts flexibility for other ingestion methods and could be a single point of failure.
Being in version 0.x (e.g., 0.8), it lacks the stability and feature completeness of alternatives, with metrics support still on the roadmap and potential breaking changes in updates.
While highly efficient for observability data, its architecture might not deliver the same performance for general-purpose search scenarios, such as complex document retrieval or e-commerce catalogs.
Requires expertise in managing cloud object storage and stateless components, which can involve more configuration and operational overhead compared to all-in-one solutions like Elasticsearch.
Open Source Alternative To
Quickwit-oss/quickwit is an open-source alternative to the following products:
Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases, built on Apache Lucene.
Loki is a horizontally-scalable, highly-available log aggregation system inspired by Prometheus, designed to store and query logs from applications and infrastructure.
Datadog is a monitoring and analytics platform for cloud-scale applications, providing monitoring of servers, databases, tools, and services.
Tempo is Grafana's open-source, high-volume distributed tracing backend that stores and queries trace data, helping with performance monitoring and troubleshooting.
Frequently Asked Questions
Related Projects
vectorA high-performance observability data pipeline.
OpenObserveOpen source observability platform for logs, metrics, traces, frontend monitoring, pipelines and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.
Scaphandre⚡ Energy consumption metrology agent. Let "scaph" dive and bring back the metrics that will help you make your systems and applications more sustainable !
bioyinoHigh performance and high-precision multithreaded StatsD server
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.