A Go implementation of Roaring bitmaps, a compressed bitmap data structure for fast set operations on large integer datasets.
Roaring is a Go implementation of the Roaring bitmap data structure, which provides compressed storage and fast set operations (e.g., unions, intersections) on large collections of integers. It solves the problem of efficiently representing and querying integer sets in memory-constrained environments like databases and search engines, where traditional bitsets or hash sets become impractical.
Developers building high-performance systems that require efficient set operations on large integer datasets, such as database engineers (e.g., for InfluxDB, Bleve), search engine contributors, or data analytics tool creators.
Roaring offers better compression and faster random access compared to run-length-encoded bitmap alternatives (e.g., WAH, EWAH), with cross-language compatibility and production-ready reliability. Its adaptive container approach ensures optimal performance across diverse data distributions.
Roaring bitmaps in Go (golang), used by InfluxDB, Bleve, DataDog
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Uses arrays, bitmaps, or run-length encoding to minimize memory usage dynamically based on data distribution, as described in the README, ensuring optimal performance for both sparse and dense datasets.
Includes functions like ParOr and ParAnd that leverage multiple CPU cores for accelerated set computations, enabling high-throughput processing in analytics and indexing systems.
32-bit bitmaps are binary compatible with Java and C/C++ implementations, allowing seamless data exchange between different systems, as highlighted in the format specification.
Widely adopted in major projects like Apache Lucene, InfluxDB, and Elasticsearch, ensuring robust performance and real-world testing in high-demand environments.
Accessing bitmaps from multiple goroutines requires explicit synchronization, such as using channels or mutexes, which adds complexity and potential for bugs in concurrent applications.
The 64-bit extension is not guaranteed to be cross-language compatible, limiting interoperability for applications requiring larger integers, as noted in the documentation.
Only supports integer sets, making it unsuitable for general-purpose collections with other data types, which can restrict its use in broader application scenarios.