A Go package implementing Bloom filters, a space-efficient probabilistic data structure for set membership queries.
Bloom is a Go library that provides an implementation of Bloom filters, a probabilistic data structure used to test whether an element is a member of a set. It offers a memory-efficient way to handle membership queries with a configurable false-positive rate, making it suitable for applications where storage optimization is critical. The library is designed for performance and reliability, leveraging murmurhash for fast hashing and including features like serialization and parameter estimation.
Go developers building systems that require efficient set membership testing with large datasets, such as database systems, caching layers, or network applications where minimizing memory usage is a priority. It is also suitable for projects like Milvus and Beego that integrate Bloom filters for performance optimization.
Developers choose this library for its empirical tuning, practical performance optimizations, and ease of use with configurable accuracy and serialization support. Its integration with popular systems and thread-safe design (with proper synchronization) make it a reliable choice over custom implementations.
Go package implementing Bloom filters, used by Milvus and Beego.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Uses significantly less memory than storing full sets, ideal for large-scale data handling, as highlighted in the README's emphasis on storage optimization.
Allows precise control over capacity and false-positive rate during construction, with empirical tuning via functions like EstimateFalsePositiveRate for validation.
Enables reading and writing Bloom filters to bytes for persistence, with performance tips for I/O buffering using bufio, as detailed in the serialization section.
Leverages murmurhash for fast non-cryptographic hashing and is designed for real-world use, evidenced by integration in systems like Milvus and Beego.
Bloom filters are not dynamic; capacity must be known upfront, which restricts use in scenarios with unpredictable data growth, as the README explicitly warns.
Default implementation is unsynchronized for performance, requiring manual synchronization like mutexes for concurrent writes, adding complexity for multi-threaded applications.
As a Go-only library, it's not suitable for projects using other programming languages or requiring cross-platform compatibility beyond the Go ecosystem.