How do I use a custom hash function with Bloomex?

Install a hash function package like Murmur, then pass the function reference when creating the bloom filter, as shown in the README example with &Murmur.hash_x86_128/1. This allows flexibility but requires managing additional dependencies.

Is Bloomex better than other Elixir bloom filter libraries?

Bloomex excels in scalability and configurability with no external dependencies, but alternatives like 'bloom' might offer different trade-offs in performance or features. Compare based on your specific needs for error rates, memory usage, and integration ease.

What are the performance benchmarks for Bloomex in production?

The README doesn't provide benchmarks, so performance depends on factors like dataset size and hash function choice. For high-performance needs, test in your environment, as pure Elixir implementation may have overheads versus native code.

How to handle false positives when using Bloomex?

False positives are inherent to bloom filters; tune parameters like error rate and capacity in Bloomex.scalable() to minimize them, and design your application to handle occasional false positives gracefully, such as with fallback checks.

Can I save a bloom filter from Bloomex to disk?

Bloomex doesn't include built-in serialization; you'd need to implement custom serialization using Elixir's serialization tools, which adds complexity for persistence and sharing between processes or systems.

Best practices for tuning Bloomex parameters?

Start with conservative error rates and scaling factors based on expected dataset growth; use the configurable options in Bloomex.scalable() to adjust memory vs. accuracy, and monitor false positive rates in testing to refine settings.

bloomex — Scalable Bloom Filters in Elixir

What is bloomex?

Bloomex is a pure Elixir library that implements Scalable Bloom Filters, a probabilistic data structure for efficient set membership testing. It provides space-efficient storage with controlled false positive rates, making it suitable for applications where exact answers are not required but performance and memory usage are critical.

Target Audience

Elixir developers building applications that require efficient probabilistic set membership checks, such as caching systems, duplicate detection, or network security tools.

Value Proposition

Developers choose Bloomex for its idiomatic Elixir interface, no external dependencies, and configurable parameters that allow tuning of capacity, error rates, and hash functions to balance memory and accuracy.

:hibiscus: A pure Elixir implementation of Scalable Bloom Filters

Use Cases

Best For

Implementing scalable duplicate detection in data processing pipelines
Building memory-efficient caching layers in Elixir applications
Creating probabilistic security filters for network traffic analysis
Developing systems that require approximate set membership queries with bounded false positives
Integrating bloom filters into Elixir projects without external dependencies
Tuning bloom filter parameters like initial capacity, error rates, and scaling factor for specific use cases

Not Ideal For

Applications requiring exact set membership with zero false positives, such as financial transaction validation
High-throughput, latency-sensitive systems where C or Rust-based bloom filter implementations offer better performance
Projects not using the BEAM ecosystem or Elixir, as it's a pure Elixir library
Use cases with small, static datasets where a simple hash table or set would be more memory-efficient and faster

Pros & Cons

Pros

Pure Elixir Implementation

No external dependencies ensure easy integration into Elixir projects, as highlighted in the README's key features, simplifying deployment and maintenance.

Configurable Scalability

Automatically grows to accommodate more elements while maintaining bounded false positive rates, based on scalable bloom filter principles, making it suitable for dynamic datasets.

Flexible Hashing Strategies

Supports pluggable hash functions like Murmur, allowing customization for performance or cryptographic needs, as demonstrated in the README examples.

Tunable Accuracy and Memory

Allows adjustment of initial capacity, error rates, and scaling factor to balance memory usage and false positive probability, per the configurable parameters feature.

Cons

Performance Overheads

As a pure Elixir library, it may have higher CPU and memory usage compared to native implementations, potentially impacting latency in high-throughput scenarios.

Limited Advanced Features

Focuses on core bloom filter functionality and lacks built-in support for features like serialization, persistence, or distributed operations, which some applications might require.

External Hash Dependency

For optimal hashing, users must install and configure external packages like Murmur, adding complexity to setup and maintenance, as noted in the README examples.

Frequently Asked Questions

What is bloomex?

Target Audience

Elixir developers building applications that require efficient probabilistic set membership checks, such as caching systems, duplicate detection, or network security tools.

Value Proposition

Use Cases

Best For

Implementing scalable duplicate detection in data processing pipelines
Building memory-efficient caching layers in Elixir applications
Creating probabilistic security filters for network traffic analysis
Developing systems that require approximate set membership queries with bounded false positives
Integrating bloom filters into Elixir projects without external dependencies
Tuning bloom filter parameters like initial capacity, error rates, and scaling factor for specific use cases

Not Ideal For

Applications requiring exact set membership with zero false positives, such as financial transaction validation
High-throughput, latency-sensitive systems where C or Rust-based bloom filter implementations offer better performance
Projects not using the BEAM ecosystem or Elixir, as it's a pure Elixir library
Use cases with small, static datasets where a simple hash table or set would be more memory-efficient and faster

Pros & Cons

Pros

Pure Elixir Implementation

No external dependencies ensure easy integration into Elixir projects, as highlighted in the README's key features, simplifying deployment and maintenance.

Configurable Scalability

Automatically grows to accommodate more elements while maintaining bounded false positive rates, based on scalable bloom filter principles, making it suitable for dynamic datasets.

Flexible Hashing Strategies

Supports pluggable hash functions like Murmur, allowing customization for performance or cryptographic needs, as demonstrated in the README examples.

Tunable Accuracy and Memory

Allows adjustment of initial capacity, error rates, and scaling factor to balance memory usage and false positive probability, per the configurable parameters feature.

Cons

Performance Overheads

As a pure Elixir library, it may have higher CPU and memory usage compared to native implementations, potentially impacting latency in high-throughput scenarios.

Limited Advanced Features

Focuses on core bloom filter functionality and lacks built-in support for features like serialization, persistence, or distributed operations, which some applications might require.

External Hash Dependency

For optimal hashing, users must install and configure external packages like Murmur, adding complexity to setup and maintenance, as noted in the README examples.

Frequently Asked Questions

bloomex

What is bloomex?

Overview

Use Cases

Best For

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions

Related Projects

Found a gem we're missing?

bloomex

What is bloomex?

Overview

Use Cases

Best For

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions

Related Projects

Found a gem we're missing?