Question 1

How do I measure memory usage with Benchee?

Accepted Answer

Enable memory measurement by setting the `memory_time` option in configuration, such as `memory_time: 2`, which runs separate tests for memory consumption. This tracks allocated memory during function execution, excluding other processes, and results are displayed in the output.

Question 2

Benchee vs Benchfella: which is better for Elixir benchmarking?

Accepted Answer

Benchee offers more features out-of-the-box, including multiple metrics, hooks, and plugin support, while Benchfella is simpler and focused on basic timing. Choose Benchee for comprehensive analysis and Benchfella for quick, straightforward benchmarks.

Question 3

How to save and compare benchmark results across runs?

Accepted Answer

Use the `save` and `load` configuration options; for example, `save: [path: 'results.benchee']` stores results, and `load: 'results.benchee'` loads them for comparison. This is ideal for tracking performance changes in CI/CD or across code branches.

Question 4

What are hooks in Benchee and when should I use them?

Accepted Answer

Hooks allow setup and teardown actions before or after benchmarks, such as initializing databases or clearing caches. Use them when state management is needed without affecting measurements, but they add complexity and are often unnecessary for simple functions.

Question 5

How to profile code after benchmarking with Benchee?

Accepted Answer

Set the `profile_after` option to true or specify a profiler like `:eprof`; Benchee runs the profiler after each scenario to identify bottlenecks. This integrates profiling seamlessly with benchmarking, but note that `after_each` hooks may not work with profiling in some Elixir versions.

Question 6

Best practices for using inputs in Benchee benchmarks?

Accepted Answer

Define inputs with the `inputs` option to test functions across different data sizes and structures, revealing performance characteristics. For example, use small, medium, and large lists to see how algorithms scale, which is recommended in the documentation for accurate analysis.

benchee

What is benchee?

Overview

Use Cases

Best For

Found a gem we're missing?

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions