Scientist
A Ruby library for carefully refactoring critical paths by comparing old and new code behavior in production.
What is Scientist?
Scientist is a Ruby library designed to safely refactor critical code paths in production. It allows developers to run new code alongside existing code, compare results, measure performance differences, and catch errors without impacting users. This helps ensure that refactored code behaves correctly under real load before fully replacing the old implementation.
Ruby developers working on large applications who need to refactor critical, high-risk code paths with minimal disruption. It's especially useful for teams managing legacy systems or introducing major architectural changes.
Developers choose Scientist because it provides a structured, low-risk way to validate code changes in production. Unlike traditional testing, it offers real-time comparison, performance insights, and error handling, reducing the fear of breaking existing functionality during refactors.
Overview
:microscope: A Ruby library for carefully refactoring critical paths.
Use Cases
Best For
- Refactoring permission or authentication logic in web applications
- Comparing performance of new algorithms against legacy code
- Safely migrating from one data service to another
- Validating API response changes during backend updates
- Testing new database queries alongside existing ones
- Gradually rolling out new features with automatic rollback on mismatch
Not Ideal For
- Projects not using Ruby or requiring cross-language support
- Code paths that modify data or have irreversible side effects
- Applications where even minimal runtime overhead is unacceptable
- Teams lacking production monitoring systems like statsd or redis
Pros & Cons
Pros
Runs new candidate code alongside old control code in production, comparing results with == or custom logic to catch mismatches without affecting users, as demonstrated in the basic experiment setup.
Measures wall time and CPU time for both control and candidate paths, providing concrete data to assess performance impacts during refactoring.
Safely captures exceptions in candidate code via RESCUES list configuration and offers a raised callback for custom error logging, preventing experiment failures from cascading.
Allows custom compare and compare_errors blocks for nuanced validation, plus ignore blocks to filter known mismatches, adapting to complex refactoring scenarios.
Includes raise_on_mismatches to fail tests when results diverge, ensuring refactored code matches expected behavior during development.
Cons
Explicitly warns against using Scientist for methods that change data, as enabling logic isn't guaranteed to run every time, risking inconsistent states.
Requires implementing a custom experiment class with methods like enabled?, publish, and context, adding development effort before gaining value.
Runs both control and candidate code paths sequentially in random order, doubling execution time and resource usage during experiments, which can impact latency-sensitive applications.
Full utility depends on integrating publish results with systems like statsd or redis for timing and mismatch storage, which may not be available in all environments.
Frequently Asked Questions
Related Projects
brakemanA static analysis security vulnerability scanner for Ruby on Rails applications
SimpleCovCode coverage for Ruby with a powerful configuration library and automatic merging of coverage across test suites
reekCode smell detector for Ruby
SorbetA fast, powerful type checker designed for Ruby
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.