GuacaMol
A Python package for benchmarking generative models in de novo molecular design.
What is GuacaMol?
GuacaMol is a Python package that provides benchmarks for evaluating generative models in de novo molecular design. It solves the problem of inconsistent evaluation in computational chemistry by offering standardized tests to measure how well models can generate novel, drug-like molecules. The package includes both distribution-learning and goal-directed benchmarks to assess different aspects of generative performance.
Computational chemists, machine learning researchers, and pharmaceutical scientists developing or using generative models for molecular design and drug discovery.
Researchers choose GuacaMol because it provides rigorous, reproducible benchmarks that enable fair comparison of generative models, includes standardized datasets, and offers containerized environments for consistent evaluation across different systems.
Overview
Benchmarks for generative chemistry
Use Cases
Best For
- Comparing different generative models for molecular design
- Evaluating new algorithms for de novo drug discovery
- Establishing performance baselines for generative chemistry research
- Reproducible benchmarking in computational chemistry
- Testing models against standardized chemical property optimization tasks
- Academic research requiring transparent evaluation metrics
Not Ideal For
- Projects requiring real-time molecular generation for interactive applications
- Teams heavily invested in non-Python cheminformatics or machine learning frameworks
- Small-scale academic projects lacking resources for Docker-based reproducible setups
- Applications focused solely on molecular property prediction without generation tasks
Pros & Cons
Pros
Provides both distribution-learning and goal-directed benchmarks, enabling fair comparison of generative models as outlined in the benchmarking models section and the accompanying paper.
Includes pre-processed ChEMBL datasets and Docker support for consistent benchmarking, with detailed instructions in the Data and Docker sections to ensure reproducibility.
Offers reference implementations of common generative models via a separate repository, helping establish performance baselines as mentioned in the Key Features and linked guacamol_baselines.
Features a public leaderboard for transparent comparison of model performances, encouraging community engagement and progress tracking as highlighted in the README.
Cons
Requires specific versions of RDKit and FCD libraries, with pinned dependencies that can lead to installation conflicts, as noted in the installation section and change log updates.
Docker is recommended for data generation and benchmarking, adding complexity for users unfamiliar with containerization, evident in the Docker commands and reproducibility warnings.
Primarily relies on ChEMBL datasets, which may not cover all chemical spaces, and custom dataset integration requires handling forbidden symbols, as mentioned in the data generation and change log.
Frequently Asked Questions
Related Projects
MOSESMolecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models
TAPE (Tasks Assessing Protein Embeddings)Tasks Assessing Protein Embeddings (TAPE), a set of five biologically relevant semi-supervised learning tasks spread across different domains of protein biology.
ProteinGymOfficial repository for the ProteinGym benchmarks
scIB (Single-cell Integration Benchmarks)Benchmarking analysis of data integration tools
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.