SciBERT

Uses a custom scivocab built from 3.1 billion tokens of scientific papers, reducing vocabulary mismatch and improving handling of technical terms like chemical names or academic jargon.

Achieves state-of-the-art results on multiple benchmarks such as BC5CDR and SciERC, as shown by numerous Papers with Code badges for tasks like named entity recognition and relation extraction.

Provides models in TensorFlow, PyTorch (via AllenNLP), and Hugging Face formats, ensuring easy adoption within popular NLP ecosystems without vendor lock-in.

Includes evaluation code and datasets for various scientific NLP tasks, facilitating reproducibility and allowing researchers to validate and extend findings directly from the repository.