How to preprocess scRNA-seq data for scBERT?

You must revise gene symbols according to the NCBI Gene database (updated Jan. 10, 2020), remove unmatched and duplicated genes, then normalize using scanpy's sc.pp.normalize_total and sc.pp.log1p methods, as detailed in preprocess.py.

scBERT vs traditional tools like Seurat for cell annotation?

scBERT uses a deep learning approach that may better handle batch effects and gene interactions, but Seurat is more established and interpretable; the choice depends on data complexity, computational resources, and user expertise in machine learning.

How long does it take to fine-tune scBERT on custom data?

Fine-tuning time varies, but the README notes the demo task runs in about 4 hours on a normal desktop, depending on dataset size and hardware capabilities.

Can scBERT work with single-cell ATAC-seq or other omics data?

No, scBERT is specifically designed for scRNA-seq data, as it relies on gene expression patterns; adapting it to other data types would require significant modification and retraining.

What hardware is needed to run scBERT efficiently?

A desktop with sufficient RAM and a GPU is recommended for faster performance; the README mentions typical run times on a 'normal' desktop, but larger datasets may require more powerful systems.

How to detect novel cell types with scBERT?

Use the predict.py script with the --novel_type True flag and optionally set --unassign_thres for a custom threshold, as described in the README, to identify cells with low prediction confidence.

scBERT — BERT Model for Single-Cell Analysis

What is scBERT?

scBERT is a BERT-based foundation model pretrained on large-scale single-cell RNA sequencing (scRNA-seq) data for automated cell type annotation. It addresses common challenges in scRNA-seq analysis, such as batch effects and reliance on curated marker gene lists, by leveraging deep learning to capture gene-gene interactions. The model follows a pre-train and fine-tune approach, enabling accurate annotation on user-specific datasets.

Target Audience

Bioinformaticians, computational biologists, and researchers working with single-cell RNA sequencing data who need reliable, scalable cell type annotation tools. It is suited for those familiar with deep learning frameworks and Python-based bioinformatics workflows.

Value Proposition

Developers choose scBERT because it provides a state-of-the-art, pretrained deep learning model specifically designed for scRNA-seq data, offering improved accuracy over traditional methods by effectively handling batch effects and leveraging latent gene interactions without requiring extensive manual curation.

Overview

scBERT is a deep learning model designed to address the challenges of cell type annotation in single-cell RNA sequencing (scRNA-seq) data. It leverages the pre-train and fine-tune paradigm to overcome issues like batch effects, reliance on curated marker genes, and inefficient use of gene-gene interaction information.

Key Features

BERT-based Architecture — Uses a transformer encoder (PerformerLM) pretrained on massive unlabeled scRNA-seq data to understand gene-gene interactions.
Pre-train and Fine-tune — Pretrained on large-scale data for general understanding, then fine-tuned on specific datasets for accurate cell annotation.
Novel Cell Type Detection — Includes functionality to detect novel cell types by thresholding predicted probabilities.
Batch Effect Handling — Designed to better manage batch effects compared to traditional annotation algorithms.
Scalable Inference — Can infer cell types for thousands of cells efficiently (e.g., ~25 minutes for 10,000 cells on a desktop).

Philosophy

scBERT applies the success of large-scale pretrained language models to computational biology, aiming to provide a robust, data-driven foundation for cell type annotation that reduces reliance on manually curated knowledge.

scBERT

What is scBERT?

Overview

Key Features

Philosophy

Related Projects

Found a gem we're missing?

Use Cases

Best For

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions