metadata

tags:
  - model_hub_mixin
  - pytorch_model_hub_mixin

sCellTransformer

sCellTransformer (sCT) is a long-range foundation model designed for zero-shot prediction tasks in single-cell RNA-seq and spatial transcriptomics data. It processes raw gene expression profiles across multiple cells to predict discretized gene expression levels for unseen cells without retraining. The model handles up to 20,000 protein-coding genes and outputs around a million gene expression tokens, mitigating the sparsity typical in single-cell datasets.

Developed by: InstaDeep

Model Sources

Repository: Nucleotide Transformer
Paper: A long range foundation model for zero-shot predictions in single-cell and spatial transcriptomics data

How to use

Until its next release, the transformers library needs to be installed from source with the following command in order to use the models. PyTorch should also be installed.

pip install --upgrade git+https://github.com/huggingface/transformers.git
pip install torch

A small snippet of code is given here in order to infer with the model from random input.

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "InstaDeepAI/sCellTransformer",
    trust_remote_code=True,
)
num_cells = model.config.num_cells
dummy_gene_expressions = torch.randint(0, 5, (1, 19968 * num_cells))
torch_output = model(dummy_gene_expressions)