README.md

metadata

library_name: peft
datasets:
  - InstaDeepAI/nucleotide_transformer_downstream_tasks_revised
metrics:
  - f1
base_model:
  - tattabio/gLM2_150M
model-index:
  - name: alejandralopezsosa/gLM2_150M-promoter_tata-lora
    results:
      - task:
          type: sequence-classification
        dataset:
          type: InstaDeepAI/nucleotide_transformer_downstream_tasks_revised
          name: nucleotide_transformer_downstream_tasks_revised
          config: promoter_tata
          split: test
          revision: c8c94743d3d2838b943398ee676247ac2f774122
        metrics:
          - type: f1
            value: 0.9811

gLM2 LoRA adapter for TATA promoter recognition

This model demonstrates the use of gLM2_150M embeddings for downstream classification. The model is fine-tuned using LoRA and obtains an F1 score of 98.11% on the TATA promoter task from the Nucleotide Transformer benchmarks.

How to Get Started with the Model

Use the code below to use the model for inference:

from peft import PeftModel
from transformers import AutoConfig, AutoModelForSequenceClassification, AutoModel

glm2 = "tattabio/gLM2_150M"
adapter = "alejandralopezsosa/gLM2_150M-promoter_tata-lora"

load_kwargs = {
    'trust_remote_code': True,
    'torch_dtype': torch.bfloat16,
}

config = AutoConfig.from_pretrained(adapter, **load_kwargs)
base_model = AutoModelForSequenceClassification.from_config(config, **load_kwargs)
base_model.glm2 = AutoModel.from_pretrained("tattabio/gLM2_150M", **load_kwargs)

model = PeftModel.from_pretrained(base_model, adapter)