metadata
language: protein
tags:
- protein
datasets:
- uniref-100
RITA-M
RITA is a family of autoregressive protein models, developed by a collaboration of Lighton, the OATML group at Oxford, and the Debbie Marks Lab at Harvard.
| Model | #Params | d_model | layers | lm loss uniref-100 |
|---|---|---|---|---|
| Small | 85M | 768 | 12 | 2.31 |
| Medium | 300M | 1024 | 24 | 2.01 |
| Large | 680M | 1536 | 24 | 1.82 |
| XLarge | 1.2B | 2048 | 24 | 1.70 |
Usage
Instantiate a model like so:
from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_m, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_m")
for generation use we support pipelines:
rita_gen = pipeline('text-generation', model=model, tokenizer = tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, num_return_sequences=2, eos_token_id=2)
for seq in sequences:
print(f"seq: {seq['generated_text'].replace(' ', '')}")