|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: Qwen/Qwen3-1.7B |
|
|
tags: |
|
|
- biology |
|
|
- protein |
|
|
- gene-ontology |
|
|
- GO-terms |
|
|
--- |
|
|
|
|
|
# Qwen3-1.7B-GO |
|
|
|
|
|
Qwen3 1.7B model enhanced with pre-trained Gene Ontology (GO) term embeddings. |
|
|
|
|
|
## Model Description |
|
|
|
|
|
This model is based on Qwen3 1.7B and includes: |
|
|
- Pre-trained embeddings for GO terms |
|
|
- Special tokens for protein sequence handling |
|
|
- Fine-tuned on GO term descriptions and relationships |
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("wanglab/Qwen3-1.7B-go") |
|
|
tokenizer = AutoTokenizer.from_pretrained("wanglab/Qwen3-1.7B-go") |
|
|
|
|
|
# Example with GO terms |
|
|
text = "What is the function of GO:0008150?" |
|
|
inputs = tokenizer(text, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=100) |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
``` |
|
|
|
|
|
## GO Terms |
|
|
|
|
|
The model includes embeddings for Gene Ontology terms, allowing it to understand and reason about: |
|
|
- Biological processes (GO:0008150) |
|
|
- Molecular functions (GO:0003674) |
|
|
- Cellular components (GO:0005575) |
|
|
|
|
|
## Training |
|
|
|
|
|
GO embeddings were pre-trained using QLora on GO term descriptions and relationships. |
|
|
|
