File size: 656 Bytes
d57a808 ddcbd88 d57a808 954a6d6 d57a808 2b025ae d57a808 2b025ae d57a808 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
---
base_model: google/gemma-2-2b
datasets: mlabonne/TheTome
---
# Distil Gemma 2 2b
This model is a gemma 2 2b model distilled from google/gemma-2-9b-it and finetuned on the tome.
## Prompt Template
ChatML
```
<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant
```
## Training Information
This model trained on 8x Nvidia H100 NVL for the equivalent of 120 GPU hours.
+ Loss Achieved: 0.27
+ Epochs: 3
Checkpoints are available in the repo to continue training
## Evals
IN PROGRESS |