|
--- |
|
library_name: transformers |
|
tags: |
|
- quantization |
|
- phi-3 |
|
license: apache-2.0 |
|
language: |
|
- en |
|
--- |
|
|
|
# Model Card for Model ID |
|
A 4-bits double-quantized version of [ernestoBocini/Phi3-mini-DPO-Tuned](https://huggingface.co/ernestoBocini/Phi3-mini-DPO-Tuned). |
|
|
|
## Model Details |
|
|
|
This is a Phi-3-mini-4k-instruct fine-tuned with SFT and DPO on STEM domains, and finally quantized to a 4 bits precision, to serve the purpose of being an AI university tutor. |
|
|
|
Quantization config used: |
|
```python |
|
BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_quant_type="nf4", |
|
bnb_4bit_use_double_quant=True, |
|
bnb_4bit_compute_dtype=torch.bfloat16 |
|
) |
|
``` |
