README.md

---
language:
- en
datasets:
- kyujinpy/orca_math_dpo
pipeline_tag: text-generation
license: cc-by-nc-sa-4.0
---

# **Sakura-SOLRCA-Math-Instruct-DPO-v2**  
<img src='./sakura.png' width=512>

## Model Details

**Model Developers** Kyujin Han (kyujinpy)

**Method**  
Using DPO method.  
With [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs) and [argilla/distilabel-math-preference-dpo](https://huggingface.co/datasets/argilla/distilabel-math-preference-dpo).  

I shared the merge version [kyujinpy/orca_math_dpo](https://huggingface.co/datasets/kyujinpy/orca_math_dpo).  
     
I will share the information about my model. (training and code)  
Please see: ⭐[Sakura-SOLAR(will update)]().  

# **Model Benchmark**  

## Open leaderboard
- Follow up as [link](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).  

| Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Sakura-SOLRCA-Math-Instruct-DPO-v2 | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| Sakura-SOLRCA-Math-Instruct-DPO-v1 | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| Sakura-SOLRCA-Instruct-DPO | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| Sakura-SOLAR-Instruct-DPO-v2 | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| Sakura-SOLAR-Instruct-DPO-v1 | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| [kyujinpy/Sakura-SOLAR-Instruct](https://huggingface.co/kyujinpy/Sakura-SOLAR-Instruct) | NaN | NaN | NaN | NaN | NaN | NaN | NaN |

   
# Implementation Code
```python
### KO-Platypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "kyujinpy/Sakura-SOLRCA-Math-Instruct-DPO-v2"
OpenOrca = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)
```

---