README.md

---
license: cc-by-nc-4.0
datasets:
- kyujinpy/KOR-gugugu-platypus-set
language:
- en
- ko
base_model:
- yanolja/KoSOLAR-10.7B-v0.2
pipeline_tag: text-generation
---

# KoSOLAR-v0.2-gugutypus-10.7B

<img src="logo.png" height=350, width=350>


---


## Model Details

**Model Developers**  
- DongGeon Lee ([oneonlee](https://huggingface.co/oneonlee))

**Model Architecture**  
- **KoSOLAR-v0.2-gugutypus-10.7B** is a instruction fine-tuned auto-regressive language model, based on the [SOLAR](https://huggingface.co/upstage/SOLAR-10.7B-v1.0) transformer architecture.

**Base Model**
- [yanolja/KoSOLAR-10.7B-v0.2](https://huggingface.co/yanolja/KoSOLAR-10.7B-v0.2)

**Training Dataset**
- [kyujinpy/KOR-gugugu-platypus-set](https://huggingface.co/datasets/kyujinpy/KOR-gugugu-platypus-set)

**Environments**
- Google Colab (Pro)
  - GPU : NVIDIA A100 40GB


---  


## Model comparisons

- **Ko-LLM leaderboard (YYYY/MM/DD)** [[link]](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)

| Model                 | Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
| --------------------- | ------- | ------ | ------------ | ------- | ------------- | --------------- |
| **KoSOLAR-gugutypus** | NaN     | NaN    | NaN          | NaN     | NaN           | NaN             |

<br>

- **AI-Harness evaluation** [[link]](https://github.com/Beomi/ko-lm-evaluation-harness)  


| Model                 | Copa   | Copa   | HellaSwag | HellaSwag | BoolQ  | BoolQ  | Sentineg | Sentineg |
| --------------------- | ------ | ------ | --------- | --------- | ------ | ------ | -------- | -------- |
|                       | 0-shot | 5-shot | 0-shot    | 5-shot    | 0-shot | 5-shot | 0-shot   | 5-shot   |
| **KoSOLAR-gugutypus** | NaN    | NaN    | NaN       | NaN       | NaN    | NaN    | NaN      | NaN      |

  
---


## Implementation Code

```python
### KoSOLAR-gugutypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "oneonlee/KoSOLAR-v0.2-gugutypus-10.7B"
model = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
tokenizer = AutoTokenizer.from_pretrained(repo)
```