Upload README.md
Browse files
README.md
CHANGED
@@ -32,7 +32,8 @@ Same as [KO-Platypus2](https://github.com/Marker-Inc-Korea/CoT-llama2).
|
|
32 |
**Training Dataset**
|
33 |
I use [KOpen-platypus dataset](https://huggingface.co/datasets/kyujinpy/KOpen-platypus).
|
34 |
I use A100 GPU 40GB and COLAB, when trianing.
|
35 |
-
|
|
|
36 |
# **Model Bechmark1**
|
37 |
|
38 |
## KO-LLM leaderboard
|
@@ -64,7 +65,7 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
64 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7436 | 0.7927 | 0.8037 | 0.8259 |
|
65 |
| [KO-platypus2-7B-EX](https://huggingface.co/kyujinpy/KO-Platypus2-7B-ex) | 0.7509 | 0.7899 | 0.8029 | 0.8290 |
|
66 |
| [KoT-platypus2-7B](https://huggingface.co/kyujinpy/KoT-platypus2-7B) | 0.7517 | 0.7868 | 0.8009 | 0.8239 |
|
67 |
-
| **Poly-platypus-ko-12.8b(ours)** | 0.7876 | 0.8099 |
|
68 |
|
69 |
> Natural Language Inference (NLI; 자연어 추론 평가)
|
70 |
### HellaSwag (F1)
|
@@ -76,7 +77,7 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
76 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4562 | 0.4657 | 0.4698 | 0.4774 |
|
77 |
| [KO-platypus2-7B-EX](https://huggingface.co/kyujinpy/KO-Platypus2-7B-ex) | 0.4571 | 0.4461 | 0.4371 | 0.4525 |
|
78 |
| [KoT-platypus2-7B](https://huggingface.co/kyujinpy/KoT-platypus2-7B) | 0.4432 | 0.4382 | 0.4550 | 0.4534 |
|
79 |
-
| **Poly-platypus-ko-12.8b(ours)** | 0.4838 | 0.4858 |
|
80 |
|
81 |
> Question Answering (QA)
|
82 |
### BoolQ (F1)
|
@@ -88,7 +89,7 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
88 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.5786 | 0.6977 | 0.7084 | 0.7144 |
|
89 |
| [KO-platypus2-7B-EX](https://huggingface.co/kyujinpy/KO-Platypus2-7B-ex) | 0.6028 | 0.6979 | 0.7016 | 0.6988 |
|
90 |
| [KoT-platypus2-7B](https://huggingface.co/kyujinpy/KoT-platypus2-7B) | 0.6142 | 0.6757 | 0.6839 | 0.6878 |
|
91 |
-
| **Poly-platypus-ko-12.8b(ours)** | 0.4888 | 0.6520 |
|
92 |
|
93 |
> Classification
|
94 |
### SentiNeg (F1)
|
@@ -100,7 +101,7 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
100 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4594 | 0.7611 | 0.7276 | 0.9370 |
|
101 |
| [KO-platypus2-7B-EX](https://huggingface.co/kyujinpy/KO-Platypus2-7B-ex) | 0.5821 | 0.7653 | 0.7991 | 0.8643 |
|
102 |
| [KoT-platypus2-7B](https://huggingface.co/kyujinpy/KoT-platypus2-7B) | 0.6127 | 0.7199 | 0.7531 | 0.8381 |
|
103 |
-
| **Poly-platypus-ko-12.8b(ours)** | 0.8490 | 0.9597 |
|
104 |
|
105 |
# Implementation Code
|
106 |
```python
|
|
|
32 |
**Training Dataset**
|
33 |
I use [KOpen-platypus dataset](https://huggingface.co/datasets/kyujinpy/KOpen-platypus).
|
34 |
I use A100 GPU 40GB and COLAB, when trianing.
|
35 |
+
|
36 |
+
---
|
37 |
# **Model Bechmark1**
|
38 |
|
39 |
## KO-LLM leaderboard
|
|
|
65 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7436 | 0.7927 | 0.8037 | 0.8259 |
|
66 |
| [KO-platypus2-7B-EX](https://huggingface.co/kyujinpy/KO-Platypus2-7B-ex) | 0.7509 | 0.7899 | 0.8029 | 0.8290 |
|
67 |
| [KoT-platypus2-7B](https://huggingface.co/kyujinpy/KoT-platypus2-7B) | 0.7517 | 0.7868 | 0.8009 | 0.8239 |
|
68 |
+
| **Poly-platypus-ko-12.8b(ours)** | 0.7876 | 0.8099 | 0.8008 | 0.8239 |
|
69 |
|
70 |
> Natural Language Inference (NLI; 자연어 추론 평가)
|
71 |
### HellaSwag (F1)
|
|
|
77 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4562 | 0.4657 | 0.4698 | 0.4774 |
|
78 |
| [KO-platypus2-7B-EX](https://huggingface.co/kyujinpy/KO-Platypus2-7B-ex) | 0.4571 | 0.4461 | 0.4371 | 0.4525 |
|
79 |
| [KoT-platypus2-7B](https://huggingface.co/kyujinpy/KoT-platypus2-7B) | 0.4432 | 0.4382 | 0.4550 | 0.4534 |
|
80 |
+
| **Poly-platypus-ko-12.8b(ours)** | 0.4838 | 0.4858 | 0.5005 | 0.5062 |
|
81 |
|
82 |
> Question Answering (QA)
|
83 |
### BoolQ (F1)
|
|
|
89 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.5786 | 0.6977 | 0.7084 | 0.7144 |
|
90 |
| [KO-platypus2-7B-EX](https://huggingface.co/kyujinpy/KO-Platypus2-7B-ex) | 0.6028 | 0.6979 | 0.7016 | 0.6988 |
|
91 |
| [KoT-platypus2-7B](https://huggingface.co/kyujinpy/KoT-platypus2-7B) | 0.6142 | 0.6757 | 0.6839 | 0.6878 |
|
92 |
+
| **Poly-platypus-ko-12.8b(ours)** | 0.4888 | 0.6520 | 0.6568 | 0.6835 |
|
93 |
|
94 |
> Classification
|
95 |
### SentiNeg (F1)
|
|
|
101 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4594 | 0.7611 | 0.7276 | 0.9370 |
|
102 |
| [KO-platypus2-7B-EX](https://huggingface.co/kyujinpy/KO-Platypus2-7B-ex) | 0.5821 | 0.7653 | 0.7991 | 0.8643 |
|
103 |
| [KoT-platypus2-7B](https://huggingface.co/kyujinpy/KoT-platypus2-7B) | 0.6127 | 0.7199 | 0.7531 | 0.8381 |
|
104 |
+
| **Poly-platypus-ko-12.8b(ours)** | 0.8490 | 0.9597 | 0.9723 | 0.9847 |
|
105 |
|
106 |
# Implementation Code
|
107 |
```python
|