Hugging Face's logo

PracticeLLM
/
Custom-KoLLM-13B-v3

Text Generation
Transformers
PyTorch
Korean
llama
text-generation-inference
Inference Endpoints
Model card Files Community
kyujinpy commited on
Commit
b740421
•
1 Parent(s): 8cedaa4

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -4
README.md CHANGED
@@ -29,11 +29,23 @@ license: cc-by-nc-sa-4.0
29
 
30
  | Model | Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
31
  | --- | --- | --- | --- | --- | --- | --- |
32
- | ⭐My custom LLM 13B-v1⭐ | 50.19 | 45.99 | 56.93 | 41.78 | 41.66 | **64.58** |
33
- | ⭐My custom LLM 13B v2⭐ | 48.28 | 45.73 | 56.97 | 38.77 | 38.75 | 61.16 |
34
- | ⭐My custom LLM 13B v3⭐ | 46.40 | 44.71 | 56.89 | 40.86 | 44.22 | 45.34 |
35
-
36
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  # Implementation Code
38
  ```python
39
  ### KO-Platypus
 
29
 
30
  | Model | Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
31
  | --- | --- | --- | --- | --- | --- | --- |
32
+ | ⭐My custom LLM 13B-v1⭐ | **50.19** | **45.99** | 56.93 | **41.78** | 41.66 | **64.58** |
33
+ | ⭐My custom LLM 13B-v2⭐ | 48.28 | 45.73 | **56.97** | 38.77 | 38.75 | 61.16 |
34
+ | **⭐My custom LLM 13B-v3⭐** | 46.40 | 44.71 | 56.89 | 40.86 | **44.22** | 45.34 |
 
35
 
36
+ ---
37
+ # Model comparisons2
38
+ > AI-Harness evaluation; [link](https://github.com/Beomi/ko-lm-evaluation-harness)
39
+
40
+ | Model | Copa | Copa | HellaSwag | HellaSwag | BoolQ | BoolQ | Sentineg | Sentineg |
41
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- |
42
+ | | 0-shot | 5-shot | 0-shot | 5-shot | 0-shot | 5-shot | 0-shot | 5-shot |
43
+ | ⭐My custom LLM 13B-v1⭐ | 0.7987 | 0.8269 | 0.4994 | 0.5660 | 0.3343 | 0.5060 | **0.6984** | 0.9723 |
44
+ | ⭐My custom LLM 13B-v2⭐ | 0.7938 | 0.8209 | 0.4978 | 0.4893 | 0.3343 | 0.5614 | 0.6283 | 0.9773 |
45
+ | **⭐My custom LLM 13B-v3⭐** | **0.8107** | 0.8359 | **0.5176** | 0.5182 | **0.6702** | 0.7851 | 0.5241 | 0.9698 |
46
+ | [beomi/llama-2-koen-13b](https://huggingface.co/beomi/llama-2-koen-13b) | 0.7768 | 0.8128 | 0.4999 | 0.5127 | 0.3988 | 0.7038 | 0.5870 | 0.9748 |
47
+
48
+ ---
49
  # Implementation Code
50
  ```python
51
  ### KO-Platypus