More quality results
Browse files
README.md
CHANGED
@@ -185,6 +185,7 @@ Okay, the user is asking if I can talk to them. First, I need to clarify that I
|
|
185 |
| mmlu_pro | - | - |
|
186 |
| bbh | - | - |
|
187 |
| **Reasoning** | | |
|
|
|
188 |
| gpqa_main_zeroshot | - | 27.46 |
|
189 |
| **Multilingual** | | |
|
190 |
| m_mmlu | - | - |
|
@@ -203,7 +204,7 @@ Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation
|
|
203 |
|
204 |
## baseline
|
205 |
```Shell
|
206 |
-
lm_eval --model hf --model_args pretrained=
|
207 |
```
|
208 |
|
209 |
## int8 dynamic activation and int4 weight quantization (8da4w)
|
|
|
185 |
| mmlu_pro | - | - |
|
186 |
| bbh | - | - |
|
187 |
| **Reasoning** | | |
|
188 |
+
| hellaswag | - | 54.39 |
|
189 |
| gpqa_main_zeroshot | - | 27.46 |
|
190 |
| **Multilingual** | | |
|
191 |
| m_mmlu | - | - |
|
|
|
204 |
|
205 |
## baseline
|
206 |
```Shell
|
207 |
+
lm_eval --model hf --model_args pretrained=HuggingFaceTB/SmolLM3-3B --tasks mmlu --device cuda:0 --batch_size auto
|
208 |
```
|
209 |
|
210 |
## int8 dynamic activation and int4 weight quantization (8da4w)
|