guangy10 commited on
Commit
3b45b2f
·
verified ·
1 Parent(s): adde4e0

More quality results

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -185,6 +185,7 @@ Okay, the user is asking if I can talk to them. First, I need to clarify that I
185
  | mmlu_pro | - | - |
186
  | bbh | - | - |
187
  | **Reasoning** | | |
 
188
  | gpqa_main_zeroshot | - | 27.46 |
189
  | **Multilingual** | | |
190
  | m_mmlu | - | - |
@@ -203,7 +204,7 @@ Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation
203
 
204
  ## baseline
205
  ```Shell
206
- lm_eval --model hf --model_args pretrained=pytorch/SmolLM3-3B-8da4w --tasks mmlu --device cuda:0 --batch_size auto
207
  ```
208
 
209
  ## int8 dynamic activation and int4 weight quantization (8da4w)
 
185
  | mmlu_pro | - | - |
186
  | bbh | - | - |
187
  | **Reasoning** | | |
188
+ | hellaswag | - | 54.39 |
189
  | gpqa_main_zeroshot | - | 27.46 |
190
  | **Multilingual** | | |
191
  | m_mmlu | - | - |
 
204
 
205
  ## baseline
206
  ```Shell
207
+ lm_eval --model hf --model_args pretrained=HuggingFaceTB/SmolLM3-3B --tasks mmlu --device cuda:0 --batch_size auto
208
  ```
209
 
210
  ## int8 dynamic activation and int4 weight quantization (8da4w)