OPEA
/

Safetensors
qwen2
4-bit precision
intel/auto-round
cicdatopea commited on
Commit
20e4b60
1 Parent(s): c3d1f4b
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -39,7 +39,7 @@ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
39
  generated_ids = model.generate(
40
  **model_inputs,
41
  max_new_tokens=512,
42
- do_sample=False
43
  )
44
  generated_ids = [
45
  output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
@@ -192,7 +192,7 @@ I think the answer is three.
192
  pip3 install lm-eval==0.4.5
193
 
194
  ```bash
195
- auto-round --model "Intel/OPEA/QwQ-32B-Preview-int4-sym-mixed-inc" --eval --eval_bs 16 --tasks leaderboard_ifeval,leaderboard_mmlu_pro,gsm8k,lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,arc_easy,arc_challenge,cmmlu,ceval-valid
196
  ```
197
 
198
  | Metric | BF16 | INT4 | iter1000 nsamples 512 |
 
39
  generated_ids = model.generate(
40
  **model_inputs,
41
  max_new_tokens=512,
42
+ do_sample=False ##change this to follow official usage
43
  )
44
  generated_ids = [
45
  output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
 
192
  pip3 install lm-eval==0.4.5
193
 
194
  ```bash
195
+ auto-round --model "OPEA/QwQ-32B-Preview-int4-sym-mixed-inc" --eval --eval_bs 16 --tasks leaderboard_ifeval,leaderboard_mmlu_pro,gsm8k,lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,arc_easy,arc_challenge,cmmlu,ceval-valid
196
  ```
197
 
198
  | Metric | BF16 | INT4 | iter1000 nsamples 512 |