pytorch
/

Qwen3-32B-float8dq

Text Generation

text-generation-inference

Model card Files Files and versions Community

jerryzh168 commited on May 22

Commit

8213695

·

verified ·

1 Parent(s): 93746d0

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -165,7 +165,7 @@ lm_eval --model hf --model_args pretrained=$MODEL --tasks mmlu --device cuda:0 -
 | Memory (tested on H100)          |                |                               |
 |----------------------------------|----------------|-------------------------------|
 |                                  | Qwen3-32B      | Qwen3-32B-float8dq            |
-| Peak Memory                      | 65.72 GB       | 34.54 GB (-47.44%)            |
 <details>
 <summary> Reproduce Peak Memory Usage Results </summary>

 | Memory (tested on H100)          |                |                               |
 |----------------------------------|----------------|-------------------------------|
 |                                  | Qwen3-32B      | Qwen3-32B-float8dq            |
+| Peak Memory                      | 65.72 GB       | 34.54 GB (47.44% reduction)   |
 <details>
 <summary> Reproduce Peak Memory Usage Results </summary>