OPEA
/

QwQ-32B-Preview-int4-sym-mixed-inc

4-bit precision

intel/auto-round

Model card Files Files and versions Community

cicdatopea commited on 25 days ago

Commit

2d7521c

•

1 Parent(s): 20e4b60

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -5,7 +5,7 @@ datasets:
 ---
 ## Model Details
-This model is an int4 model with group_size 128 and symmetric quantization of [Qwen/QwQ-32B-Preview](https://huggingface.co/Qwen/QwQ-32B-Preview) generated by [intel/auto-round](https://github.com/intel/auto-round). We excluded 3 layers from quantization due to the overflow issue on some int4 backends.
 ## How To Use

 ---
 ## Model Details
+This model is an int4 model with group_size 128 and symmetric quantization of [Qwen/QwQ-32B-Preview](https://huggingface.co/Qwen/QwQ-32B-Preview) generated by [intel/auto-round](https://github.com/intel/auto-round). We excluded 3 layers from quantization due to the overflow issue on some int4 backends. You could find AutoAWQ format [here](https://huggingface.co/OPEA/QwQ-32B-Preview-int4-sym-mixed-awq-inc),which is a little different from this one.
 ## How To Use