Hugging Face's logo

csdc-atl
/
Baichuan2-13B-Chat-GPTQ-Int4

Text Generation
Transformers
Safetensors
English
Chinese
baichuan
custom_code
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
xiaotinghe commited on
Commit
6004920
·
1 Parent(s): d9e7596

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -31,7 +31,7 @@ tasks:
31
  - Sequence Length: 4096
32
  | 模型版本 | agieval | ceval | cmmlu | size | 推理速度(A100-40G) |
33
  |---|---|---|---|---|---|
34
- | [Baichuan2-13B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat) | ~ | ~ | ~ | 27.79g | 31.55 tokens/s |
35
  | [Baichuan2-13B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat-4bits) | ~ | ~ | ~ | 9.08g | 18.45 tokens/s |
36
  | [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-13B-Chat-GPTQ-Int4/tree/4bit-32g) | ~ | ~ | ~ | 9.87g | 27.35(hf) \ 38.28(autogptq) tokens/s |
37
  | [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-13B-Chat-GPTQ-Int4/tree/main) | 38.78 | 56.42 | 57.78 | 9.14g | 28.74(hf) \ 39.24(autogptq) tokens/s |
 
31
  - Sequence Length: 4096
32
  | 模型版本 | agieval | ceval | cmmlu | size | 推理速度(A100-40G) |
33
  |---|---|---|---|---|---|
34
+ | [Baichuan2-13B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat) | 40.25 | 56.33 | 58.44 | 27.79g | 31.55 tokens/s |
35
  | [Baichuan2-13B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat-4bits) | ~ | ~ | ~ | 9.08g | 18.45 tokens/s |
36
  | [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-13B-Chat-GPTQ-Int4/tree/4bit-32g) | ~ | ~ | ~ | 9.87g | 27.35(hf) \ 38.28(autogptq) tokens/s |
37
  | [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-13B-Chat-GPTQ-Int4/tree/main) | 38.78 | 56.42 | 57.78 | 9.14g | 28.74(hf) \ 39.24(autogptq) tokens/s |