SocialLocalMobile commited on
Commit
cfd5a4d
·
verified ·
1 Parent(s): 9e5aed4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -8
README.md CHANGED
@@ -12,13 +12,13 @@ pipeline_tag: text-generation
12
 
13
  [Qwen3-32B](https://huggingface.co/Qwen3/Qwen3-32B) model quantized with [torchao](https://huggingface.co/docs/transformers/main/en/quantization/torchao) float8 dynamic activation and float8 weight quantization (per row granularity), by PyTorch team. Use it directly, or serve using [vLLM](https://docs.vllm.ai/en/latest/) with TODO VRAM reduction, TODO speedup and little to no accuracy impact on H100.
14
 
15
- # Inference with vLLM
16
  TODO
17
 
18
- # Inference with Transformers
19
  TODO
20
 
21
- # Quantization Recipe
22
 
23
  Install the required packages:
24
 
@@ -77,17 +77,16 @@ print("thinking content:", thinking_content)
77
  print("content:", content)
78
  ```
79
 
80
- # Model Quality
81
  TODO
82
 
83
- # Peak Memory Usage
84
  TODO
85
 
86
- # Model Performance
87
  TODO
88
 
89
-
90
- # Disclaimer
91
  PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.
92
 
93
  Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the licenses the models are released under, including any limitations of liability or disclaimers of warranties provided therein.
 
12
 
13
  [Qwen3-32B](https://huggingface.co/Qwen3/Qwen3-32B) model quantized with [torchao](https://huggingface.co/docs/transformers/main/en/quantization/torchao) float8 dynamic activation and float8 weight quantization (per row granularity), by PyTorch team. Use it directly, or serve using [vLLM](https://docs.vllm.ai/en/latest/) with TODO VRAM reduction, TODO speedup and little to no accuracy impact on H100.
14
 
15
+ # 1. Inference with vLLM
16
  TODO
17
 
18
+ # 2. Inference with Transformers
19
  TODO
20
 
21
+ # 3. Quantization Recipe
22
 
23
  Install the required packages:
24
 
 
77
  print("content:", content)
78
  ```
79
 
80
+ # 4. Model Quality
81
  TODO
82
 
83
+ # 5. Peak Memory Usage
84
  TODO
85
 
86
+ # 6. Model Performance
87
  TODO
88
 
89
+ # 7. Disclaimer
 
90
  PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.
91
 
92
  Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the licenses the models are released under, including any limitations of liability or disclaimers of warranties provided therein.