Update README.md

README.md

@@ -15,7 +15,7 @@ pipeline_tag: text-generation
 
 [Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B) model quantized with [torchao](https://huggingface.co/docs/transformers/main/en/quantization/torchao) float8 dynamic activation and float8 weight quantization (per row granularity), by PyTorch team. Use it directly, or serve using [vLLM](https://docs.vllm.ai/en/latest/) with TODO VRAM reduction, TODO speedup and little to no accuracy impact on H100.
 
-# 1. Inference with vLLM
+# Inference with vLLM
 ```Shell
 # Server
 VLLM_DISABLE_COMPILE_CACHE=1 vllm serve SocialLocalMobile/Qwen3-32B-float8dq --tokenizer Qwen/Qwen3-32B -O3
@@ -36,10 +36,7 @@ curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/jso
 ```
 
 
-# 2. Inference with Transformers
-TODO
-
-# 3. Quantization Recipe
+# Quantization Recipe
 
 Install the required packages:
 
@@ -112,23 +109,20 @@ quantized_model.push_to_hub(save_to, safe_serialization=False)
 tokenizer.push_to_hub(save_to)
 ```
 
-# 4. Model Quality
+# Model Quality
 TODO
 
-# 5. Peak Memory Usage
+# Peak Memory Usage
 TODO
 
-# 6. Model Performance
+# Model Performance
 
-## Results (H100 machine)
 
 | Benchmark                        |                |                               |
 |----------------------------------|----------------|-------------------------------|
 |                                  | Qwen3-32B      | Qwen3-32B-float8dq            | 
 | latency (batch_size=1)           | 9.1s           | 5.77s (-36.6%)                |
 | latency (batch_size=128)         | 12.45s         | 8.40s (-32.5%)                |
-| serving (num_prompts=1)          | TODO           | TODO                          |
-| serving (num_prompts=1000)       | TODO           | TODO                          |
 
 <details>
 <summary> Reproduce latency benchmarks </summary>
@@ -145,29 +139,9 @@ VLLM_USE_PRECOMPILED=1 pip install --editable .
 export MODEL=Qwen/Qwen3-32B # or pytorch/Qwen3-32B-float8dq
 VLLM_DISABLE_COMPILE_CACHE=1 python benchmarks/benchmark_latency.py --input-len 256 --output-len 256 --model MODEL --batch-size 1
 ```
-
-**3. Serving benchmarking**
-
-Setup: 
-```Shell
-wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json
-```
-
-Server:
-```Shell
-export MODEL=Qwen/Qwen3-32B # or pytorch/Qwen3-32B-float8dq
-VLLM_DISABLE_COMPILE_CACHE=1 vllm serve $MODEL --tokenizer Qwen/Qwen3-32B -O3
-```
-
-Client:
-```Shell
-export MODEL=Qwen/Qwen3-32B # or pytorch/Qwen3-32B-float8dq
-python benchmarks/benchmark_serving.py --backend vllm --dataset-name sharegpt --tokenizer Qwen/Qwen3-32B --dataset-path ./ShareGPT_V3_unfiltered_cleaned_split.json --model MODEL --num-prompts 1
-```
-
 </details>
 
-# 7. Disclaimer
+# Disclaimer
 PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.
 
 Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the licenses the models are released under, including any limitations of liability or disclaimers of warranties provided therein.