Update README.md

README.md

@@ -119,7 +119,53 @@ TODO
 TODO
 
 # 6. Model Performance
-TODO
+
+## Results (H100 machine)
+
+| Benchmark                        |                |                               |
+|----------------------------------|----------------|-------------------------------|
+|                                  | Qwen3-32B      | Qwen3-32B-float8dq            | 
+| latency (batch_size=1)           | 9.1s           | TODO                          |
+| latency (batch_size=128)         | 12.45s         | TODO                          |
+| serving (num_prompts=1)          | TODO           | TODO                          |
+| serving (num_prompts=1000)       | TODO           | TODO                          |
+
+<details>
+<summary> Reproduce latency benchmarks </summary>
+  
+**1. Setup**
+```Shell
+git clone [email protected]:vllm-project/vllm.git
+cd vllm
+VLLM_USE_PRECOMPILED=1 pip install --editable .
+```
+
+**2. Latency benchmarking**
+```Shell
+export MODEL=Qwen/Qwen3-32B # or pytorch/Qwen3-32B-float8dq
+VLLM_DISABLE_COMPILE_CACHE=1 python benchmarks/benchmark_latency.py --input-len 256 --output-len 256 --model MODEL --batch-size 1
+```
+
+**3. Serving benchmarking**
+
+Setup: 
+```Shell
+wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json
+```
+
+Server:
+```Shell
+export MODEL=Qwen/Qwen3-32B # or pytorch/Qwen3-32B-float8dq
+VLLM_DISABLE_COMPILE_CACHE=1 vllm serve MODEL --tokenizer Qwen/Qwen3-32B -O3
+```
+
+Client:
+```Shell
+export MODEL=Qwen/Qwen3-32B # or pytorch/Qwen3-32B-float8dq
+python benchmarks/benchmark_serving.py --backend vllm --dataset-name sharegpt --tokenizer Qwen/Qwen3-32B --dataset-path ./ShareGPT_V3_unfiltered_cleaned_split.json --model MODEL --num-prompts 1
+```
+
+</details>
 
 # 7. Disclaimer
 PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.