Update README.md
Browse files
README.md
CHANGED
|
@@ -211,7 +211,7 @@ python benchmarks/benchmark_serving.py --backend vllm --dataset-name sharegpt --
|
|
| 211 |
### int4wo-hqq
|
| 212 |
Server:
|
| 213 |
```
|
| 214 |
-
vllm serve pytorch/Phi-4-mini-instruct-int4wo-hqq --tokenizer microsoft/Phi-4-mini-instruct -O3
|
| 215 |
```
|
| 216 |
|
| 217 |
Client:
|
|
|
|
| 211 |
### int4wo-hqq
|
| 212 |
Server:
|
| 213 |
```
|
| 214 |
+
vllm serve pytorch/Phi-4-mini-instruct-int4wo-hqq --tokenizer microsoft/Phi-4-mini-instruct -O3 --pt-load-map-location cuda:0
|
| 215 |
```
|
| 216 |
|
| 217 |
Client:
|