redmoe-ai-v1 commited on
Commit
00a40af
·
verified ·
1 Parent(s): 0b22634

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -73,7 +73,12 @@ We release the quantized `dots.llm1.inst.FP8-dynamic` model, which retains appro
73
  For convenience, we recommend running vLLM inference using our Docker image `rednote-hilab/vllm-openai-v0.9.1`, , which is available on [Docker Hub](https://hub.docker.com/repository/docker/rednotehilab/dots1/tags).
74
 
75
  ```bash
76
- python3 -m vllm.entrypoints.openai.api_server --model rednote-hilab/dots.llm1.inst.FP8-dynamic --tensor-parallel-size 4 --pipeline-parallel-size 1 --trust-remote-code --served-model-name dots1
 
 
 
 
 
77
  ```
78
 
79
 
 
73
  For convenience, we recommend running vLLM inference using our Docker image `rednote-hilab/vllm-openai-v0.9.1`, , which is available on [Docker Hub](https://hub.docker.com/repository/docker/rednotehilab/dots1/tags).
74
 
75
  ```bash
76
+ python3 -m vllm.entrypoints.openai.api_server \
77
+ --model rednote-hilab/dots.llm1.inst.FP8-dynamic \
78
+ --tensor-parallel-size 4 \
79
+ --pipeline-parallel-size 1 \
80
+ --trust-remote-code \
81
+ --served-model-name dots1
82
  ```
83
 
84