Gaivoronsky
/

Mistral-7B-Saiga

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Gaivoronsky commited on Oct 13, 2023

Commit

75a98eb

·

1 Parent(s): b25f8b7

Update README.md

Files changed (1) hide show

README.md +5 -0

README.md CHANGED Viewed

@@ -11,7 +11,12 @@ pipeline_tag: text-generation
 ---
 This is a generative model converted to fp16 format based on [IlyaGusev/saiga_mistral_7b_lora](https://huggingface.co/IlyaGusev/saiga_mistral_7b_lora)
 ```bash
 python -u -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --model Gaivoronsky/Mistral-7B-Saiga
 ```

 ---
 This is a generative model converted to fp16 format based on [IlyaGusev/saiga_mistral_7b_lora](https://huggingface.co/IlyaGusev/saiga_mistral_7b_lora)
+Install vLLM
+```bash
+pip install vllm
+```
+Start server:
 ```bash
 python -u -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --model Gaivoronsky/Mistral-7B-Saiga
 ```