Spaces:

PharynxAI
/

LLM_FinetuneR

Paused

Diksha2001 commited on Dec 2, 2024

Commit

82235bb

verified ·

1 Parent(s): b69431f

Update VLLM_evaluation.py

Files changed (1) hide show

VLLM_evaluation.py CHANGED Viewed

@@ -54,7 +54,7 @@ def start_vllm_server(model_name):
         "vllm",
         "serve",
         f"PharynxAI/{model_name}",
-        "--gpu_memory_utilization=0.80",
         "--max_model_len=4096",
         "--enable-chunked-prefill=False",
         "--num_scheduler_steps=2"

         "vllm",
         "serve",
         f"PharynxAI/{model_name}",
+        "--gpu_memory_utilization=0.98",
         "--max_model_len=4096",
         "--enable-chunked-prefill=False",
         "--num_scheduler_steps=2"