Spaces:

Luigi
/

SmolVLM2-on-llama.cpp

Running

Luigi commited on Jun 17

Commit

1aba000

1 Parent(s): cc08312

reduce n_ctx to 512

Files changed (1) hide show

app.py CHANGED Viewed

@@ -98,7 +98,7 @@ def update_llm(size, model_file, clip_file, verbose_mode):
         llm = Llama(
             model_path=mf,
             chat_handler=handler,
-            n_ctx=8192,
             verbose=verbose_mode,
             n_threads=max(2, os.cpu_count())
         )

         llm = Llama(
             model_path=mf,
             chat_handler=handler,
+            n_ctx=512,
             verbose=verbose_mode,
             n_threads=max(2, os.cpu_count())
         )