Spaces:

matthoffner
/

serp-chat

Paused

matthoffner commited on May 23, 2023

Commit

fef1031

1 Parent(s): 6bf5156

Update llm.py

Files changed (1) hide show

llm.py CHANGED Viewed

@@ -195,7 +195,7 @@ def ask_ai(
         n_threads=4,
         f16_kv=True,
         max_tokens=200,
-        n_gpu_layers=20
     )
     embeddings = HuggingFaceEmbeddings(model_kwargs={"device": "cuda"})
     embed_model = LangchainEmbedding(embeddings)

         n_threads=4,
         f16_kv=True,
         max_tokens=200,
+        n_gpu_layers=40
     )
     embeddings = HuggingFaceEmbeddings(model_kwargs={"device": "cuda"})
     embed_model = LangchainEmbedding(embeddings)