Spaces:

matthoffner
/

serp-chat

Paused

matthoffner commited on May 22, 2023

Commit

6bf5156

1 Parent(s): 6ce25db

Update llm.py

Files changed (1) hide show

llm.py CHANGED Viewed

@@ -194,7 +194,8 @@ def ask_ai(
         last_n_tokens_size=100,
         n_threads=4,
         f16_kv=True,
-        max_tokens=200
     )
     embeddings = HuggingFaceEmbeddings(model_kwargs={"device": "cuda"})
     embed_model = LangchainEmbedding(embeddings)

         last_n_tokens_size=100,
         n_threads=4,
         f16_kv=True,
+        max_tokens=200,
+        n_gpu_layers=20
     )
     embeddings = HuggingFaceEmbeddings(model_kwargs={"device": "cuda"})
     embed_model = LangchainEmbedding(embeddings)