CreitinGameplays
/

Llama-3.1-8b-reasoning-test-Q4_K_M-GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

CreitinGameplays commited on 16 days ago

Commit

6833b1b

·

verified ·

1 Parent(s): 70a11e3

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -66,8 +66,8 @@ llm = Llama.from_pretrained(
     verbose=False,
     n_gpu_layers=0,  # CPU-only; increase if using GPU
     n_batch=512,
-    n_ctx=131072,
-    n_ctx_per_seq=131072,
     f16_kv=True
 )

     verbose=False,
     n_gpu_layers=0,  # CPU-only; increase if using GPU
     n_batch=512,
+    n_ctx=8192,
+    n_ctx_per_seq=8192,
     f16_kv=True
 )