Update README.md
Browse files
README.md
CHANGED
@@ -89,9 +89,11 @@ Refer to the Provided Files table below to see what files use which methods, and
|
|
89 |
On Linux I use the following command line to launch the KoboldCpp UI with OpenCL aceleration and a context size of 4096:
|
90 |
|
91 |
```
|
92 |
-
python ./koboldcpp.py --stream --unbantokens --threads 8 --usecublas --gpulayers 100 longchat-7b-16k.ggmlv3.q4_K_M.bin
|
93 |
```
|
94 |
|
|
|
|
|
95 |
Change `--gpulayers 100` to the number of layers you want/are able to offload to the GPU. Remove it if you don't have GPU acceleration.
|
96 |
|
97 |
For OpenCL acceleration, change `--usecublas` to `--useclblast 0 0`. You may need to change the second `0` to `1` if you have both an iGPU and a discrete GPU.
|
|
|
89 |
On Linux I use the following command line to launch the KoboldCpp UI with OpenCL aceleration and a context size of 4096:
|
90 |
|
91 |
```
|
92 |
+
python ./koboldcpp.py --contextsize 4096 --stream --unbantokens --threads 8 --usecublas --gpulayers 100 longchat-7b-16k.ggmlv3.q4_K_M.bin
|
93 |
```
|
94 |
|
95 |
+
Change `--contextsize` to the context size you want - **it must be higher than 2048 else the model will produce gibberish**
|
96 |
+
|
97 |
Change `--gpulayers 100` to the number of layers you want/are able to offload to the GPU. Remove it if you don't have GPU acceleration.
|
98 |
|
99 |
For OpenCL acceleration, change `--usecublas` to `--useclblast 0 0`. You may need to change the second `0` to `1` if you have both an iGPU and a discrete GPU.
|