parameters guide
samplers guide
model generation
role play settings
quant selection
arm quants
iq quants vs q quants
optimal model setting
gibberish fixes
coherence
instructing following
quality generation
chat settings
quality settings
llamacpp server
llamacpp
lmstudio
sillytavern
koboldcpp
backyard
ollama
model generation steering
steering
model generation fixes
text generation webui
ggufs
exl2
full precision
quants
imatrix
neo imatrix
Update README.md
Browse files
README.md
CHANGED
@@ -166,7 +166,9 @@ Then test "at temp" to see the MODELS in action. (5-10 generations recommended)
|
|
166 |
PENALITY SAMPLERS:
|
167 |
------------------------------------------------------------------------------
|
168 |
|
169 |
-
--repeat-last-n N
|
|
|
|
|
170 |
("repetition_penalty_range" in oobabooga/text-generation-webui , "rp_range" in kobold)
|
171 |
|
172 |
THIS IS CRITICAL. Too high you can get all kinds of issues (repeat words, sentences, paragraphs or "gibberish"), especially with class 3 or 4 models.
|
@@ -174,7 +176,9 @@ THIS IS CRITICAL. Too high you can get all kinds of issues (repeat words, senten
|
|
174 |
This setting also works in conjunction with all other "rep pens" below.
|
175 |
|
176 |
|
177 |
-
--repeat-penalty N
|
|
|
|
|
178 |
(commonly called "rep pen")
|
179 |
|
180 |
Generally this is set from 1.0 to 1.15 ; smallest increments are best IE: 1.01... 1,.02 or even 1.001... 1.002.
|
@@ -182,7 +186,9 @@ Generally this is set from 1.0 to 1.15 ; smallest increments are best IE: 1.01..
|
|
182 |
This affects creativity of the model over all , not just how words are penalized.
|
183 |
|
184 |
|
185 |
-
--presence-penalty N
|
|
|
|
|
186 |
|
187 |
Generally leave this at zero IF repeat-last-n is 256 or less. You may want to use this for higher repeat-last-n settings.
|
188 |
|
@@ -191,7 +197,9 @@ CLASS 3: 0.05 may assist generation BUT SET "--repeat-last-n" to 512 or less. Be
|
|
191 |
CLASS 4: 0.1 to 0.25 may assist generation BUT SET "--repeat-last-n" to 64
|
192 |
|
193 |
|
194 |
-
--frequency-penalty N
|
|
|
|
|
195 |
|
196 |
Generally leave this at zero IF repeat-last-n is 512 or less. You may want to use this for higher repeat-last-n settings.
|
197 |
|
@@ -208,24 +216,33 @@ SECONDARY SAMPLERS / FILTERS:
|
|
208 |
------------------------------------------------------------------------------
|
209 |
|
210 |
|
211 |
-
--tfs N
|
|
|
|
|
212 |
|
213 |
Tries to detect a tail of low-probability tokens in the distribution and removes those tokens. The closer to 0, the more discarded tokens.
|
214 |
( https://www.trentonbricken.com/Tail-Free-Sampling/ )
|
215 |
|
216 |
|
217 |
-
--typical N
|
|
|
|
|
218 |
|
219 |
If not set to 1, select only tokens that are at least this much more likely to appear than random tokens, given the prior text.
|
220 |
|
221 |
|
222 |
-
--mirostat N
|
223 |
-
|
|
|
224 |
(default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
|
225 |
|
226 |
-
--mirostat-lr N
|
|
|
|
|
227 |
|
228 |
-
--mirostat-ent N
|
|
|
|
|
229 |
|
230 |
Activates the Mirostat sampling technique. It aims to control perplexity during sampling. See the paper. (https://arxiv.org/abs/2007.14966)
|
231 |
|
@@ -244,8 +261,13 @@ For Class 3 models it is suggested to use this to assist with generation (min se
|
|
244 |
For Class 4 models it is highly recommended with Microstat 1 or 2 + mirostat-lr @ 6 to 8 and mirostat_eta at .1 to .5
|
245 |
|
246 |
|
247 |
-
--dynatemp-range N
|
248 |
-
|
|
|
|
|
|
|
|
|
|
|
249 |
|
250 |
In: oobabooga/text-generation-webui (has on/off, and high / low) :
|
251 |
|
@@ -268,11 +290,15 @@ To set manually (IE: Api, lmstudio, etc) using "range" and "exp" ; this is a bit
|
|
268 |
This is both an enhancement and in some ways fixes issues in a model when too little temp (or too much/too much of the same) affects generation.
|
269 |
|
270 |
|
271 |
-
--xtc-probability N
|
|
|
|
|
272 |
|
273 |
Probability that the removal will actually happen. 0 disables the sampler. 1 makes it always happen.
|
274 |
|
275 |
-
--xtc-threshold N
|
|
|
|
|
276 |
|
277 |
If 2 or more tokens have probability above this threshold, consider removing all but the last one.
|
278 |
|
@@ -281,7 +307,9 @@ Suggest you experiment with this one, with other advanced samplers disabled to s
|
|
281 |
|
282 |
|
283 |
|
284 |
-
-l, --logit-bias TOKEN_ID(+/-)BIAS
|
|
|
|
|
285 |
i.e. `--logit-bias 15043+1` to increase likelihood of token ' Hello',
|
286 |
or `--logit-bias 15043-1` to decrease likelihood of token ' Hello'
|
287 |
|
@@ -301,14 +329,21 @@ OTHER:
|
|
301 |
------------------------------------------------------------------------------
|
302 |
|
303 |
|
304 |
-
-s, --seed SEED
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
305 |
|
306 |
-
|
307 |
-
(default: top_k;tfs_z;typ_p;top_p;min_p;xtc;temperature)
|
308 |
|
309 |
-
--
|
310 |
|
311 |
-
|
312 |
|
313 |
|
314 |
------------------------------------------------------------------------------
|
|
|
166 |
PENALITY SAMPLERS:
|
167 |
------------------------------------------------------------------------------
|
168 |
|
169 |
+
--repeat-last-n N
|
170 |
+
|
171 |
+
last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size)
|
172 |
("repetition_penalty_range" in oobabooga/text-generation-webui , "rp_range" in kobold)
|
173 |
|
174 |
THIS IS CRITICAL. Too high you can get all kinds of issues (repeat words, sentences, paragraphs or "gibberish"), especially with class 3 or 4 models.
|
|
|
176 |
This setting also works in conjunction with all other "rep pens" below.
|
177 |
|
178 |
|
179 |
+
--repeat-penalty N
|
180 |
+
|
181 |
+
penalize repeat sequence of tokens (default: 1.0, 1.0 = disabled)
|
182 |
(commonly called "rep pen")
|
183 |
|
184 |
Generally this is set from 1.0 to 1.15 ; smallest increments are best IE: 1.01... 1,.02 or even 1.001... 1.002.
|
|
|
186 |
This affects creativity of the model over all , not just how words are penalized.
|
187 |
|
188 |
|
189 |
+
--presence-penalty N
|
190 |
+
|
191 |
+
repeat alpha presence penalty (default: 0.0, 0.0 = disabled)
|
192 |
|
193 |
Generally leave this at zero IF repeat-last-n is 256 or less. You may want to use this for higher repeat-last-n settings.
|
194 |
|
|
|
197 |
CLASS 4: 0.1 to 0.25 may assist generation BUT SET "--repeat-last-n" to 64
|
198 |
|
199 |
|
200 |
+
--frequency-penalty N
|
201 |
+
|
202 |
+
repeat alpha frequency penalty (default: 0.0, 0.0 = disabled)
|
203 |
|
204 |
Generally leave this at zero IF repeat-last-n is 512 or less. You may want to use this for higher repeat-last-n settings.
|
205 |
|
|
|
216 |
------------------------------------------------------------------------------
|
217 |
|
218 |
|
219 |
+
--tfs N
|
220 |
+
|
221 |
+
tail free sampling, parameter z (default: 1.0, 1.0 = disabled)
|
222 |
|
223 |
Tries to detect a tail of low-probability tokens in the distribution and removes those tokens. The closer to 0, the more discarded tokens.
|
224 |
( https://www.trentonbricken.com/Tail-Free-Sampling/ )
|
225 |
|
226 |
|
227 |
+
--typical N
|
228 |
+
|
229 |
+
locally typical sampling, parameter p (default: 1.0, 1.0 = disabled)
|
230 |
|
231 |
If not set to 1, select only tokens that are at least this much more likely to appear than random tokens, given the prior text.
|
232 |
|
233 |
|
234 |
+
--mirostat N
|
235 |
+
|
236 |
+
use Mirostat sampling. "Top K", "Nucleus", "Tail Free" (TFS) and "Locally Typical" (TYPICAL) samplers are ignored if used.
|
237 |
(default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
|
238 |
|
239 |
+
--mirostat-lr N
|
240 |
+
|
241 |
+
Mirostat learning rate, parameter eta (default: 0.1) " mirostat_tau "
|
242 |
|
243 |
+
--mirostat-ent N
|
244 |
+
|
245 |
+
Mirostat target entropy, parameter tau (default: 5.0) " mirostat_eta "
|
246 |
|
247 |
Activates the Mirostat sampling technique. It aims to control perplexity during sampling. See the paper. (https://arxiv.org/abs/2007.14966)
|
248 |
|
|
|
261 |
For Class 4 models it is highly recommended with Microstat 1 or 2 + mirostat-lr @ 6 to 8 and mirostat_eta at .1 to .5
|
262 |
|
263 |
|
264 |
+
--dynatemp-range N
|
265 |
+
|
266 |
+
dynamic temperature range (default: 0.0, 0.0 = disabled)
|
267 |
+
|
268 |
+
--dynatemp-exp N
|
269 |
+
|
270 |
+
dynamic temperature exponent (default: 1.0)
|
271 |
|
272 |
In: oobabooga/text-generation-webui (has on/off, and high / low) :
|
273 |
|
|
|
290 |
This is both an enhancement and in some ways fixes issues in a model when too little temp (or too much/too much of the same) affects generation.
|
291 |
|
292 |
|
293 |
+
--xtc-probability N
|
294 |
+
|
295 |
+
xtc probability (default: 0.0, 0.0 = disabled)
|
296 |
|
297 |
Probability that the removal will actually happen. 0 disables the sampler. 1 makes it always happen.
|
298 |
|
299 |
+
--xtc-threshold N
|
300 |
+
|
301 |
+
xtc threshold (default: 0.1, 1.0 = disabled)
|
302 |
|
303 |
If 2 or more tokens have probability above this threshold, consider removing all but the last one.
|
304 |
|
|
|
307 |
|
308 |
|
309 |
|
310 |
+
-l, --logit-bias TOKEN_ID(+/-)BIAS
|
311 |
+
|
312 |
+
modifies the likelihood of token appearing in the completion,
|
313 |
i.e. `--logit-bias 15043+1` to increase likelihood of token ' Hello',
|
314 |
or `--logit-bias 15043-1` to decrease likelihood of token ' Hello'
|
315 |
|
|
|
329 |
------------------------------------------------------------------------------
|
330 |
|
331 |
|
332 |
+
-s, --seed SEED
|
333 |
+
|
334 |
+
RNG seed (default: -1, use random seed for -1)
|
335 |
+
|
336 |
+
--samplers SAMPLERS
|
337 |
+
|
338 |
+
samplers that will be used for generation in the order, separated by ';' (default: top_k;tfs_z;typ_p;top_p;min_p;xtc;temperature)
|
339 |
+
|
340 |
+
--sampling-seq SEQUENCE
|
341 |
|
342 |
+
simplified sequence for samplers that will be used (default: kfypmxt)
|
|
|
343 |
|
344 |
+
--ignore-eos
|
345 |
|
346 |
+
ignore end of stream token and continue generating (implies --logit-bias EOS-inf)
|
347 |
|
348 |
|
349 |
------------------------------------------------------------------------------
|