DavidAU commited on
Commit
807d83b
·
verified ·
1 Parent(s): 6baeadc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -20
README.md CHANGED
@@ -166,7 +166,9 @@ Then test "at temp" to see the MODELS in action. (5-10 generations recommended)
166
  PENALITY SAMPLERS:
167
  ------------------------------------------------------------------------------
168
 
169
- --repeat-last-n N last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size)
 
 
170
  ("repetition_penalty_range" in oobabooga/text-generation-webui , "rp_range" in kobold)
171
 
172
  THIS IS CRITICAL. Too high you can get all kinds of issues (repeat words, sentences, paragraphs or "gibberish"), especially with class 3 or 4 models.
@@ -174,7 +176,9 @@ THIS IS CRITICAL. Too high you can get all kinds of issues (repeat words, senten
174
  This setting also works in conjunction with all other "rep pens" below.
175
 
176
 
177
- --repeat-penalty N penalize repeat sequence of tokens (default: 1.0, 1.0 = disabled)
 
 
178
  (commonly called "rep pen")
179
 
180
  Generally this is set from 1.0 to 1.15 ; smallest increments are best IE: 1.01... 1,.02 or even 1.001... 1.002.
@@ -182,7 +186,9 @@ Generally this is set from 1.0 to 1.15 ; smallest increments are best IE: 1.01..
182
  This affects creativity of the model over all , not just how words are penalized.
183
 
184
 
185
- --presence-penalty N repeat alpha presence penalty (default: 0.0, 0.0 = disabled)
 
 
186
 
187
  Generally leave this at zero IF repeat-last-n is 256 or less. You may want to use this for higher repeat-last-n settings.
188
 
@@ -191,7 +197,9 @@ CLASS 3: 0.05 may assist generation BUT SET "--repeat-last-n" to 512 or less. Be
191
  CLASS 4: 0.1 to 0.25 may assist generation BUT SET "--repeat-last-n" to 64
192
 
193
 
194
- --frequency-penalty N repeat alpha frequency penalty (default: 0.0, 0.0 = disabled)
 
 
195
 
196
  Generally leave this at zero IF repeat-last-n is 512 or less. You may want to use this for higher repeat-last-n settings.
197
 
@@ -208,24 +216,33 @@ SECONDARY SAMPLERS / FILTERS:
208
  ------------------------------------------------------------------------------
209
 
210
 
211
- --tfs N tail free sampling, parameter z (default: 1.0, 1.0 = disabled)
 
 
212
 
213
  Tries to detect a tail of low-probability tokens in the distribution and removes those tokens. The closer to 0, the more discarded tokens.
214
  ( https://www.trentonbricken.com/Tail-Free-Sampling/ )
215
 
216
 
217
- --typical N locally typical sampling, parameter p (default: 1.0, 1.0 = disabled)
 
 
218
 
219
  If not set to 1, select only tokens that are at least this much more likely to appear than random tokens, given the prior text.
220
 
221
 
222
- --mirostat N use Mirostat sampling.
223
- "Top K", "Nucleus", "Tail Free" (TFS) and "Locally Typical" (TYPICAL) samplers are ignored if used.
 
224
  (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
225
 
226
- --mirostat-lr N Mirostat learning rate, parameter eta (default: 0.1) " mirostat_tau "
 
 
227
 
228
- --mirostat-ent N Mirostat target entropy, parameter tau (default: 5.0) " mirostat_eta "
 
 
229
 
230
  Activates the Mirostat sampling technique. It aims to control perplexity during sampling. See the paper. (https://arxiv.org/abs/2007.14966)
231
 
@@ -244,8 +261,13 @@ For Class 3 models it is suggested to use this to assist with generation (min se
244
  For Class 4 models it is highly recommended with Microstat 1 or 2 + mirostat-lr @ 6 to 8 and mirostat_eta at .1 to .5
245
 
246
 
247
- --dynatemp-range N dynamic temperature range (default: 0.0, 0.0 = disabled)
248
- --dynatemp-exp N dynamic temperature exponent (default: 1.0)
 
 
 
 
 
249
 
250
  In: oobabooga/text-generation-webui (has on/off, and high / low) :
251
 
@@ -268,11 +290,15 @@ To set manually (IE: Api, lmstudio, etc) using "range" and "exp" ; this is a bit
268
  This is both an enhancement and in some ways fixes issues in a model when too little temp (or too much/too much of the same) affects generation.
269
 
270
 
271
- --xtc-probability N xtc probability (default: 0.0, 0.0 = disabled)
 
 
272
 
273
  Probability that the removal will actually happen. 0 disables the sampler. 1 makes it always happen.
274
 
275
- --xtc-threshold N xtc threshold (default: 0.1, 1.0 = disabled)
 
 
276
 
277
  If 2 or more tokens have probability above this threshold, consider removing all but the last one.
278
 
@@ -281,7 +307,9 @@ Suggest you experiment with this one, with other advanced samplers disabled to s
281
 
282
 
283
 
284
- -l, --logit-bias TOKEN_ID(+/-)BIAS modifies the likelihood of token appearing in the completion,
 
 
285
  i.e. `--logit-bias 15043+1` to increase likelihood of token ' Hello',
286
  or `--logit-bias 15043-1` to decrease likelihood of token ' Hello'
287
 
@@ -301,14 +329,21 @@ OTHER:
301
  ------------------------------------------------------------------------------
302
 
303
 
304
- -s, --seed SEED RNG seed (default: -1, use random seed for -1)
 
 
 
 
 
 
 
 
305
 
306
- --samplers SAMPLERS samplers that will be used for generation in the order, separated by ';'
307
- (default: top_k;tfs_z;typ_p;top_p;min_p;xtc;temperature)
308
 
309
- --sampling-seq SEQUENCE simplified sequence for samplers that will be used (default: kfypmxt)
310
 
311
- --ignore-eos ignore end of stream token and continue generating (implies --logit-bias EOS-inf)
312
 
313
 
314
  ------------------------------------------------------------------------------
 
166
  PENALITY SAMPLERS:
167
  ------------------------------------------------------------------------------
168
 
169
+ --repeat-last-n N
170
+
171
+ last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size)
172
  ("repetition_penalty_range" in oobabooga/text-generation-webui , "rp_range" in kobold)
173
 
174
  THIS IS CRITICAL. Too high you can get all kinds of issues (repeat words, sentences, paragraphs or "gibberish"), especially with class 3 or 4 models.
 
176
  This setting also works in conjunction with all other "rep pens" below.
177
 
178
 
179
+ --repeat-penalty N
180
+
181
+ penalize repeat sequence of tokens (default: 1.0, 1.0 = disabled)
182
  (commonly called "rep pen")
183
 
184
  Generally this is set from 1.0 to 1.15 ; smallest increments are best IE: 1.01... 1,.02 or even 1.001... 1.002.
 
186
  This affects creativity of the model over all , not just how words are penalized.
187
 
188
 
189
+ --presence-penalty N
190
+
191
+ repeat alpha presence penalty (default: 0.0, 0.0 = disabled)
192
 
193
  Generally leave this at zero IF repeat-last-n is 256 or less. You may want to use this for higher repeat-last-n settings.
194
 
 
197
  CLASS 4: 0.1 to 0.25 may assist generation BUT SET "--repeat-last-n" to 64
198
 
199
 
200
+ --frequency-penalty N
201
+
202
+ repeat alpha frequency penalty (default: 0.0, 0.0 = disabled)
203
 
204
  Generally leave this at zero IF repeat-last-n is 512 or less. You may want to use this for higher repeat-last-n settings.
205
 
 
216
  ------------------------------------------------------------------------------
217
 
218
 
219
+ --tfs N
220
+
221
+ tail free sampling, parameter z (default: 1.0, 1.0 = disabled)
222
 
223
  Tries to detect a tail of low-probability tokens in the distribution and removes those tokens. The closer to 0, the more discarded tokens.
224
  ( https://www.trentonbricken.com/Tail-Free-Sampling/ )
225
 
226
 
227
+ --typical N
228
+
229
+ locally typical sampling, parameter p (default: 1.0, 1.0 = disabled)
230
 
231
  If not set to 1, select only tokens that are at least this much more likely to appear than random tokens, given the prior text.
232
 
233
 
234
+ --mirostat N
235
+
236
+ use Mirostat sampling. "Top K", "Nucleus", "Tail Free" (TFS) and "Locally Typical" (TYPICAL) samplers are ignored if used.
237
  (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
238
 
239
+ --mirostat-lr N
240
+
241
+ Mirostat learning rate, parameter eta (default: 0.1) " mirostat_tau "
242
 
243
+ --mirostat-ent N
244
+
245
+ Mirostat target entropy, parameter tau (default: 5.0) " mirostat_eta "
246
 
247
  Activates the Mirostat sampling technique. It aims to control perplexity during sampling. See the paper. (https://arxiv.org/abs/2007.14966)
248
 
 
261
  For Class 4 models it is highly recommended with Microstat 1 or 2 + mirostat-lr @ 6 to 8 and mirostat_eta at .1 to .5
262
 
263
 
264
+ --dynatemp-range N
265
+
266
+ dynamic temperature range (default: 0.0, 0.0 = disabled)
267
+
268
+ --dynatemp-exp N
269
+
270
+ dynamic temperature exponent (default: 1.0)
271
 
272
  In: oobabooga/text-generation-webui (has on/off, and high / low) :
273
 
 
290
  This is both an enhancement and in some ways fixes issues in a model when too little temp (or too much/too much of the same) affects generation.
291
 
292
 
293
+ --xtc-probability N
294
+
295
+ xtc probability (default: 0.0, 0.0 = disabled)
296
 
297
  Probability that the removal will actually happen. 0 disables the sampler. 1 makes it always happen.
298
 
299
+ --xtc-threshold N
300
+
301
+ xtc threshold (default: 0.1, 1.0 = disabled)
302
 
303
  If 2 or more tokens have probability above this threshold, consider removing all but the last one.
304
 
 
307
 
308
 
309
 
310
+ -l, --logit-bias TOKEN_ID(+/-)BIAS
311
+
312
+ modifies the likelihood of token appearing in the completion,
313
  i.e. `--logit-bias 15043+1` to increase likelihood of token ' Hello',
314
  or `--logit-bias 15043-1` to decrease likelihood of token ' Hello'
315
 
 
329
  ------------------------------------------------------------------------------
330
 
331
 
332
+ -s, --seed SEED
333
+
334
+ RNG seed (default: -1, use random seed for -1)
335
+
336
+ --samplers SAMPLERS
337
+
338
+ samplers that will be used for generation in the order, separated by ';' (default: top_k;tfs_z;typ_p;top_p;min_p;xtc;temperature)
339
+
340
+ --sampling-seq SEQUENCE
341
 
342
+ simplified sequence for samplers that will be used (default: kfypmxt)
 
343
 
344
+ --ignore-eos
345
 
346
+ ignore end of stream token and continue generating (implies --logit-bias EOS-inf)
347
 
348
 
349
  ------------------------------------------------------------------------------