parameters guide
samplers guide
model generation
role play settings
quant selection
arm quants
iq quants vs q quants
optimal model setting
gibberish fixes
coherence
instructing following
quality generation
chat settings
quality settings
llamacpp server
llamacpp
lmstudio
sillytavern
koboldcpp
backyard
ollama
model generation steering
steering
model generation fixes
text generation webui
ggufs
exl2
full precision
quants
imatrix
neo imatrix
Update README.md
Browse files
README.md
CHANGED
@@ -675,13 +675,13 @@ Generally this is not used.
|
|
675 |
|
676 |
---
|
677 |
|
678 |
-
Section 1c : SECONDARY SAMPLERS / FILTERS - ALL APPS
|
679 |
|
680 |
---
|
681 |
|
682 |
In some AI/LLM apps, these may only be available via JSON file modification and/or API.
|
683 |
|
684 |
-
For "text-gen-webui" and "Koboldcpp" these are directly accessible.
|
685 |
|
686 |
<B>i) OVERALL GENERATION CHANGES (affect per token as well as over all generation):</B>
|
687 |
|
@@ -689,13 +689,13 @@ For "text-gen-webui" and "Koboldcpp" these are directly accessible.
|
|
689 |
|
690 |
Use Mirostat sampling. "Top K", "Nucleus", "Tail Free" (TFS) and "Locally Typical" (TYPICAL) samplers are ignored if used. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
|
691 |
|
692 |
-
mirostat-lr
|
693 |
|
694 |
Mirostat learning rate, parameter eta (default: 0.1) " mirostat_tau "
|
695 |
|
696 |
mirostat_tau: 5-8 is a good value.
|
697 |
|
698 |
-
mirostat-ent
|
699 |
|
700 |
Mirostat target entropy, parameter tau (default: 5.0) " mirostat_eta "
|
701 |
|
@@ -717,11 +717,11 @@ CLASS 4: models it is highly recommended with Microstat 1 or 2 + mirostat_tau @
|
|
717 |
|
718 |
<b>Dynamic Temperature</b>
|
719 |
|
720 |
-
dynatemp-range
|
721 |
|
722 |
dynamic temperature range (default: 0.0, 0.0 = disabled)
|
723 |
|
724 |
-
dynatemp-exp
|
725 |
|
726 |
dynamic temperature exponent (default: 1.0)
|
727 |
|
@@ -765,11 +765,11 @@ If not set to 1, select only tokens that are at least this much more likely to a
|
|
765 |
|
766 |
xtc-probability
|
767 |
|
768 |
-
xtc probability (default: 0.0, 0.0 = disabled)
|
769 |
|
770 |
Probability that the removal will actually happen. 0 disables the sampler. 1 makes it always happen.
|
771 |
|
772 |
-
xtc-threshold
|
773 |
|
774 |
xtc threshold (default: 0.1, 1.0 = disabled)
|
775 |
|
@@ -798,20 +798,32 @@ Careful testing is required, as this can have unclear side effects.
|
|
798 |
|
799 |
---
|
800 |
|
801 |
-
<h2>SECTION 2: ADVANCED SAMPLERS - "text-generation-webui" / "KOBOLDCPP": </h2>
|
802 |
|
803 |
<B>Additional Parameters / Samplers, including "DRY", "QUADRATIC" and "ANTI-SLOP".</B>
|
804 |
|
805 |
---
|
806 |
|
|
|
|
|
|
|
|
|
|
|
|
|
807 |
Hopefully ALL these samplers / controls will be LLAMACPP and available to all users via AI/LLM apps soon.
|
808 |
|
|
|
|
|
|
|
|
|
809 |
For more info on what they do / how they affect generation see:
|
810 |
|
811 |
https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-Tab
|
812 |
|
813 |
(also see the section above "Additional Links" for more info on the parameters/samplers)
|
814 |
|
|
|
|
|
815 |
Keep in mind these parameters/samplers become available (for GGUFs) in "oobabooga/text-generation-webui" when you use the llamacpp_HF loader.
|
816 |
|
817 |
Most of these are also available in KOBOLDCPP too (via settings -> samplers) after start up (no "llamacpp_HF loader" step required).
|
@@ -856,7 +868,7 @@ Note 4:
|
|
856 |
These control selection of tokens, in some case providing more relevant and/or more options. See [ https://arxiv.org/pdf/2210.15191 ]
|
857 |
|
858 |
|
859 |
-
<B>MAIN ADVANCED SAMPLERS (affects per token AND overall generation): </B>
|
860 |
|
861 |
What I will touch on here are special settings for CLASS 3 and CLASS 4 models (for the first TWO samplers).
|
862 |
|
|
|
675 |
|
676 |
---
|
677 |
|
678 |
+
<h2>Section 1c : SECONDARY SAMPLERS / FILTERS - ALL APPS:</h2>
|
679 |
|
680 |
---
|
681 |
|
682 |
In some AI/LLM apps, these may only be available via JSON file modification and/or API.
|
683 |
|
684 |
+
For "text-gen-webui" and "Koboldcpp" these are directly accessible (and via Sillytavern IF you use either of these APPS to connect Silly Tavern to their API).
|
685 |
|
686 |
<B>i) OVERALL GENERATION CHANGES (affect per token as well as over all generation):</B>
|
687 |
|
|
|
689 |
|
690 |
Use Mirostat sampling. "Top K", "Nucleus", "Tail Free" (TFS) and "Locally Typical" (TYPICAL) samplers are ignored if used. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
|
691 |
|
692 |
+
"mirostat-lr"
|
693 |
|
694 |
Mirostat learning rate, parameter eta (default: 0.1) " mirostat_tau "
|
695 |
|
696 |
mirostat_tau: 5-8 is a good value.
|
697 |
|
698 |
+
"mirostat-ent"
|
699 |
|
700 |
Mirostat target entropy, parameter tau (default: 5.0) " mirostat_eta "
|
701 |
|
|
|
717 |
|
718 |
<b>Dynamic Temperature</b>
|
719 |
|
720 |
+
"dynatemp-range "
|
721 |
|
722 |
dynamic temperature range (default: 0.0, 0.0 = disabled)
|
723 |
|
724 |
+
"dynatemp-exp"
|
725 |
|
726 |
dynamic temperature exponent (default: 1.0)
|
727 |
|
|
|
765 |
|
766 |
xtc-probability
|
767 |
|
768 |
+
"xtc probability" (default: 0.0, 0.0 = disabled)
|
769 |
|
770 |
Probability that the removal will actually happen. 0 disables the sampler. 1 makes it always happen.
|
771 |
|
772 |
+
"xtc-threshold"
|
773 |
|
774 |
xtc threshold (default: 0.1, 1.0 = disabled)
|
775 |
|
|
|
798 |
|
799 |
---
|
800 |
|
801 |
+
<h2>SECTION 2: ADVANCED SAMPLERS - "text-generation-webui" / "KOBOLDCPP" / "SillyTavern" (see note 1 below): </h2>
|
802 |
|
803 |
<B>Additional Parameters / Samplers, including "DRY", "QUADRATIC" and "ANTI-SLOP".</B>
|
804 |
|
805 |
---
|
806 |
|
807 |
+
Note #1 :
|
808 |
+
|
809 |
+
You can use these samplers via Sillytavern IF you use either of these APPS (Koboldcpp/Text Generation Webui) to connect Silly Tavern to their API.
|
810 |
+
|
811 |
+
Other Notes:
|
812 |
+
|
813 |
Hopefully ALL these samplers / controls will be LLAMACPP and available to all users via AI/LLM apps soon.
|
814 |
|
815 |
+
"DRY" sampler has been added to Llamacpp as of the time of this writing (and available via SERVER/LLAMA-SERVER.EXE) and MAY appear in other "downstream" apps that use Llamacpp.
|
816 |
+
|
817 |
+
INFORMATION ON THESE SAMPLERS:
|
818 |
+
|
819 |
For more info on what they do / how they affect generation see:
|
820 |
|
821 |
https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-Tab
|
822 |
|
823 |
(also see the section above "Additional Links" for more info on the parameters/samplers)
|
824 |
|
825 |
+
ADVANCED SAMPLERS - PART 1:
|
826 |
+
|
827 |
Keep in mind these parameters/samplers become available (for GGUFs) in "oobabooga/text-generation-webui" when you use the llamacpp_HF loader.
|
828 |
|
829 |
Most of these are also available in KOBOLDCPP too (via settings -> samplers) after start up (no "llamacpp_HF loader" step required).
|
|
|
868 |
These control selection of tokens, in some case providing more relevant and/or more options. See [ https://arxiv.org/pdf/2210.15191 ]
|
869 |
|
870 |
|
871 |
+
<B>MAIN ADVANCED SAMPLERS PART 2 (affects per token AND overall generation): </B>
|
872 |
|
873 |
What I will touch on here are special settings for CLASS 3 and CLASS 4 models (for the first TWO samplers).
|
874 |
|