DavidAU commited on
Commit
ccb970d
·
verified ·
1 Parent(s): a4711b1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -10
README.md CHANGED
@@ -675,13 +675,13 @@ Generally this is not used.
675
 
676
  ---
677
 
678
- Section 1c : SECONDARY SAMPLERS / FILTERS - ALL APPS:
679
 
680
  ---
681
 
682
  In some AI/LLM apps, these may only be available via JSON file modification and/or API.
683
 
684
- For "text-gen-webui" and "Koboldcpp" these are directly accessible.
685
 
686
  <B>i) OVERALL GENERATION CHANGES (affect per token as well as over all generation):</B>
687
 
@@ -689,13 +689,13 @@ For "text-gen-webui" and "Koboldcpp" these are directly accessible.
689
 
690
  Use Mirostat sampling. "Top K", "Nucleus", "Tail Free" (TFS) and "Locally Typical" (TYPICAL) samplers are ignored if used. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
691
 
692
- mirostat-lr
693
 
694
  Mirostat learning rate, parameter eta (default: 0.1) " mirostat_tau "
695
 
696
  mirostat_tau: 5-8 is a good value.
697
 
698
- mirostat-ent
699
 
700
  Mirostat target entropy, parameter tau (default: 5.0) " mirostat_eta "
701
 
@@ -717,11 +717,11 @@ CLASS 4: models it is highly recommended with Microstat 1 or 2 + mirostat_tau @
717
 
718
  <b>Dynamic Temperature</b>
719
 
720
- dynatemp-range
721
 
722
  dynamic temperature range (default: 0.0, 0.0 = disabled)
723
 
724
- dynatemp-exp
725
 
726
  dynamic temperature exponent (default: 1.0)
727
 
@@ -765,11 +765,11 @@ If not set to 1, select only tokens that are at least this much more likely to a
765
 
766
  xtc-probability
767
 
768
- xtc probability (default: 0.0, 0.0 = disabled)
769
 
770
  Probability that the removal will actually happen. 0 disables the sampler. 1 makes it always happen.
771
 
772
- xtc-threshold
773
 
774
  xtc threshold (default: 0.1, 1.0 = disabled)
775
 
@@ -798,20 +798,32 @@ Careful testing is required, as this can have unclear side effects.
798
 
799
  ---
800
 
801
- <h2>SECTION 2: ADVANCED SAMPLERS - "text-generation-webui" / "KOBOLDCPP": </h2>
802
 
803
  <B>Additional Parameters / Samplers, including "DRY", "QUADRATIC" and "ANTI-SLOP".</B>
804
 
805
  ---
806
 
 
 
 
 
 
 
807
  Hopefully ALL these samplers / controls will be LLAMACPP and available to all users via AI/LLM apps soon.
808
 
 
 
 
 
809
  For more info on what they do / how they affect generation see:
810
 
811
  https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-Tab
812
 
813
  (also see the section above "Additional Links" for more info on the parameters/samplers)
814
 
 
 
815
  Keep in mind these parameters/samplers become available (for GGUFs) in "oobabooga/text-generation-webui" when you use the llamacpp_HF loader.
816
 
817
  Most of these are also available in KOBOLDCPP too (via settings -> samplers) after start up (no "llamacpp_HF loader" step required).
@@ -856,7 +868,7 @@ Note 4:
856
  These control selection of tokens, in some case providing more relevant and/or more options. See [ https://arxiv.org/pdf/2210.15191 ]
857
 
858
 
859
- <B>MAIN ADVANCED SAMPLERS (affects per token AND overall generation): </B>
860
 
861
  What I will touch on here are special settings for CLASS 3 and CLASS 4 models (for the first TWO samplers).
862
 
 
675
 
676
  ---
677
 
678
+ <h2>Section 1c : SECONDARY SAMPLERS / FILTERS - ALL APPS:</h2>
679
 
680
  ---
681
 
682
  In some AI/LLM apps, these may only be available via JSON file modification and/or API.
683
 
684
+ For "text-gen-webui" and "Koboldcpp" these are directly accessible (and via Sillytavern IF you use either of these APPS to connect Silly Tavern to their API).
685
 
686
  <B>i) OVERALL GENERATION CHANGES (affect per token as well as over all generation):</B>
687
 
 
689
 
690
  Use Mirostat sampling. "Top K", "Nucleus", "Tail Free" (TFS) and "Locally Typical" (TYPICAL) samplers are ignored if used. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
691
 
692
+ "mirostat-lr"
693
 
694
  Mirostat learning rate, parameter eta (default: 0.1) " mirostat_tau "
695
 
696
  mirostat_tau: 5-8 is a good value.
697
 
698
+ "mirostat-ent"
699
 
700
  Mirostat target entropy, parameter tau (default: 5.0) " mirostat_eta "
701
 
 
717
 
718
  <b>Dynamic Temperature</b>
719
 
720
+ "dynatemp-range "
721
 
722
  dynamic temperature range (default: 0.0, 0.0 = disabled)
723
 
724
+ "dynatemp-exp"
725
 
726
  dynamic temperature exponent (default: 1.0)
727
 
 
765
 
766
  xtc-probability
767
 
768
+ "xtc probability" (default: 0.0, 0.0 = disabled)
769
 
770
  Probability that the removal will actually happen. 0 disables the sampler. 1 makes it always happen.
771
 
772
+ "xtc-threshold"
773
 
774
  xtc threshold (default: 0.1, 1.0 = disabled)
775
 
 
798
 
799
  ---
800
 
801
+ <h2>SECTION 2: ADVANCED SAMPLERS - "text-generation-webui" / "KOBOLDCPP" / "SillyTavern" (see note 1 below): </h2>
802
 
803
  <B>Additional Parameters / Samplers, including "DRY", "QUADRATIC" and "ANTI-SLOP".</B>
804
 
805
  ---
806
 
807
+ Note #1 :
808
+
809
+ You can use these samplers via Sillytavern IF you use either of these APPS (Koboldcpp/Text Generation Webui) to connect Silly Tavern to their API.
810
+
811
+ Other Notes:
812
+
813
  Hopefully ALL these samplers / controls will be LLAMACPP and available to all users via AI/LLM apps soon.
814
 
815
+ "DRY" sampler has been added to Llamacpp as of the time of this writing (and available via SERVER/LLAMA-SERVER.EXE) and MAY appear in other "downstream" apps that use Llamacpp.
816
+
817
+ INFORMATION ON THESE SAMPLERS:
818
+
819
  For more info on what they do / how they affect generation see:
820
 
821
  https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-Tab
822
 
823
  (also see the section above "Additional Links" for more info on the parameters/samplers)
824
 
825
+ ADVANCED SAMPLERS - PART 1:
826
+
827
  Keep in mind these parameters/samplers become available (for GGUFs) in "oobabooga/text-generation-webui" when you use the llamacpp_HF loader.
828
 
829
  Most of these are also available in KOBOLDCPP too (via settings -> samplers) after start up (no "llamacpp_HF loader" step required).
 
868
  These control selection of tokens, in some case providing more relevant and/or more options. See [ https://arxiv.org/pdf/2210.15191 ]
869
 
870
 
871
+ <B>MAIN ADVANCED SAMPLERS PART 2 (affects per token AND overall generation): </B>
872
 
873
  What I will touch on here are special settings for CLASS 3 and CLASS 4 models (for the first TWO samplers).
874