DavidAU commited on
Commit
0c65e54
1 Parent(s): cbb7e45

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -133,6 +133,33 @@ Quants:
133
 
134
  Please note for Q2k quant you may need to raise rep pen and lower temp to account for quality loss at this quant level.
135
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
136
  <B>Model Template:</B>
137
 
138
  This is a custom model, and requires ChatML OR Alpaca OR Vicuna template, but may work with other template(s) and has maximum context of 4k / 4096.
 
133
 
134
  Please note for Q2k quant you may need to raise rep pen and lower temp to account for quality loss at this quant level.
135
 
136
+ <B>Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:</B>
137
+
138
+ In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;
139
+
140
+ Set the "Smoothing_factor" to 1.5 to 2.5
141
+
142
+ : in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"
143
+
144
+ : in text-generation-webui -> parameters -> lower right.
145
+
146
+ : In Silly Tavern this is called: "Smoothing"
147
+
148
+
149
+ NOTE: For "text-generation-webui"
150
+
151
+ -> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)
152
+
153
+ Source versions (and config files) of my models are here:
154
+
155
+ https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be
156
+
157
+ OTHER OPTIONS:
158
+
159
+ - Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
160
+
161
+ - If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.
162
+
163
  <B>Model Template:</B>
164
 
165
  This is a custom model, and requires ChatML OR Alpaca OR Vicuna template, but may work with other template(s) and has maximum context of 4k / 4096.