Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
## Notes
|
| 2 |
+
There is no template, just BOS+text
|
| 3 |
+
|
| 4 |
+
It can also start from nothing
|
| 5 |
+
|
| 6 |
+
Temperature, repetition penalty, etc should all be left as defaults
|
| 7 |
+
|
| 8 |
+
It will not go lewd immediately, it will try to form a coherent story
|
| 9 |
+
|
| 10 |
+
It's best to generate 1~3 paragraphs at a time, it loses coherence if you try to make it generate the full context all at once
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
## LLaMA-3-8B base
|
| 15 |
+
RoPEd to 16k context
|
| 16 |
+
|
| 17 |
+
| Name | Quant | Size | VRAM (With FA) | VRAM (No FA) |
|
| 18 |
+
|---------------------------------------|--------|------|----------------|--------------|
|
| 19 |
+
| llama-3-8b-lewd-stories-v6-16k.F16 | F16 | 14.9 | 16.6 | 17.4 |
|
| 20 |
+
| llama-3-8b-lewd-stories-v6-16k.Q8_0 | Q8_0 | 8.0 | 10.1 | 10.5 |
|
| 21 |
+
| llama-3-8b-lewd-stories-v6-16k.Q6_K | Q6_K | 6.1 | 8.4 | 9.2 |
|
| 22 |
+
| llama-3-8b-lewd-stories-v6-16k.Q5_K_M | Q5_K_M | 5.3 | 7.6 | 8.1 |
|
| 23 |
+
| llama-3-8b-lewd-stories-v6-16k.Q4_K_M | Q4_K_M | 4.6 | 6.9 | 7.8 |
|
| 24 |
+
|
| 25 |
+
## Yi-1.5-9B-32K
|
| 26 |
+
|
| 27 |
+
Native 32k context
|
| 28 |
+
|
| 29 |
+
| Name | Quant | Size | VRAM (With FA) | VRAM (No FA) |
|
| 30 |
+
|----------------------------|--------|------|----------------|--------------|
|
| 31 |
+
| yi-lewd-stories-32k.F16 | F16 | 16.4 | | |
|
| 32 |
+
| yi-lewd-stories-32k.Q8_0 | Q8_0 | 8.7 | | |
|
| 33 |
+
| yi-lewd-stories-32k.Q6_K | Q6_K | 6.7 | | |
|
| 34 |
+
| yi-lewd-stories-32k.Q5_K_M | Q5_K_M | 5.8 | | |
|
| 35 |
+
| yi-lewd-stories-32k.Q4_K_M | Q4_K_M | 5.0 | | |
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
## Mistral-7B-v0.3
|
| 40 |
+
|
| 41 |
+
Native 32k context
|
| 42 |
+
|
| 43 |
+
| Name | Quant | Size | VRAM (With FA) | VRAM (No FA) |
|
| 44 |
+
|---------------------------------|--------|------|----------------|--------------|
|
| 45 |
+
| mistral-lewd-stories-32k.F16 | F16 | 13.5 | | |
|
| 46 |
+
| mistral-lewd-stories-32k.Q8_0 | Q8_0 | 7.2 | | |
|
| 47 |
+
| mistral-lewd-stories-32k.Q6_K | Q6_K | 5.5 | | |
|
| 48 |
+
| mistral-lewd-stories-32k.Q5_K_M | Q5_K_M | 4.8 | | |
|
| 49 |
+
| mistral-lewd-stories-32k.Q4_K_M | Q4_K_M | 4.0 | | |
|