Ttimofeyka
commited on
Commit
•
9e77667
1
Parent(s):
67b5d41
Update README.md
Browse files
README.md
CHANGED
@@ -1,7 +1,47 @@
|
|
1 |
-
---
|
2 |
-
|
3 |
-
|
4 |
-
-
|
5 |
-
|
6 |
-
-
|
7 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
base_model:
|
3 |
+
- arcee-ai/SuperNova-Medius
|
4 |
+
- Ttimofeyka/Tissint-14B-128k-RP
|
5 |
+
library_name: transformers
|
6 |
+
license: apache-2.0
|
7 |
+
tags:
|
8 |
+
- unsloth
|
9 |
+
- trl
|
10 |
+
- sft
|
11 |
+
---
|
12 |
+
# Tissint-14B-v1.1-128k-RP
|
13 |
+
---
|
14 |
+
![Chat Example](https://i.ibb.co/VqgjW3H/image.png)
|
15 |
+
---
|
16 |
+
The model is based on [SuperNova-Medius](https://huggingface.co/arcee-ai/SuperNova-Medius) (as the current best 14B model) with a 128k context with an emphasis on creativity, including NSFW and multi-turn conversations.
|
17 |
+
|
18 |
+
According to my tests, this finetune is much more stable with different samplers than the original model. Censorship and refusals have been reduced.
|
19 |
+
|
20 |
+
The model started to follow the system prompt better, and the responses in ChatML format with bad samplers stopped reaching 800+ tokens for no reason.
|
21 |
+
|
22 |
+
### V1.1
|
23 |
+
|
24 |
+
I have increased the amount of dataset and added instructions mixed with NSFW RP, which in theory will improve the quality of the model.
|
25 |
+
|
26 |
+
### Chat Template - ChatML
|
27 |
+
## Samplers
|
28 |
+
|
29 |
+
### Balance
|
30 |
+
```
|
31 |
+
Temp : 0.8 - 1.15
|
32 |
+
Min P : 0.1
|
33 |
+
|
34 |
+
Repetition Penalty : 1.02
|
35 |
+
|
36 |
+
DRY 0.8, 1.75, 2, 2048 (change to 4096 or more if needed)
|
37 |
+
```
|
38 |
+
|
39 |
+
### Creativity
|
40 |
+
```
|
41 |
+
Temp : 1.15 - 1.5
|
42 |
+
Top P : 0.9
|
43 |
+
|
44 |
+
Repetition Penalty : 1.03
|
45 |
+
|
46 |
+
DRY 0.82, 1.75, 2, 2048 (change to 4096 or more if needed)
|
47 |
+
```
|