File size: 3,599 Bytes
2010bee 3a4abcf 2010bee 3a4abcf abaa4fe 3a4abcf 2010bee 5f68b3e 2010bee 5366f4a 2010bee abaa4fe 3a4abcf abaa4fe 3a4abcf 2010bee 3a4abcf 2010bee |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 |
---
library_name: transformers
tags:
- mergekit
- merge
- mistral
---
<img src="https://huggingface.co/Virt-io/Deris-SSS-7B/resolve/main/README.png">
---
# Tips
SillyTavern presets in [presets folder](https://huggingface.co/Virt-io/Deris-SSS-7B/tree/main/presets). (Thanks to [Lewdiculous](https://huggingface.co/Lewdiculous) for ideas.)
Model has formatting issues when using asterisks. Recommended to use novel like formatting (only use quotes)
System prompt can be improved, help welcomed.
Model seems to take characters too seriously, if you find it too stubborn regenerate reply or edit it. It should comply after.(Seems to be fixed now, appears to have been a bad system prompt. oops)
---
# Models Used
[ChaoticNeutrals/Eris_Floramix_DPO_7B](https://huggingface.co/ChaoticNeutrals/Eris_Floramix_DPO_7B)
[ResplendentAI/Datura_7B](https://huggingface.co/ResplendentAI/Datura_7B)
[FuseAI/FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM)
[Weyaxi/Einstein-v4-7B](https://huggingface.co/Weyaxi/Einstein-v4-7B)
[ogno-monarch-jaskier-merge-7b-OH-PREF-DPO](https://huggingface.co/eren23/ogno-monarch-jaskier-merge-7b-OH-PREF-DPO)
---
# Deris-SSS
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the SLERP merge method.
### Models Merged
The following models were included in the merge:
* Deris-v2
* SmartyPants-step2
### Configuration
The following YAML configuration was used to produce this model:
### Deris-SSS
Final merge combine the smart models with the unhinged ones
```yaml
slices:
- sources:
- model: ./Mergekit/Deris-v2
layer_range: [0, 32]
- model: ./Mergekit/SmartyPants-step2
layer_range: [0, 32]
merge_method: slerp
base_model: ./Mergekit/Deris-v2
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: float16
```
### Deris-v2
Combine a bit of Datura_7B into Eris_Floramix_DPO_7B
Reason Datura is extremely unhinged :) more so than Eris
```
slices:
- sources:
- model: ChaoticNeutrals/Eris_Floramix_DPO_7B
layer_range: [0, 32]
- model: ResplendentAI/Datura_7B
layer_range: [0, 32]
merge_method: slerp
base_model: ChaoticNeutrals/Eris_Floramix_DPO_7B
parameters:
t:
- filter: self_attn
value: [0, 0.20, 0.15, 0.25, 0.35]
- filter: mlp
value: [0.35, 0.20, 0.25, 0.15, 0]
- value: 0.20
dtype: float16
```
### SmartyPants-step1
Combine OMJ into Einstein
Reason Einstein looks interesting and OMJ was a high ranking model
```
slices:
- sources:
- model: Weyaxi/Einstein-v4-7B
layer_range: [0, 32]
- model: eren23/ogno-monarch-jaskier-merge-7b-OH-PREF-DPO
layer_range: [0, 32]
merge_method: slerp
base_model: Weyaxi/Einstein-v4-7B
parameters:
t:
- filter: self_attn
value: [0, 0.45, 0.3, 0.55, 0.65]
- filter: mlp
value: [0.65, 0.45, 0.55, 0.3, 0]
- value: 0.45
dtype: float16
```
### SmartyPants-step2
Combine Smarty pants into FuseChat-VaRM
Reason IDK I just like FuseChat-VaRM
```
slices:
- sources:
- model: FuseAI/FuseChat-7B-VaRM
layer_range: [0, 32]
- model: ./Mergekit/SmartyPants-step1
layer_range: [0, 32]
merge_method: slerp
base_model: FuseAI/FuseChat-7B-VaRM
parameters:
t:
- filter: self_attn
value: [0, 0.45, 0.3, 0.55, 0.65]
- filter: mlp
value: [0.65, 0.45, 0.55, 0.3, 0]
- value: 0.45
dtype: float16
```
|