File size: 2,041 Bytes
84411e9 07f3986 fabc4f0 07f3986 fabc4f0 07f3986 b934050 07f3986 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
# Lemon Kunoichi Wizard - 7b
A merge of three models, LemonadeRP-4.5.3, Kunoichi-DPO-v2, and WizardLM-2. I used Lemonade as a base with Kunoichi being the second bigest influence and WizardLM-2 for logic capabilities.
The end result is a Roleplay-focused model with great character card inference. I ran 4 merges at varying values to see which provided the most accurate output to a character cards quirk, with this v3 version being the winner out of the four.
## Context Template - Alpaca
Alpaca preset seems to work well with your own System Prompt.
## Context Size - 8192
The model loads at 8192 on my end, but theoretically it should be able to go up to 32k. Not that it'll be coherent at 32k. Most models based on Mistral like this end up being - at best - 12k context size for coherent output. I only tested at 8k which is where the base models tend to shine. YMMV otherwise.
---
base_model:
- SanjiWatsuki/Kunoichi-DPO-v2-7B
- dreamgen/WizardLM-2-7B
- KatyTheCutie/LemonadeRP-4.5.3
library_name: transformers
tags:
- mergekit
- merge
---
# merge
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.
### Models Merged
The following models were included in the merge:
* [SanjiWatsuki/Kunoichi-DPO-v2-7B](https://huggingface.co/SanjiWatsuki/Kunoichi-DPO-v2-7B)
* [dreamgen/WizardLM-2-7B](https://huggingface.co/dreamgen/WizardLM-2-7B)
* [KatyTheCutie/LemonadeRP-4.5.3](https://huggingface.co/KatyTheCutie/LemonadeRP-4.5.3)
### Configuration
The following YAML configuration was used to produce this model:
```yaml
models:
- model: KatyTheCutie/LemonadeRP-4.5.3
parameters:
weight: 1.0
- model: dreamgen/WizardLM-2-7B
parameters:
weight: 0.2
- model: SanjiWatsuki/Kunoichi-DPO-v2-7B
parameters:
weight: 0.6
merge_method: linear
dtype: float16
``` |