metadata
base_model:
- NeuralNovel/Senzu-7B-v0.1-DPO
- senseable/WestLake-7B-v2
library_name: transformers
tags:
- mergekit
- merge
license: apache-2.0
It's experimental, but seems fine for me, I didn't run it deeply yet but should be good for Role-play 😈 considering the two merged models, feel free to leave a suggestion or feedback.
This is a merge of pre-trained language models created using mergekit(my experimental branch swapping here )
Merge Details
Merge Method
This model was merged using the task_swapping merge method using NeuralNovel/Senzu-7B-v0.1-DPO as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
merge_method: task_swapping
base_model: NeuralNovel/Senzu-7B-v0.1-DPO
models:
- model: senseable/WestLake-7B-v2
parameters:
weight: 0.75
diagonal_offset: 2 #it doesn't do anything when you use random_mask
random_mask: 0.3333
random_mask_seed: 98557
dtype: bfloat16