File size: 2,152 Bytes
d357fd4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
--- base_model: - inflatebot/MN-12B-Mag-Mell-R1 - TheDrummer/UnslopNemo-12B-v4.1 library_name: transformers tags: - mergekit - merge - 12b - chat - creative --- # patricide-Unslop-Mell >The sins of the Father shan't ever be repeated this way. # WARNING: Something went wrong during the upload! It will be fixed soon. This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). This is my first merge — I still have no idea how writing the parameters in the config actually works. If anyone has more extensive guides for merging, please let me know. I would also like to get into the science behind all this. Both models produced enjoyable results, so I decided to merge them, to create a model hopefully inheriting good traits of the parents. I've tested this model on the *Q_6K GGUF* Quant (will get uploaded later) and it provided satisfactory results, thus I decided to upload it. Although I've not extensively tested it in Storywriting nor RP, the results were stable and *at least* coherent. I tested it on a **Temperature of 1** (Temperature last) and **Min-P of 0.1**. I don't know the effects **DRY** or **XTC** have on the stability of the output, or how it fares on high context sizes. Both parent models use the **ChatML** Template. Although [Unslop-Nemo](https://huggingface.co/TheDrummer/UnslopNemo-12B-v4.1) also uses **Metharme/Pygmalion**. I've not yet tested which works better. Feel free to experiment, as I am only experimenting myself. ## Merge Details ### Merge Method This model was merged using the SLERP merge method. ### Models Merged The following models were included in the merge: * [inflatebot/MN-12B-Mag-Mell-R1](https://huggingface.co/inflatebot/MN-12B-Mag-Mell-R1) * [TheDrummer/UnslopNemo-12B-v4.1](https://huggingface.co/TheDrummer/UnslopNemo-12B-v4.1) ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: TheDrummer/UnslopNemo-12B-v4.1 - model: inflatebot/MN-12B-Mag-Mell-R1 merge_method: slerp base_model: TheDrummer/UnslopNemo-12B-v4.1 dtype: bfloat16 parameters: t: [0, 0.5, 1, 0.5, 0] ``` |