File size: 3,702 Bytes
a7dc396 e9851f2 099644a 10e1946 937c00d 10e1946 099644a 99d4bcc a7dc396 601603e a7dc396 601603e a7dc396 601603e a7dc396 601603e a7dc396 601603e a7dc396 e9851f2 a7dc396 e9851f2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
---
base_model:
- SentientAGI/Dobby-Unhinged-Llama-3.3-70B
- Black-Ink-Guild/Pernicious_Prophecy_70B
- LatitudeGames/Wayfarer-Large-70B-Llama-3.3
- TheDrummer/Fallen-Llama-3.3-R1-70B-v1
- Nexesenex/Llama_3.x_70b_SmarTracks_V1.01
- Nexesenex/Llama_3.1_70b_TearDrops_V1.11
library_name: transformers
tags:
- mergekit
- merge
license: llama3.3
---
# about
After a lot of "lego merges" to experiment, let's start a basket merge series!
Base is the third version of Smarteaz, Smartracks, in which the R1 model is itself a merge between R1, R1 without chinese censorship, and R1 Fallen Llama.
That base has shown itself excellent to empower any model thrown at it. Nemotron and Tulu complete the mix.
My 5 favorite L3.3 (Negative Llama, EVA, Dobby, Fallen Llama ofc and Wayfarer) are included in submerges, starting with the well doted Permiscious Prophecy (Including a bit of Sao10K's Euryale 2.2 through the 70Blivion model).
Hermes and Tess are also included in submerges, in their abliterated version. Hermes has also its Gutemberg Doppel version.
Some abliterated or uncensored L3 are also wrapped in, like Lumitron Abliterated (including some NeverSleep work) or Creative Llama.
---
# benchs
Benchs are traded for creativity in this merge, so :
- PPL Wikitext Eng 512 : 3.54 (good)
- ARC-C : 59.20 (good)
- ARC-E : 80.70 (good also)
---
# merge
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [Nexesenex/Llama_3.x_70b_SmarTracks_V1.01](https://huggingface.co/Nexesenex/Llama_3.x_70b_SmarTracks_V1.01) as a base.
- The "Smart base" of the model, a 3 levels merge-stock mix of Llama 3.3 abliterated finetuned (the root), Deepseek R1 Distill based Fallen Llama, Nemotron and Tulu.
### Models Merged
The following models were included in the merge:
* [SentientAGI/Dobby-Unhinged-Llama-3.3-70B](https://huggingface.co/SentientAGI/Dobby-Unhinged-Llama-3.3-70B)
- For its.. unhinged "personality traits".
* [Black-Ink-Guild/Pernicious_Prophecy_70B](https://huggingface.co/Black-Ink-Guild/Pernicious_Prophecy_70B)
- A balanced healed merge-stock steering with Eva (creativity), Negative Llama (debiasing), L3.1 Oblivion (general intelligence), Open-Bio (anatomy and medicine).
* [LatitudeGames/Wayfarer-Large-70B-Llama-3.3](https://huggingface.co/LatitudeGames/Wayfarer-Large-70B-Llama-3.3)
- To darken the model and the RP scenarios.
* [TheDrummer/Fallen-Llama-3.3-R1-70B-v1](https://huggingface.co/TheDrummer/Fallen-Llama-3.3-R1-70B-v1)
- To highlight and consolidate the R1 capabilities and spice-up/darken the model.
* [Nexesenex/Llama_3.1_70b_TearDrops_V1.11](https://huggingface.co/Nexesenex/Llama_3.1_70b_TearDrops_V1.11)
- A legacy 3.1 merge, led by Tess R1, including the Hermes based Gutemberg Doppel and an uncensored creative finetune.
### Configuration
The following YAML configuration was used to produce this model:
```yaml
merge_method: model_stock
models:
- model: Nexesenex/Llama_3.1_70b_TearDrops_V1.11
parameters:
weight: 1.0
- model: Black-Ink-Guild/Pernicious_Prophecy_70B
parameters:
weight: 1.0
- model: SentientAGI/Dobby-Unhinged-Llama-3.3-70B
parameters:
weight: 1.0
- model: TheDrummer/Fallen-Llama-3.3-R1-70B-v1
parameters:
weight: 1.0
- model: LatitudeGames/Wayfarer-Large-70B-Llama-3.3
parameters:
weight: 1.0
base_model: Nexesenex/Llama_3.x_70b_SmarTracks_V1.01
dtype: bfloat16
out_dtype: bfloat16
parameters:
int8_mask: true
normalize: true
rescale: false
chat_template: auto
tokenizer:
source: union
``` |