Update README.md
Browse files
README.md
CHANGED
@@ -1,28 +1,28 @@
|
|
1 |
---
|
2 |
-
base_model:
|
|
|
|
|
|
|
|
|
3 |
library_name: transformers
|
4 |
tags:
|
5 |
- mergekit
|
6 |
- merge
|
7 |
-
|
8 |
---
|
9 |
# MiquSuperdark-70B-v1
|
10 |
|
11 |
-
|
12 |
-
|
13 |
-
## Merge Details
|
14 |
-
### Merge Method
|
15 |
|
16 |
-
This
|
|
|
|
|
|
|
|
|
17 |
|
18 |
-
|
19 |
|
20 |
-
|
21 |
-
* /home/dylan/Documents/AI/merge/MiquMaid-v3-70B
|
22 |
-
* /media/dylan/SanDisk/LLMs/miqu-evil-dpo/
|
23 |
-
* /media/dylan/SanDisk/LLMs/Midnight-Miqu-70B-v1.5
|
24 |
-
|
25 |
-
### Configuration
|
26 |
|
27 |
The following YAML configuration was used to produce this model:
|
28 |
|
@@ -45,3 +45,5 @@ models:
|
|
45 |
dtype: float16
|
46 |
tokenizer_source: model:/home/dylan/Documents/AI/merge/miqu-1-70b-sf
|
47 |
```
|
|
|
|
|
|
1 |
---
|
2 |
+
base_model:
|
3 |
+
- sophosympatheia/Midnight-Miqu-70B-v1.5
|
4 |
+
- NeverSleep/MiquMaid-v3-70B
|
5 |
+
- maywell/miqu-evil-dpo
|
6 |
+
- 152334H/miqu-1-70b-sf
|
7 |
library_name: transformers
|
8 |
tags:
|
9 |
- mergekit
|
10 |
- merge
|
11 |
+
license: other
|
12 |
---
|
13 |
# MiquSuperdark-70B-v1
|
14 |
|
15 |
+
**MiquSuperdark-70B-v1** is a merge of three of the most popular Miqu-derived models, along with Miqu itself. The goal of the merge is to create an strong, well-rounded chat model that picks up desirable traits from its constituent models without sacrificing intelligence.
|
|
|
|
|
|
|
16 |
|
17 |
+
This is a DARE Linear merge with the following composition:
|
18 |
+
- [sophosympatheia/Midnight-Miqu-70B-v1.5](https://huggingface.co/sophosympatheia/Midnight-Miqu-70B-v1.5) at weight 0.4
|
19 |
+
- [NeverSleep/MiquMaid-v3-70B](https://huggingface.co/NeverSleep/MiquMaid-v3-70B) at weight 0.2
|
20 |
+
- [maywell/miqu-evil-dpo](https://huggingface.co/maywell/miqu-evil-dpo) at weight 0.2
|
21 |
+
- [152334H/miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf) at weight 0.2 (used as base model)
|
22 |
|
23 |
+
DARE Linear was chosen as the merge method based on [this HF discussion](https://huggingface.co/jukofyork/Dark-Miqu-70B/discussions/2), in which the creator of Midnight-Miqu says "*in my own testing I consistently got the best results from using a dare_linear merge when working with miqu models*".
|
24 |
|
25 |
+
## Merge Configuration
|
|
|
|
|
|
|
|
|
|
|
26 |
|
27 |
The following YAML configuration was used to produce this model:
|
28 |
|
|
|
45 |
dtype: float16
|
46 |
tokenizer_source: model:/home/dylan/Documents/AI/merge/miqu-1-70b-sf
|
47 |
```
|
48 |
+
|
49 |
+
The tokenizer is copied from the base model [152334H/miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf).
|