Update README.md
Browse files
README.md
CHANGED
@@ -1,9 +1,19 @@
|
|
1 |
---
|
2 |
-
license:
|
3 |
language:
|
4 |
- en
|
5 |
---
|
6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
# Model Description
|
8 |
This is an experiment to test merging 14 models using DARE TIES 🦙
|
9 |
|
@@ -90,9 +100,9 @@ models:
|
|
90 |
weight: 0.08
|
91 |
density: 0.5
|
92 |
merge_method: dare_ties
|
93 |
-
base_model: /
|
94 |
parameters:
|
95 |
int8_mask: true
|
96 |
dtype: bfloat16
|
97 |
|
98 |
-
```
|
|
|
1 |
---
|
2 |
+
license: cc
|
3 |
language:
|
4 |
- en
|
5 |
---
|
6 |
|
7 |
+
# Update 2023-12-19
|
8 |
+
|
9 |
+
In light of [dataset contamination issue among the merged models](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard/discussions/474)
|
10 |
+
raised by the community in recent days, in particular
|
11 |
+
[berkeley-nest/Starling-LM-7B-alpha](https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha), and
|
12 |
+
[Q-bert/MetaMath-Cybertron-Starling](https://huggingface.co/Q-bert/MetaMath-Cybertron-Starling),
|
13 |
+
we decided to remake another model without the models mentioned.
|
14 |
+
Additionally, their CC-by-NC-4.0 license is restrictive and thus are not suitable for an open model.
|
15 |
+
|
16 |
+
|
17 |
# Model Description
|
18 |
This is an experiment to test merging 14 models using DARE TIES 🦙
|
19 |
|
|
|
100 |
weight: 0.08
|
101 |
density: 0.5
|
102 |
merge_method: dare_ties
|
103 |
+
base_model: mistralai/Mistral-7B-v0.1
|
104 |
parameters:
|
105 |
int8_mask: true
|
106 |
dtype: bfloat16
|
107 |
|
108 |
+
```
|