Update README.md
Browse files
README.md
CHANGED
@@ -3,11 +3,11 @@ base_model: []
|
|
3 |
tags:
|
4 |
- mergekit
|
5 |
- merge
|
6 |
-
|
7 |
---
|
8 |
# merge
|
9 |
|
10 |
-
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
11 |
|
12 |
## Merge Details
|
13 |
### Merge Method
|
@@ -17,8 +17,19 @@ This model was merged using the SLERP merge method.
|
|
17 |
### Models Merged
|
18 |
|
19 |
The following models were included in the merge:
|
20 |
-
* [DiscoResearch/DiscoLM_German_7b_v1](https://huggingface.co/DiscoResearch/DiscoLM_German_7b_v1)
|
21 |
* [OpenPipe/mistral-ft-optimized-1227](https://huggingface.co/OpenPipe/mistral-ft-optimized-1227)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
|
23 |
### Configuration
|
24 |
|
|
|
3 |
tags:
|
4 |
- mergekit
|
5 |
- merge
|
6 |
+
- + german
|
7 |
---
|
8 |
# merge
|
9 |
|
10 |
+
This is a experimental merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
11 |
|
12 |
## Merge Details
|
13 |
### Merge Method
|
|
|
17 |
### Models Merged
|
18 |
|
19 |
The following models were included in the merge:
|
|
|
20 |
* [OpenPipe/mistral-ft-optimized-1227](https://huggingface.co/OpenPipe/mistral-ft-optimized-1227)
|
21 |
+
* [DiscoResearch/DiscoLM_German_7b_v1](https://huggingface.co/DiscoResearch/DiscoLM_German_7b_v1)
|
22 |
+
|
23 |
+
####Why this two models?
|
24 |
+
|
25 |
+
Because both used models are, to my knowledge, the two best models when it comes to German language generation.
|
26 |
+
|
27 |
+
DiscoLM German 7B is is up to this date (01/21/2024) by far the best German model and makes far fewer grammatical errors and his German generally sounds good. But it is finetuned on Mistral V0.2 or even V0.1.
|
28 |
+
|
29 |
+
Mistral FT Optimized 1227 is much better in German than Mistral 7B V0.2 and other German fine-tuning models that make grammar errors in almost every sentence. But even that model is a good step behind DiscoLM German 7B and creates not so well formed German sentences.
|
30 |
+
|
31 |
+
The ulterior motive was now combining this two models to get a even better German model, especially for German roleplay.
|
32 |
+
|
33 |
|
34 |
### Configuration
|
35 |
|