Update README.md
Browse files
README.md
CHANGED
@@ -43,11 +43,11 @@ This model was trained with QLoRa in bfloat16 with flash attention 2 on oen A100
|
|
43 |
|
44 |
## Evaluation results
|
45 |
|
46 |
-
The model was evaluated using [scandeval](https://scandeval.com/dutch-nlg/).
|
47 |
|
48 |
| Model| conll_nl | dutch_social | scala_nl | squad_nl | wiki_lingua_nl | mmlu_nl | hellaswag_nl |
|
49 |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:
|
50 |
-
Reynaerde-7B-Chat | 56.40 / 38.13 | 10.83 / 27.67 | 20.02 / 55.40 | 53.56 / 65.29 |
|
51 |
Mistral-7B-v0.3 | 57.08 / 42.65 | 14.05 / 39.13 | 8.08 / 43.07 | 45.57 / 55.20 | 62.28 / 16.46 | 20.39 / 40.03 | 13.28 / 34.13
|
52 |
Mistral-7B-v0.3-Instruct | 60.76 / 45.39 | 13.20 / 34.26 | 23.23 / 59.26 | 48.94 / 60.13 | 66.09 / 18.02 | 24.95 / 43.67 | 24.86 / 43.57
|
53 |
|
|
|
43 |
|
44 |
## Evaluation results
|
45 |
|
46 |
+
The model was evaluated using [scandeval](https://scandeval.com/dutch-nlg/). There are improvements in 4/7 benchmarks compared to the Mistral-7B-v0.3-Instruct model.
|
47 |
|
48 |
| Model| conll_nl | dutch_social | scala_nl | squad_nl | wiki_lingua_nl | mmlu_nl | hellaswag_nl |
|
49 |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:
|
50 |
+
Reynaerde-7B-Chat | 56.40 / 38.13 | 10.83 / 27.67 | 20.02 / 55.40 | 53.56 / 65.29 | 68.13 / 20.85 | 32.50 / 49.10 | 31.36 / 47.79
|
51 |
Mistral-7B-v0.3 | 57.08 / 42.65 | 14.05 / 39.13 | 8.08 / 43.07 | 45.57 / 55.20 | 62.28 / 16.46 | 20.39 / 40.03 | 13.28 / 34.13
|
52 |
Mistral-7B-v0.3-Instruct | 60.76 / 45.39 | 13.20 / 34.26 | 23.23 / 59.26 | 48.94 / 60.13 | 66.09 / 18.02 | 24.95 / 43.67 | 24.86 / 43.57
|
53 |
|