Update README.md
Browse files
README.md
CHANGED
@@ -121,9 +121,9 @@ model-index:
|
|
121 |
|
122 |
# Daredevil-8B
|
123 |
|
124 |
-
|
125 |
|
126 |
-
|
127 |
* [nbeerbower/llama-3-stella-8B](https://huggingface.co/nbeerbower/llama-3-stella-8B)
|
128 |
* [Hastagaras/llama-3-8b-okay](https://huggingface.co/Hastagaras/llama-3-8b-okay)
|
129 |
* [nbeerbower/llama-3-gutenberg-8B](https://huggingface.co/nbeerbower/llama-3-gutenberg-8B)
|
@@ -134,6 +134,8 @@ Daredevil-8B is a merge of the following models using [LazyMergekit](https://col
|
|
134 |
* [flammenai/Mahou-1.1-llama3-8B](https://huggingface.co/flammenai/Mahou-1.1-llama3-8B)
|
135 |
* [KingNish/KingNish-Llama3-8b](https://huggingface.co/KingNish/KingNish-Llama3-8b)
|
136 |
|
|
|
|
|
137 |
## π Applications
|
138 |
|
139 |
It is a highly functional censored model. You might want to add `<end_of_turn>` as an additional stop string.
|
@@ -144,6 +146,14 @@ It is a highly functional censored model. You might want to add `<end_of_turn>`
|
|
144 |
|
145 |
## π Evaluation
|
146 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
147 |
| Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
|
148 |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------: | --------: | --------: | ---------: | --------: |
|
149 |
| [**mlabonne/Daredevil-8B**](https://huggingface.co/mlabonne/Daredevil-8B) [π](https://gist.github.com/mlabonne/080f9c5f153ea57a7ab7d932cf896f21) | **55.87** | **44.13** | **73.52** | **59.05** | **46.77** |
|
@@ -226,16 +236,3 @@ pipeline = transformers.pipeline(
|
|
226 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
227 |
print(outputs[0]["generated_text"])
|
228 |
```
|
229 |
-
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
230 |
-
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_mlabonne__Daredevil-8B)
|
231 |
-
|
232 |
-
| Metric |Value|
|
233 |
-
|---------------------------------|----:|
|
234 |
-
|Avg. |72.41|
|
235 |
-
|AI2 Reasoning Challenge (25-Shot)|68.86|
|
236 |
-
|HellaSwag (10-Shot) |84.50|
|
237 |
-
|MMLU (5-Shot) |69.24|
|
238 |
-
|TruthfulQA (0-shot) |59.89|
|
239 |
-
|Winogrande (5-shot) |78.45|
|
240 |
-
|GSM8k (5-shot) |73.54|
|
241 |
-
|
|
|
121 |
|
122 |
# Daredevil-8B
|
123 |
|
124 |
+
Daredevil-8B is a mega-merge designed to maximize MMLU. On 27 May 24, it is the Llama 8B model with the **highest MMLU score**.
|
125 |
|
126 |
+
It is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
|
127 |
* [nbeerbower/llama-3-stella-8B](https://huggingface.co/nbeerbower/llama-3-stella-8B)
|
128 |
* [Hastagaras/llama-3-8b-okay](https://huggingface.co/Hastagaras/llama-3-8b-okay)
|
129 |
* [nbeerbower/llama-3-gutenberg-8B](https://huggingface.co/nbeerbower/llama-3-gutenberg-8B)
|
|
|
134 |
* [flammenai/Mahou-1.1-llama3-8B](https://huggingface.co/flammenai/Mahou-1.1-llama3-8B)
|
135 |
* [KingNish/KingNish-Llama3-8b](https://huggingface.co/KingNish/KingNish-Llama3-8b)
|
136 |
|
137 |
+
Thanks to nbeerbower, Hastagaras, openchat, Kukedlc, cstr, flammenai, and KingNish for their merges. Special thanks to Charles Goddard and Arcee.ai for MergeKit.
|
138 |
+
|
139 |
## π Applications
|
140 |
|
141 |
It is a highly functional censored model. You might want to add `<end_of_turn>` as an additional stop string.
|
|
|
146 |
|
147 |
## π Evaluation
|
148 |
|
149 |
+
### Open LLM Leaderboard
|
150 |
+
|
151 |
+
Here's the Open LLM Leaderboard sorted by MMLU score.
|
152 |
+
|
153 |
+

|
154 |
+
|
155 |
+
### Nous
|
156 |
+
|
157 |
| Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
|
158 |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------: | --------: | --------: | ---------: | --------: |
|
159 |
| [**mlabonne/Daredevil-8B**](https://huggingface.co/mlabonne/Daredevil-8B) [π](https://gist.github.com/mlabonne/080f9c5f153ea57a7ab7d932cf896f21) | **55.87** | **44.13** | **73.52** | **59.05** | **46.77** |
|
|
|
236 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
237 |
print(outputs[0]["generated_text"])
|
238 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|