fernandofernandes
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ An experimentation regarding 'lasering' each expert to denoise and enhance model
|
|
13 |
This model has half size in comparison to the Mixtral 8x7b Instruct. And it basically has the same level of performance (we are working to get a better MMLU score).
|
14 |
|
15 |
|
16 |
-
# Laserxtral - 4x7b (all lasered using laserRMT)
|
17 |
|
18 |
This model is a Mixture of Experts (MoE) made with [mergekit](https://github.com/cg123/mergekit) (mixtral branch). It uses the following base models:
|
19 |
* [cognitivecomputations/dolphin-2.6-mistral-7b-dpo](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo)
|
@@ -22,15 +22,6 @@ This model is a Mixture of Experts (MoE) made with [mergekit](https://github.com
|
|
22 |
* [Q-bert/MetaMath-Cybertron-Starling](https://huggingface.co/Q-bert/MetaMath-Cybertron-Starling)
|
23 |
* [WizardLM/WizardMath-7B-V1.1](https://huggingface.co/WizardLM/WizardMath-7B-V1.1)
|
24 |
|
25 |
-
*mlabonne/Marcoro14-7B-slerp (base)
|
26 |
-
|
27 |
-
*cognitivecomputations/dolphin-2.6-mistral-7b-dpo
|
28 |
-
|
29 |
-
*beowolx/CodeNinja-1.0-OpenChat-7B
|
30 |
-
|
31 |
-
*Q-bert/MetaMath-Cybertron-Starling
|
32 |
-
|
33 |
-
*WizardLM/WizardMath-7B-V1.1
|
34 |
|
35 |
It follows the implementation of laserRMT @ https://github.com/cognitivecomputations/laserRMT
|
36 |
|
|
|
13 |
This model has half size in comparison to the Mixtral 8x7b Instruct. And it basically has the same level of performance (we are working to get a better MMLU score).
|
14 |
|
15 |
|
16 |
+
# Laserxtral - 4x7b (all, except for base, lasered using laserRMT)
|
17 |
|
18 |
This model is a Mixture of Experts (MoE) made with [mergekit](https://github.com/cg123/mergekit) (mixtral branch). It uses the following base models:
|
19 |
* [cognitivecomputations/dolphin-2.6-mistral-7b-dpo](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo)
|
|
|
22 |
* [Q-bert/MetaMath-Cybertron-Starling](https://huggingface.co/Q-bert/MetaMath-Cybertron-Starling)
|
23 |
* [WizardLM/WizardMath-7B-V1.1](https://huggingface.co/WizardLM/WizardMath-7B-V1.1)
|
24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
25 |
|
26 |
It follows the implementation of laserRMT @ https://github.com/cognitivecomputations/laserRMT
|
27 |
|