Adding Evaluation Results
#3
by
leaderboard-pr-bot
- opened
README.md
CHANGED
@@ -1,5 +1,108 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
4 |
|
5 |
## Intel's Neural Chat v3-3 8x7B Mixtral MOE
|
@@ -67,3 +170,17 @@ Therefore, before deploying any applications of neural-chat-7b-v3-3, developers
|
|
67 |
The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please cosult an attorney before using this model for commercial purposes.
|
68 |
|
69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
+
model-index:
|
4 |
+
- name: neural-chat-v3-3-8x7b-MoE
|
5 |
+
results:
|
6 |
+
- task:
|
7 |
+
type: text-generation
|
8 |
+
name: Text Generation
|
9 |
+
dataset:
|
10 |
+
name: AI2 Reasoning Challenge (25-Shot)
|
11 |
+
type: ai2_arc
|
12 |
+
config: ARC-Challenge
|
13 |
+
split: test
|
14 |
+
args:
|
15 |
+
num_few_shot: 25
|
16 |
+
metrics:
|
17 |
+
- type: acc_norm
|
18 |
+
value: 66.64
|
19 |
+
name: normalized accuracy
|
20 |
+
source:
|
21 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=perlthoughts/neural-chat-v3-3-8x7b-MoE
|
22 |
+
name: Open LLM Leaderboard
|
23 |
+
- task:
|
24 |
+
type: text-generation
|
25 |
+
name: Text Generation
|
26 |
+
dataset:
|
27 |
+
name: HellaSwag (10-Shot)
|
28 |
+
type: hellaswag
|
29 |
+
split: validation
|
30 |
+
args:
|
31 |
+
num_few_shot: 10
|
32 |
+
metrics:
|
33 |
+
- type: acc_norm
|
34 |
+
value: 85.43
|
35 |
+
name: normalized accuracy
|
36 |
+
source:
|
37 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=perlthoughts/neural-chat-v3-3-8x7b-MoE
|
38 |
+
name: Open LLM Leaderboard
|
39 |
+
- task:
|
40 |
+
type: text-generation
|
41 |
+
name: Text Generation
|
42 |
+
dataset:
|
43 |
+
name: MMLU (5-Shot)
|
44 |
+
type: cais/mmlu
|
45 |
+
config: all
|
46 |
+
split: test
|
47 |
+
args:
|
48 |
+
num_few_shot: 5
|
49 |
+
metrics:
|
50 |
+
- type: acc
|
51 |
+
value: 62.22
|
52 |
+
name: accuracy
|
53 |
+
source:
|
54 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=perlthoughts/neural-chat-v3-3-8x7b-MoE
|
55 |
+
name: Open LLM Leaderboard
|
56 |
+
- task:
|
57 |
+
type: text-generation
|
58 |
+
name: Text Generation
|
59 |
+
dataset:
|
60 |
+
name: TruthfulQA (0-shot)
|
61 |
+
type: truthful_qa
|
62 |
+
config: multiple_choice
|
63 |
+
split: validation
|
64 |
+
args:
|
65 |
+
num_few_shot: 0
|
66 |
+
metrics:
|
67 |
+
- type: mc2
|
68 |
+
value: 63.2
|
69 |
+
source:
|
70 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=perlthoughts/neural-chat-v3-3-8x7b-MoE
|
71 |
+
name: Open LLM Leaderboard
|
72 |
+
- task:
|
73 |
+
type: text-generation
|
74 |
+
name: Text Generation
|
75 |
+
dataset:
|
76 |
+
name: Winogrande (5-shot)
|
77 |
+
type: winogrande
|
78 |
+
config: winogrande_xl
|
79 |
+
split: validation
|
80 |
+
args:
|
81 |
+
num_few_shot: 5
|
82 |
+
metrics:
|
83 |
+
- type: acc
|
84 |
+
value: 79.72
|
85 |
+
name: accuracy
|
86 |
+
source:
|
87 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=perlthoughts/neural-chat-v3-3-8x7b-MoE
|
88 |
+
name: Open LLM Leaderboard
|
89 |
+
- task:
|
90 |
+
type: text-generation
|
91 |
+
name: Text Generation
|
92 |
+
dataset:
|
93 |
+
name: GSM8k (5-shot)
|
94 |
+
type: gsm8k
|
95 |
+
config: main
|
96 |
+
split: test
|
97 |
+
args:
|
98 |
+
num_few_shot: 5
|
99 |
+
metrics:
|
100 |
+
- type: acc
|
101 |
+
value: 69.83
|
102 |
+
name: accuracy
|
103 |
+
source:
|
104 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=perlthoughts/neural-chat-v3-3-8x7b-MoE
|
105 |
+
name: Open LLM Leaderboard
|
106 |
---
|
107 |
|
108 |
## Intel's Neural Chat v3-3 8x7B Mixtral MOE
|
|
|
170 |
The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please cosult an attorney before using this model for commercial purposes.
|
171 |
|
172 |
|
173 |
+
|
174 |
+
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
175 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_perlthoughts__neural-chat-v3-3-8x7b-MoE)
|
176 |
+
|
177 |
+
| Metric |Value|
|
178 |
+
|---------------------------------|----:|
|
179 |
+
|Avg. |71.17|
|
180 |
+
|AI2 Reasoning Challenge (25-Shot)|66.64|
|
181 |
+
|HellaSwag (10-Shot) |85.43|
|
182 |
+
|MMLU (5-Shot) |62.22|
|
183 |
+
|TruthfulQA (0-shot) |63.20|
|
184 |
+
|Winogrande (5-shot) |79.72|
|
185 |
+
|GSM8k (5-shot) |69.83|
|
186 |
+
|