munish0838 commited on
Commit
c6b8dc2
·
verified ·
1 Parent(s): 10af4c8

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +169 -0
README.md ADDED
@@ -0,0 +1,169 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+
4
+ library_name: transformers
5
+ tags:
6
+ - mergekit
7
+ - merge
8
+ base_model:
9
+ - meditsolutions/Llama-3.1-MedIT-SUN-8B
10
+ - allenai/Llama-3.1-Tulu-3-8B
11
+ - arcee-ai/Llama-3.1-SuperNova-Lite
12
+ model-index:
13
+ - name: Tulu-3.1-8B-SuperNova
14
+ results:
15
+ - task:
16
+ type: text-generation
17
+ name: Text Generation
18
+ dataset:
19
+ name: IFEval (0-Shot)
20
+ type: HuggingFaceH4/ifeval
21
+ args:
22
+ num_few_shot: 0
23
+ metrics:
24
+ - type: inst_level_strict_acc and prompt_level_strict_acc
25
+ value: 81.94
26
+ name: strict accuracy
27
+ source:
28
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bunnycore/Tulu-3.1-8B-SuperNova
29
+ name: Open LLM Leaderboard
30
+ - task:
31
+ type: text-generation
32
+ name: Text Generation
33
+ dataset:
34
+ name: BBH (3-Shot)
35
+ type: BBH
36
+ args:
37
+ num_few_shot: 3
38
+ metrics:
39
+ - type: acc_norm
40
+ value: 32.5
41
+ name: normalized accuracy
42
+ source:
43
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bunnycore/Tulu-3.1-8B-SuperNova
44
+ name: Open LLM Leaderboard
45
+ - task:
46
+ type: text-generation
47
+ name: Text Generation
48
+ dataset:
49
+ name: MATH Lvl 5 (4-Shot)
50
+ type: hendrycks/competition_math
51
+ args:
52
+ num_few_shot: 4
53
+ metrics:
54
+ - type: exact_match
55
+ value: 24.32
56
+ name: exact match
57
+ source:
58
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bunnycore/Tulu-3.1-8B-SuperNova
59
+ name: Open LLM Leaderboard
60
+ - task:
61
+ type: text-generation
62
+ name: Text Generation
63
+ dataset:
64
+ name: GPQA (0-shot)
65
+ type: Idavidrein/gpqa
66
+ args:
67
+ num_few_shot: 0
68
+ metrics:
69
+ - type: acc_norm
70
+ value: 6.94
71
+ name: acc_norm
72
+ source:
73
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bunnycore/Tulu-3.1-8B-SuperNova
74
+ name: Open LLM Leaderboard
75
+ - task:
76
+ type: text-generation
77
+ name: Text Generation
78
+ dataset:
79
+ name: MuSR (0-shot)
80
+ type: TAUR-Lab/MuSR
81
+ args:
82
+ num_few_shot: 0
83
+ metrics:
84
+ - type: acc_norm
85
+ value: 8.69
86
+ name: acc_norm
87
+ source:
88
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bunnycore/Tulu-3.1-8B-SuperNova
89
+ name: Open LLM Leaderboard
90
+ - task:
91
+ type: text-generation
92
+ name: Text Generation
93
+ dataset:
94
+ name: MMLU-PRO (5-shot)
95
+ type: TIGER-Lab/MMLU-Pro
96
+ config: main
97
+ split: test
98
+ args:
99
+ num_few_shot: 5
100
+ metrics:
101
+ - type: acc
102
+ value: 31.27
103
+ name: accuracy
104
+ source:
105
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bunnycore/Tulu-3.1-8B-SuperNova
106
+ name: Open LLM Leaderboard
107
+
108
+ ---
109
+
110
+ [![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory)
111
+
112
+
113
+ # QuantFactory/Tulu-3.1-8B-SuperNova-GGUF
114
+ This is quantized version of [bunnycore/Tulu-3.1-8B-SuperNova](https://huggingface.co/bunnycore/Tulu-3.1-8B-SuperNova) created using llama.cpp
115
+
116
+ # Original Model Card
117
+
118
+ # merge
119
+
120
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
121
+
122
+ ## Merge Details
123
+ ### Merge Method
124
+
125
+ This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.
126
+
127
+ ### Models Merged
128
+
129
+ The following models were included in the merge:
130
+ * [meditsolutions/Llama-3.1-MedIT-SUN-8B](https://huggingface.co/meditsolutions/Llama-3.1-MedIT-SUN-8B)
131
+ * [allenai/Llama-3.1-Tulu-3-8B](https://huggingface.co/allenai/Llama-3.1-Tulu-3-8B)
132
+ * [arcee-ai/Llama-3.1-SuperNova-Lite](https://huggingface.co/arcee-ai/Llama-3.1-SuperNova-Lite)
133
+
134
+ ### Configuration
135
+
136
+ The following YAML configuration was used to produce this model:
137
+
138
+ ```yaml
139
+ models:
140
+ - model: arcee-ai/Llama-3.1-SuperNova-Lite
141
+ parameters:
142
+ weight: 1.0
143
+ - model: allenai/Llama-3.1-Tulu-3-8B
144
+ parameters:
145
+ weight: 1.0
146
+ - model: meditsolutions/Llama-3.1-MedIT-SUN-8B
147
+ parameters:
148
+ weight: 1.0
149
+ merge_method: linear
150
+ normalize: false
151
+ int8_mask: true
152
+ dtype: bfloat16
153
+
154
+ ```
155
+
156
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
157
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_bunnycore__Tulu-3.1-8B-SuperNova)
158
+
159
+ | Metric |Value|
160
+ |-------------------|----:|
161
+ |Avg. |30.94|
162
+ |IFEval (0-Shot) |81.94|
163
+ |BBH (3-Shot) |32.50|
164
+ |MATH Lvl 5 (4-Shot)|24.32|
165
+ |GPQA (0-shot) | 6.94|
166
+ |MuSR (0-shot) | 8.69|
167
+ |MMLU-PRO (5-shot) |31.27|
168
+
169
+