leaderboard-pt-pr-bot commited on
Commit
d34c84d
1 Parent(s): 0aa722c

Adding the Open Portuguese LLM Leaderboard Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/eduagarcia-temp/portuguese-leaderboard-results-to-modelcard

The purpose of this PR is to add evaluation results from the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard) to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/eduagarcia-temp/portuguese-leaderboard-results-to-modelcard/discussions

Files changed (1) hide show
  1. README.md +169 -4
README.md CHANGED
@@ -1,16 +1,162 @@
1
  ---
2
- license: other
3
- license_name: glm-4
4
- license_link: https://huggingface.co/THUDM/glm-4-9b-chat-1m/blob/main/LICENSE
5
-
6
  language:
7
  - zh
8
  - en
 
9
  tags:
10
  - glm
11
  - chatglm
12
  - thudm
 
 
13
  inference: false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ---
15
 
16
  # GLM-4-9B-Chat-1M
@@ -139,3 +285,22 @@ GLM-4 模型的权重的使用则需要遵循 [LICENSE](LICENSE)。
139
  primaryClass={id='cs.CL' full_name='Computation and Language' is_active=True alt_name='cmp-lg' in_archive='cs' is_general=False description='Covers natural language processing. Roughly includes material in ACM Subject Class I.2.7. Note that work on artificial languages (programming languages, logics, formal systems) that does not explicitly address natural-language issues broadly construed (natural-language processing, computational linguistics, speech, text retrieval, etc.) is not appropriate for this area.'}
140
  }
141
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
 
2
  language:
3
  - zh
4
  - en
5
+ license: other
6
  tags:
7
  - glm
8
  - chatglm
9
  - thudm
10
+ license_name: glm-4
11
+ license_link: https://huggingface.co/THUDM/glm-4-9b-chat-1m/blob/main/LICENSE
12
  inference: false
13
+ model-index:
14
+ - name: glm-4-9b-chat-1m
15
+ results:
16
+ - task:
17
+ type: text-generation
18
+ name: Text Generation
19
+ dataset:
20
+ name: ENEM Challenge (No Images)
21
+ type: eduagarcia/enem_challenge
22
+ split: train
23
+ args:
24
+ num_few_shot: 3
25
+ metrics:
26
+ - type: acc
27
+ value: 73.62
28
+ name: accuracy
29
+ source:
30
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
31
+ name: Open Portuguese LLM Leaderboard
32
+ - task:
33
+ type: text-generation
34
+ name: Text Generation
35
+ dataset:
36
+ name: BLUEX (No Images)
37
+ type: eduagarcia-temp/BLUEX_without_images
38
+ split: train
39
+ args:
40
+ num_few_shot: 3
41
+ metrics:
42
+ - type: acc
43
+ value: 61.89
44
+ name: accuracy
45
+ source:
46
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
47
+ name: Open Portuguese LLM Leaderboard
48
+ - task:
49
+ type: text-generation
50
+ name: Text Generation
51
+ dataset:
52
+ name: OAB Exams
53
+ type: eduagarcia/oab_exams
54
+ split: train
55
+ args:
56
+ num_few_shot: 3
57
+ metrics:
58
+ - type: acc
59
+ value: 54.44
60
+ name: accuracy
61
+ source:
62
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
63
+ name: Open Portuguese LLM Leaderboard
64
+ - task:
65
+ type: text-generation
66
+ name: Text Generation
67
+ dataset:
68
+ name: Assin2 RTE
69
+ type: assin2
70
+ split: test
71
+ args:
72
+ num_few_shot: 15
73
+ metrics:
74
+ - type: f1_macro
75
+ value: 93.79
76
+ name: f1-macro
77
+ source:
78
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
79
+ name: Open Portuguese LLM Leaderboard
80
+ - task:
81
+ type: text-generation
82
+ name: Text Generation
83
+ dataset:
84
+ name: Assin2 STS
85
+ type: eduagarcia/portuguese_benchmark
86
+ split: test
87
+ args:
88
+ num_few_shot: 15
89
+ metrics:
90
+ - type: pearson
91
+ value: 81.07
92
+ name: pearson
93
+ source:
94
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
95
+ name: Open Portuguese LLM Leaderboard
96
+ - task:
97
+ type: text-generation
98
+ name: Text Generation
99
+ dataset:
100
+ name: FaQuAD NLI
101
+ type: ruanchaves/faquad-nli
102
+ split: test
103
+ args:
104
+ num_few_shot: 15
105
+ metrics:
106
+ - type: f1_macro
107
+ value: 76.27
108
+ name: f1-macro
109
+ source:
110
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
111
+ name: Open Portuguese LLM Leaderboard
112
+ - task:
113
+ type: text-generation
114
+ name: Text Generation
115
+ dataset:
116
+ name: HateBR Binary
117
+ type: ruanchaves/hatebr
118
+ split: test
119
+ args:
120
+ num_few_shot: 25
121
+ metrics:
122
+ - type: f1_macro
123
+ value: 82.28
124
+ name: f1-macro
125
+ source:
126
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
127
+ name: Open Portuguese LLM Leaderboard
128
+ - task:
129
+ type: text-generation
130
+ name: Text Generation
131
+ dataset:
132
+ name: PT Hate Speech Binary
133
+ type: hate_speech_portuguese
134
+ split: test
135
+ args:
136
+ num_few_shot: 25
137
+ metrics:
138
+ - type: f1_macro
139
+ value: 70.27
140
+ name: f1-macro
141
+ source:
142
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
143
+ name: Open Portuguese LLM Leaderboard
144
+ - task:
145
+ type: text-generation
146
+ name: Text Generation
147
+ dataset:
148
+ name: tweetSentBR
149
+ type: eduagarcia/tweetsentbr_fewshot
150
+ split: test
151
+ args:
152
+ num_few_shot: 25
153
+ metrics:
154
+ - type: f1_macro
155
+ value: 73.25
156
+ name: f1-macro
157
+ source:
158
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b-chat-1m
159
+ name: Open Portuguese LLM Leaderboard
160
  ---
161
 
162
  # GLM-4-9B-Chat-1M
 
285
  primaryClass={id='cs.CL' full_name='Computation and Language' is_active=True alt_name='cmp-lg' in_archive='cs' is_general=False description='Covers natural language processing. Roughly includes material in ACM Subject Class I.2.7. Note that work on artificial languages (programming languages, logics, formal systems) that does not explicitly address natural-language issues broadly construed (natural-language processing, computational linguistics, speech, text retrieval, etc.) is not appropriate for this area.'}
286
  }
287
  ```
288
+
289
+
290
+ # Open Portuguese LLM Leaderboard Evaluation Results
291
+
292
+ Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/THUDM/glm-4-9b-chat-1m) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
293
+
294
+ | Metric | Value |
295
+ |--------------------------|--------|
296
+ |Average |**74.1**|
297
+ |ENEM Challenge (No Images)| 73.62|
298
+ |BLUEX (No Images) | 61.89|
299
+ |OAB Exams | 54.44|
300
+ |Assin2 RTE | 93.79|
301
+ |Assin2 STS | 81.07|
302
+ |FaQuAD NLI | 76.27|
303
+ |HateBR Binary | 82.28|
304
+ |PT Hate Speech Binary | 70.27|
305
+ |tweetSentBR | 73.25|
306
+