Adding the Open Portuguese LLM Leaderboard Evaluation Results
#1
by
leaderboard-pt-pr-bot
- opened
README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
-
base_model: mistralai/Mistral-7B-Instruct-v0.1
|
| 4 |
tags:
|
| 5 |
- generated_from_trainer
|
|
|
|
| 6 |
model-index:
|
| 7 |
- name: WeniGPT-Mistral-7B-instructBase
|
| 8 |
results: []
|
|
@@ -54,3 +54,22 @@ The following hyperparameters were used during training:
|
|
| 54 |
- Pytorch 2.0.0
|
| 55 |
- Datasets 2.13.0
|
| 56 |
- Tokenizers 0.14.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
| 3 |
tags:
|
| 4 |
- generated_from_trainer
|
| 5 |
+
base_model: mistralai/Mistral-7B-Instruct-v0.1
|
| 6 |
model-index:
|
| 7 |
- name: WeniGPT-Mistral-7B-instructBase
|
| 8 |
results: []
|
|
|
|
| 54 |
- Pytorch 2.0.0
|
| 55 |
- Datasets 2.13.0
|
| 56 |
- Tokenizers 0.14.1
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
# Open Portuguese LLM Leaderboard Evaluation Results
|
| 60 |
+
|
| 61 |
+
Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/Weni/WeniGPT-Mistral-7B-instructBase) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
|
| 62 |
+
|
| 63 |
+
| Metric | Value |
|
| 64 |
+
|--------------------------|---------|
|
| 65 |
+
|Average |**39.55**|
|
| 66 |
+
|ENEM Challenge (No Images)| 32.61|
|
| 67 |
+
|BLUEX (No Images) | 23.50|
|
| 68 |
+
|OAB Exams | 28.84|
|
| 69 |
+
|Assin2 RTE | 61.42|
|
| 70 |
+
|Assin2 STS | 40.02|
|
| 71 |
+
|FaQuAD NLI | 34.50|
|
| 72 |
+
|HateBR Binary | 44.16|
|
| 73 |
+
|PT Hate Speech Binary | 39.71|
|
| 74 |
+
|tweetSentBR | 51.18|
|
| 75 |
+
|