weiqipedia
commited on
Commit
•
019608c
1
Parent(s):
14b752c
Update README.md
Browse files
README.md
CHANGED
@@ -40,7 +40,9 @@ We evaluated SEA-LION-7B-Instruct-NC on the [BHASA benchmark](https://arxiv.org/
|
|
40 |
compared it against [Llama-2-7B](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf), [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)
|
41 |
and [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b-instruct).
|
42 |
|
43 |
-
We only evaluated it on the Indonesian tasks as the model was only instruction-tuned in Indonesian.
|
|
|
|
|
44 |
|
45 |
The scores shown in the tables below have been adjusted to only consider answers provided in the appropriate language.
|
46 |
|
|
|
40 |
compared it against [Llama-2-7B](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf), [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)
|
41 |
and [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b-instruct).
|
42 |
|
43 |
+
We only evaluated it on the Indonesian tasks as the model was only instruction-tuned in Indonesian.
|
44 |
+
|
45 |
+
The evaluation was done zero-shot with Indonesian prompts and only a sample of 100 - 1000 instances per dataset was used as per the setting described in the BHASA paper.
|
46 |
|
47 |
The scores shown in the tables below have been adjusted to only consider answers provided in the appropriate language.
|
48 |
|