weiqipedia commited on
Commit
019608c
1 Parent(s): 14b752c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -40,7 +40,9 @@ We evaluated SEA-LION-7B-Instruct-NC on the [BHASA benchmark](https://arxiv.org/
40
  compared it against [Llama-2-7B](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf), [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)
41
  and [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b-instruct).
42
 
43
- We only evaluated it on the Indonesian tasks as the model was only instruction-tuned in Indonesian.
 
 
44
 
45
  The scores shown in the tables below have been adjusted to only consider answers provided in the appropriate language.
46
 
 
40
  compared it against [Llama-2-7B](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf), [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)
41
  and [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b-instruct).
42
 
43
+ We only evaluated it on the Indonesian tasks as the model was only instruction-tuned in Indonesian.
44
+
45
+ The evaluation was done zero-shot with Indonesian prompts and only a sample of 100 - 1000 instances per dataset was used as per the setting described in the BHASA paper.
46
 
47
  The scores shown in the tables below have been adjusted to only consider answers provided in the appropriate language.
48