Update README.md
Browse files![Screenshot (30).png](https://cdn-uploads.huggingface.co/production/uploads/6487239cca30096ea9f52115/ip_xYv7XpC68RKq7P-Qt-.png)
README.md
CHANGED
@@ -20,6 +20,8 @@ Gaja is a Hindi/Hinglish chat model, initially trained on SarvamAI's OpenHathi m
|
|
20 |
|
21 |
# Additional Information
|
22 |
- The source code for this model and more information can be found here - [github](https://github.com/dame-cell/Gaja)
|
|
|
|
|
23 |
|
24 |
|
25 |
# π¬ Prompt template
|
@@ -58,11 +60,14 @@ Gaja is a Hindi/Hinglish chat model, initially trained on SarvamAI's OpenHathi m
|
|
58 |
|
59 |
### English eval
|
60 |
|
61 |
-
| Average
|
62 |
-
|
63 |
-
| 47.69 | 52.82 |
|
|
|
|
|
|
|
|
|
64 |
|
65 |
-
![Screenshot (26).png](https://cdn-uploads.huggingface.co/production/uploads/6487239cca30096ea9f52115/aj_zS04BlF8t1REss1lne.png)
|
66 |
|
67 |
# π Infernce(colab or kaggle notebooks)
|
68 |
|
|
|
20 |
|
21 |
# Additional Information
|
22 |
- The source code for this model and more information can be found here - [github](https://github.com/dame-cell/Gaja)
|
23 |
+
- It outperforms Airavata, AI4Bharat's chat version, on Huggingface OpenLLM benchmark suite.
|
24 |
+
- It was fine-tuned on only 1k samples
|
25 |
|
26 |
|
27 |
# π¬ Prompt template
|
|
|
60 |
|
61 |
### English eval
|
62 |
|
63 |
+
Model name| Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K|
|
64 |
+
|-------|------------------------|-----------|----------|-----------|------|--------|------------|
|
65 |
+
| [damerajee/Gaja-v1.00](https://huggingface.co/damerajee/Gaja-v1.00)| 47.69 | 52.82 | 76.31 | 40.83 | 44.64 | 70.64 | 0.91 |
|
66 |
+
| [manishiitg/open-aditi-hi-v2](https://huggingface.co/manishiitg/open-aditi-hi-v2) | 59.31 | 59.39 | 82.01 | 61.41 | 45.84 | 77.19 | 30.02 |
|
67 |
+
| [ai4bharat/Airavata](https://huggingface.co/ai4bharat/Airavata) | 45.52 | 46.5 | 69.26 | 43.9 | 40.62 | 68.82 | 4.02 |
|
68 |
+
|
69 |
+
![Screenshot (30).png](https://cdn-uploads.huggingface.co/production/uploads/6487239cca30096ea9f52115/ip_xYv7XpC68RKq7P-Qt-.png)
|
70 |
|
|
|
71 |
|
72 |
# π Infernce(colab or kaggle notebooks)
|
73 |
|