damerajee commited on
Commit
45c5c97
β€’
1 Parent(s): c6828c6

Update README.md

Browse files

![Screenshot (30).png](https://cdn-uploads.huggingface.co/production/uploads/6487239cca30096ea9f52115/ip_xYv7XpC68RKq7P-Qt-.png)

Files changed (1) hide show
  1. README.md +9 -4
README.md CHANGED
@@ -20,6 +20,8 @@ Gaja is a Hindi/Hinglish chat model, initially trained on SarvamAI's OpenHathi m
20
 
21
  # Additional Information
22
  - The source code for this model and more information can be found here - [github](https://github.com/dame-cell/Gaja)
 
 
23
 
24
 
25
  # πŸ’¬ Prompt template
@@ -58,11 +60,14 @@ Gaja is a Hindi/Hinglish chat model, initially trained on SarvamAI's OpenHathi m
58
 
59
  ### English eval
60
 
61
- | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K|
62
- |------------------------|-----------|----------|-----------|------|--------|------------|
63
- | 47.69 | 52.82 | 0.71 | 76.31 | 40.83| 44.64 | 70.64 | 0.91|
 
 
 
 
64
 
65
- ![Screenshot (26).png](https://cdn-uploads.huggingface.co/production/uploads/6487239cca30096ea9f52115/aj_zS04BlF8t1REss1lne.png)
66
 
67
  # πŸš€ Infernce(colab or kaggle notebooks)
68
 
 
20
 
21
  # Additional Information
22
  - The source code for this model and more information can be found here - [github](https://github.com/dame-cell/Gaja)
23
+ - It outperforms Airavata, AI4Bharat's chat version, on Huggingface OpenLLM benchmark suite.
24
+ - It was fine-tuned on only 1k samples
25
 
26
 
27
  # πŸ’¬ Prompt template
 
60
 
61
  ### English eval
62
 
63
+ Model name| Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K|
64
+ |-------|------------------------|-----------|----------|-----------|------|--------|------------|
65
+ | [damerajee/Gaja-v1.00](https://huggingface.co/damerajee/Gaja-v1.00)| 47.69 | 52.82 | 76.31 | 40.83 | 44.64 | 70.64 | 0.91 |
66
+ | [manishiitg/open-aditi-hi-v2](https://huggingface.co/manishiitg/open-aditi-hi-v2) | 59.31 | 59.39 | 82.01 | 61.41 | 45.84 | 77.19 | 30.02 |
67
+ | [ai4bharat/Airavata](https://huggingface.co/ai4bharat/Airavata) | 45.52 | 46.5 | 69.26 | 43.9 | 40.62 | 68.82 | 4.02 |
68
+
69
+ ![Screenshot (30).png](https://cdn-uploads.huggingface.co/production/uploads/6487239cca30096ea9f52115/ip_xYv7XpC68RKq7P-Qt-.png)
70
 
 
71
 
72
  # πŸš€ Infernce(colab or kaggle notebooks)
73