lucifertrj Chaitanya890 commited on
Commit
3d00a29
1 Parent(s): 5272bef

Update README.md (#7)

Browse files

- Update README.md (7836faa52bf013d1e50247872c3ccca71a805cc2)


Co-authored-by: Chaitanya Singhal <[email protected]>

Files changed (1) hide show
  1. README.md +18 -1
README.md CHANGED
@@ -138,9 +138,25 @@ In order to leverage instruction fine-tuning, your prompt should be surrounded b
138
 
139
  ```
140
 
141
- ## Benchmarks
142
 
 
143
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
144
  | Model | # Params | Average | ARC (25-shot) | HellaSwag (10-shot) | Winogrande (5-shot) | TruthfulOA (0-shot) | MMLU (5-shot) |
145
  |-----------------------------------|----------|---------|---------------|---------------------|---------------------|---------------------|---------------|
146
  | aiplanet/buddhi-128k-chat-7b | 7B | 64.42 | 60.84 | 84 | 77.27 | 65.72 | 60.42 |
@@ -151,6 +167,7 @@ In order to leverage instruction fine-tuning, your prompt should be surrounded b
151
  | CallComply/openchat-3.5-0106-128k | 7B | 59.38 | 64.25 | 77.31 | 77.66 | 46.5 | 57.58 |
152
  | CallComply/zephyr-7b-beta-128k | 7B | 54.45 | 58.28 | 81 | 74.74 | 46.1 | 53.57 |
153
 
 
154
 
155
  ## Get in Touch
156
 
 
138
 
139
  ```
140
 
141
+ # Benchmarks
142
 
143
+ ### Long Context Benchmark
144
 
145
+ <strong>LongICLBench Banking77</strong>
146
+ <div>
147
+
148
+ | Model | 1R/2k | 2R/4K | 3R/7K | 4R/9K | 5R/14K |
149
+ |-----------------------------------------|-------|-------|-------|-------|--------|
150
+ | aiplanet/buddhi-128k-chat-7b | 47.8 | 60.8 | 57.8 | 62.4 | 57.2 |
151
+ | NousResearch/Yarn-Mistral-7b-128k | 31.6 | 68.6 | 68 | 47 | 65.6 |
152
+ | CallComply/zephyr-7b-beta-128k | 40.2 | 41.2 | 33.6 | 03 | 0 |
153
+ | Eric111/Yarn-Mistral-7b-128k-DPO | 28.6 | 62.8 | 58 | 41.6 | 59.8 |
154
+
155
+ </div>
156
+
157
+ <strong>Short Context Benchmark</strong>
158
+ <div>
159
+
160
  | Model | # Params | Average | ARC (25-shot) | HellaSwag (10-shot) | Winogrande (5-shot) | TruthfulOA (0-shot) | MMLU (5-shot) |
161
  |-----------------------------------|----------|---------|---------------|---------------------|---------------------|---------------------|---------------|
162
  | aiplanet/buddhi-128k-chat-7b | 7B | 64.42 | 60.84 | 84 | 77.27 | 65.72 | 60.42 |
 
167
  | CallComply/openchat-3.5-0106-128k | 7B | 59.38 | 64.25 | 77.31 | 77.66 | 46.5 | 57.58 |
168
  | CallComply/zephyr-7b-beta-128k | 7B | 54.45 | 58.28 | 81 | 74.74 | 46.1 | 53.57 |
169
 
170
+ </div>
171
 
172
  ## Get in Touch
173