Update README.md
Browse files
README.md
CHANGED
|
@@ -21,14 +21,14 @@ license: apache-2.0
|
|
| 21 |
|
| 22 |
#### palmer-004 update
|
| 23 |
|
| 24 |
-
This model has improved overall performance at the expense of small degradation on winogrande.
|
| 25 |
-
|
| 26 |
-
| Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande |
|
| 27 |
-
|----------------|-------|-------|-----------|--------|------------|
|
| 28 |
-
| tinyllama-3t | 0.2577| 0.3029| 0.5935 | 0.7329 | 0.5959 |
|
| 29 |
-
| palmer-003 | 0.2523| 0.3439| 0.6208 | 0.7524 | 0.6590 |
|
| 30 |
-
| palmer-004 | 0.2601| 0.3456| 0.6138 | 0.7443 | 0.6511 |
|
| 31 |
-
| palmer-004-2406| 0.2661| 0.3490| 0.6173 | 0.7481 | 0.6417 |
|
| 32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
</div>
|
|
|
|
| 21 |
|
| 22 |
#### palmer-004 update
|
| 23 |
|
| 24 |
+
This model has improved overall performance at the expense of small degradation on winogrande. As all palmer models, the model is biased to respond to answers without using any specific prompt, feel free to further fine-tune it for your specific use case.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
+
| Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande | Average |
|
| 27 |
+
|--------------------------------|-------|-------|-----------|--------|------------|---------|
|
| 28 |
+
| tinyllama-3t | 0.2577| 0.3029| 0.5935 | 0.7329 | 0.5959 | 0.4966 |
|
| 29 |
+
| palmer-003 | 0.2523| 0.3439| 0.6208 | 0.7524 | 0.6590 | 0.5257 |
|
| 30 |
+
| palmer-004 | 0.2601| 0.3456| 0.6138 | 0.7443 | 0.6511 | 0.5229 |
|
| 31 |
+
| palmer-004-2406 | 0.2661| 0.3490| 0.6173 | 0.7481 | 0.6417 | 0.5244 |
|
| 32 |
+
| coven_tiny_1.1b_32k_orpo_alpha | 0.4032| 0.3507| 0.6057 | 0.7231 | 0.6030 | 0.5371 |
|
| 33 |
|
| 34 |
</div>
|