appvoid commited on
Commit
db97be3
·
verified ·
1 Parent(s): be1601b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -6
README.md CHANGED
@@ -19,9 +19,9 @@ license: apache-2.0
19
  ![palmer-004](https://huggingface.co/appvoid/palmer-004/resolve/main/palmer-004.jpeg)
20
 
21
 
22
- #### Introducing palmer-004
23
 
24
- We are thrilled to unveil palmer-004, the fourth iteration in our esteemed palmer series, designed to push the boundaries of performance and context window capacity. Merging the best of palmer-003 with the expansive capabilities of palmer-002-32k, palmer-004 is a testament to the power of merging similar trained models, think of it as palmer-003 with 32k context with minimal performance degradation.
25
 
26
  | Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande |
27
  |----------------|-------|-------|-----------|--------|------------|
@@ -30,9 +30,5 @@ We are thrilled to unveil palmer-004, the fourth iteration in our esteemed palme
30
  | palmer-004 | 0.2601| 0.3456| 0.6138 | 0.7443 | 0.6511 |
31
  | palmer-004-2406| 0.2661| 0.3490| 0.6173 | 0.7481 | 0.6417 |
32
 
33
- Crafted using the TIES merge method, palmer-004 elevates its predecessor's prowess to a remarkable 32k context window without needing to train on additional data. This model doesn't just aim to match; it surpasses, outperforming the current state-of-the-art coven_tiny_1.1b_32k_orpo_alpha on several benchmarks. As of June 1, 2024, palmer-004 stands as the premier 32k 1B model to fine-tune from on Hugging Face, setting a new standard for excellence in the open-source AI community.
34
-
35
-
36
-
37
 
38
  </div>
 
19
  ![palmer-004](https://huggingface.co/appvoid/palmer-004/resolve/main/palmer-004.jpeg)
20
 
21
 
22
+ #### palmer-004 update
23
 
24
+ This model has improved overall performance at the expense of small degradation on winogrande.
25
 
26
  | Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande |
27
  |----------------|-------|-------|-----------|--------|------------|
 
30
  | palmer-004 | 0.2601| 0.3456| 0.6138 | 0.7443 | 0.6511 |
31
  | palmer-004-2406| 0.2661| 0.3490| 0.6173 | 0.7481 | 0.6417 |
32
 
 
 
 
 
33
 
34
  </div>