File size: 1,377 Bytes
c00a325
 
 
 
 
 
 
 
 
 
38efa25
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
We released **Abel-7B-002**, resulting in a stronger (35% improvement on GSM8K, 126% improvement on MATH) and more generalized model, achieving the best performance among all 7B models (80.44 on GSM8K, 29.46 on MATH)

| Model | GSM8k  | MATH |MathQA | SVAMP |SCQ5K-EN | ARC-E|ARC-C|HellaSwag|MMLU |
|-----------|------------|----------|--------------|-----------|----------------|---------|----------|---------------|----------|
|Abel-7B-002 |	**80.44** | **29.46**	| **69.78**	|77.67	|**55.95**	|77.67	|**55.05**	|77.72	|61.19	|
|Abel-7B-001 |59.74	|13	|1.21	|57.67	|9.3	|53.32	|38.97	|63.51|40.59	|
|MetaMath-Mistral-7B|77.7	|28.2	|33.94	|**79.33**	|37.6|	**78.48**	|51.93	|76.44|	61.93|
|Qwen-7b|47.84	|9.34	|27.44	|53	|40.05	|74.97	|53.05	|**86.85**|57.98	|
|Mistral-7b|37.83	|9.06	|25.73	|63	|39.6	|76.83	|53.22|	76.31|**64.05**	|
|Yi-6b| 32.6	|5.78	|26.98	|55.67	|35.5	|73.66	|49.53	|68.97|64.02	|
|LLaMA2-7b|12.96	|2.78	|11.52	|44	|28.24	|71.12	|46.61	|71.32|46.7	|

Please cite the repo if the model/code/conclusion in this repo are helpful to you.
```
@misc{abel,
  author = {Chern, Ethan and Zou, Haoyang and Li, Xuefeng and Hu, Jiewen and Feng, Kehua and Li, Junlong and Liu, Pengfei},
  title = {Generative AI for Math: Abel},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/GAIR-NLP/abel}},
}
```