Commit
Β·
164b617
1
Parent(s):
2f23fe4
Update README.md
Browse files
README.md
CHANGED
@@ -14,6 +14,15 @@ language:
|
|
14 |
- fr
|
15 |
- bn
|
16 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
### Introduction
|
18 |
|
19 |
We introduce π MathOctopus, a series of open-source large language models (LLMs) specifically tailored for multilingual math problem-solving. The MathOctopus models are trained on π€ MGSM8KInstruct Dataset, encompassing ten distinct languages.
|
@@ -61,7 +70,9 @@ Our dataset and models are all available at Huggingface.
|
|
61 |
*-Cross refers to our model trained with cross-training strategy.
|
62 |
|
63 |
*-xRFT means we train the model with multilingual rejection sampling.
|
|
|
64 |
### **Overall Results on MGSM**
|
|
|
65 |
| 7B Model | En | Sw | Zh | Bn | De | Es | Fr | Ja | Ru | Th | Overall |
|
66 |
|:--------------------------------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|
|
67 |
| MathOctopus<sup>C</sup> | 52.0 | 23.6 | 31.6 | 18.8 | 38.0 | 39.2 | 36.4 | 27.2 | 33.6 | 21.6 | 32.2 |
|
@@ -85,7 +96,10 @@ Our dataset and models are all available at Huggingface.
|
|
85 |
| **xRFT**-MathOctopus<sup>C</sup>| 53.6 | 27.6 | 34.4 | 19.2 | 47.2 | 47.6 | 44.8 | 30.8 | 38.8 | 22.8 | 36.7 |
|
86 |
| MathOctopus<sup>P</sup> | 56.4 | 46.8 | 52.0 | 35.2 | 47.2 | 53.2 | 48.0 | 39.2 | 45.6 | 41.2 | 46.5 |
|
87 |
| **xRFT**-MathOctopus<sup>P</sup>| 51.6 | 47.2 | 52.4 | 37.6 | 51.2 | 52.8 | 44.4 | 41.6 | 50.0 | 47.6 | 47.6 |
|
|
|
|
|
88 |
### **Overall Results on MSVAMP**
|
|
|
89 |
| 7B Model | En | Sw | Zh | Bn | De | Es | Fr | Ja | Ru | Th | Overall |
|
90 |
|:--------------------------------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|
|
91 |
| MathOctopus<sup>C</sup> | 49.2 | 36.6 | 43.6 | 30.2 | 48.6 | 46.8 | 46.4 | 42.5 | 46.7 | 34.0 | 42.5 |
|
@@ -109,7 +123,10 @@ Our dataset and models are all available at Huggingface.
|
|
109 |
| **xRFT**-MathOctopus<sup>C</sup>| 48.1 | 42.8 | 43.6 | 23.3 | 48.7 | 50.0 | 48.9 | 43.4 | 44.6 | 35.5 | 42.9 |
|
110 |
| MathOctopus<sup>P</sup> | 56.4 | 46.8 | 52.0 | 35.2 | 47.2 | 53.2 | 48.0 | 39.2 | 45.6 | 41.2 | 46.5 |
|
111 |
| **xRFT**-MathOctopus<sup>P</sup>| 48.0 | 42.3 | 46.1 | 36.2 | 47.5 | 48.5 | 48.3 | 45.8 | 47.2 | 41.2 | 45.1 |
|
|
|
|
|
112 |
### **MathOctopus in English**
|
|
|
113 |
| Models | GSM8K | SVAMP |
|
114 |
|:--------------------------------|:--------|:--------|
|
115 |
| LLaMA 2-7B | 42.4 | 38.3 |
|
@@ -121,5 +138,20 @@ Our dataset and models are all available at Huggingface.
|
|
121 |
| LLaMA 1-33B | 50.0 | 49.0 |
|
122 |
| MathOctopus<sup>P</sup>-33B | 56.0 | 52.5 |
|
123 |
| MathOctopus<sup>C</sup>-33B | 53.7 | 51.5 |
|
|
|
124 |
## Intended Uses
|
125 |
These models are trained for research purposes. They are designed to solve multilingual math problems. They can be used in educational software, tutoring systems, or any application where a solution to a math problem is needed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
- fr
|
15 |
- bn
|
16 |
---
|
17 |
+
|
18 |
+
# π Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations
|
19 |
+
|
20 |
+
Project Page: [https://mathoctopus.github.io/](https://mathoctopus.github.io/)
|
21 |
+
|
22 |
+
Paper: [https://arxiv.org/abs/2310.20246.pdf](https://arxiv.org/abs/2310.20246.pdf)
|
23 |
+
|
24 |
+
Code: [https://github.com/microsoft/MathOctopus](https://github.com/microsoft/MathOctopus)
|
25 |
+
|
26 |
### Introduction
|
27 |
|
28 |
We introduce π MathOctopus, a series of open-source large language models (LLMs) specifically tailored for multilingual math problem-solving. The MathOctopus models are trained on π€ MGSM8KInstruct Dataset, encompassing ten distinct languages.
|
|
|
70 |
*-Cross refers to our model trained with cross-training strategy.
|
71 |
|
72 |
*-xRFT means we train the model with multilingual rejection sampling.
|
73 |
+
|
74 |
### **Overall Results on MGSM**
|
75 |
+
|
76 |
| 7B Model | En | Sw | Zh | Bn | De | Es | Fr | Ja | Ru | Th | Overall |
|
77 |
|:--------------------------------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|
|
78 |
| MathOctopus<sup>C</sup> | 52.0 | 23.6 | 31.6 | 18.8 | 38.0 | 39.2 | 36.4 | 27.2 | 33.6 | 21.6 | 32.2 |
|
|
|
96 |
| **xRFT**-MathOctopus<sup>C</sup>| 53.6 | 27.6 | 34.4 | 19.2 | 47.2 | 47.6 | 44.8 | 30.8 | 38.8 | 22.8 | 36.7 |
|
97 |
| MathOctopus<sup>P</sup> | 56.4 | 46.8 | 52.0 | 35.2 | 47.2 | 53.2 | 48.0 | 39.2 | 45.6 | 41.2 | 46.5 |
|
98 |
| **xRFT**-MathOctopus<sup>P</sup>| 51.6 | 47.2 | 52.4 | 37.6 | 51.2 | 52.8 | 44.4 | 41.6 | 50.0 | 47.6 | 47.6 |
|
99 |
+
|
100 |
+
|
101 |
### **Overall Results on MSVAMP**
|
102 |
+
|
103 |
| 7B Model | En | Sw | Zh | Bn | De | Es | Fr | Ja | Ru | Th | Overall |
|
104 |
|:--------------------------------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|
|
105 |
| MathOctopus<sup>C</sup> | 49.2 | 36.6 | 43.6 | 30.2 | 48.6 | 46.8 | 46.4 | 42.5 | 46.7 | 34.0 | 42.5 |
|
|
|
123 |
| **xRFT**-MathOctopus<sup>C</sup>| 48.1 | 42.8 | 43.6 | 23.3 | 48.7 | 50.0 | 48.9 | 43.4 | 44.6 | 35.5 | 42.9 |
|
124 |
| MathOctopus<sup>P</sup> | 56.4 | 46.8 | 52.0 | 35.2 | 47.2 | 53.2 | 48.0 | 39.2 | 45.6 | 41.2 | 46.5 |
|
125 |
| **xRFT**-MathOctopus<sup>P</sup>| 48.0 | 42.3 | 46.1 | 36.2 | 47.5 | 48.5 | 48.3 | 45.8 | 47.2 | 41.2 | 45.1 |
|
126 |
+
|
127 |
+
|
128 |
### **MathOctopus in English**
|
129 |
+
|
130 |
| Models | GSM8K | SVAMP |
|
131 |
|:--------------------------------|:--------|:--------|
|
132 |
| LLaMA 2-7B | 42.4 | 38.3 |
|
|
|
138 |
| LLaMA 1-33B | 50.0 | 49.0 |
|
139 |
| MathOctopus<sup>P</sup>-33B | 56.0 | 52.5 |
|
140 |
| MathOctopus<sup>C</sup>-33B | 53.7 | 51.5 |
|
141 |
+
|
142 |
## Intended Uses
|
143 |
These models are trained for research purposes. They are designed to solve multilingual math problems. They can be used in educational software, tutoring systems, or any application where a solution to a math problem is needed.
|
144 |
+
|
145 |
+
## Citation
|
146 |
+
Please cite our paper if you use our data, model or code. Please also kindly cite the original dataset papers.
|
147 |
+
|
148 |
+
```
|
149 |
+
@misc{chen2023breaking,
|
150 |
+
title={Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations},
|
151 |
+
author={Nuo Chen and Zinan Zheng and Ning Wu and Linjun Shou and Ming Gong and Yangqiu Song and Dongmei Zhang and Jia Li},
|
152 |
+
year={2023},
|
153 |
+
eprint={2310.20246},
|
154 |
+
archivePrefix={arXiv},
|
155 |
+
primaryClass={cs.CL}
|
156 |
+
}
|
157 |
+
```
|