Update README.md
Browse files
README.md
CHANGED
@@ -16,32 +16,6 @@ You can use the raw model for either masked language modeling or next sentence p
|
|
16 |
Note that this model is primarily aimed at being fine-tuned on math-related tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as math text generation you should look at model like GPT2.
|
17 |
|
18 |
#### How to use
|
19 |
-
<!---You can use this model directly with a pipeline for masked language modeling:
|
20 |
-
|
21 |
-
>>> from transformers import pipeline
|
22 |
-
>>> unmasker = pipeline('fill-mask', model='bert-base-uncased')
|
23 |
-
>>> unmasker("Hello I'm a [MASK] model.")
|
24 |
-
|
25 |
-
[{'sequence': "[CLS] hello i'm a fashion model. [SEP]",
|
26 |
-
'score': 0.1073106899857521,
|
27 |
-
'token': 4827,
|
28 |
-
'token_str': 'fashion'},
|
29 |
-
{'sequence': "[CLS] hello i'm a role model. [SEP]",
|
30 |
-
'score': 0.08774490654468536,
|
31 |
-
'token': 2535,
|
32 |
-
'token_str': 'role'},
|
33 |
-
{'sequence': "[CLS] hello i'm a new model. [SEP]",
|
34 |
-
'score': 0.05338378623127937,
|
35 |
-
'token': 2047,
|
36 |
-
'token_str': 'new'},
|
37 |
-
{'sequence': "[CLS] hello i'm a super model. [SEP]",
|
38 |
-
'score': 0.04667217284440994,
|
39 |
-
'token': 3565,
|
40 |
-
'token_str': 'super'},
|
41 |
-
{'sequence': "[CLS] hello i'm a fine model. [SEP]",
|
42 |
-
'score': 0.027095865458250046,
|
43 |
-
'token': 2986,
|
44 |
-
'token_str': 'fine'}]--->
|
45 |
|
46 |
Here is how to use this model to get the features of a given text in PyTorch:
|
47 |
|
@@ -191,6 +165,12 @@ From above, one can tell that MathBERT is specifically designed for mathematics
|
|
191 |
'token': 3182,
|
192 |
'token_str': 'places'}]
|
193 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
194 |
#### Training data
|
195 |
The MathBERT model was pretrained on pre-k to HS math curriculum (engageNY, Utah Math, Illustrative Math), college math books from openculture.com as well as graduate level math from arxiv math paper abstracts. There is about 100M tokens got pretrained on.
|
196 |
|
|
|
16 |
Note that this model is primarily aimed at being fine-tuned on math-related tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as math text generation you should look at model like GPT2.
|
17 |
|
18 |
#### How to use
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
|
20 |
Here is how to use this model to get the features of a given text in PyTorch:
|
21 |
|
|
|
165 |
'token': 3182,
|
166 |
'token_str': 'places'}]
|
167 |
```
|
168 |
+
Therefore, to try the 'fill-mask' hosted API on the right corner of the page, please use the sentences similar to below:
|
169 |
+
|
170 |
+
```
|
171 |
+
1 tenth times any [MASK] on the place value chart moves it one place value to the right. #from https://www.engageny.org/resource/grade-5-mathematics-module-1
|
172 |
+
```
|
173 |
+
|
174 |
#### Training data
|
175 |
The MathBERT model was pretrained on pre-k to HS math curriculum (engageNY, Utah Math, Illustrative Math), college math books from openculture.com as well as graduate level math from arxiv math paper abstracts. There is about 100M tokens got pretrained on.
|
176 |
|