Commit
·
f87bf66
1
Parent(s):
4727dc7
Update README.md
Browse files
README.md
CHANGED
|
@@ -53,6 +53,8 @@ NeMo Megatron-mT5 3B is a *multilingual* transformer-based masked language model
|
|
| 53 |
|
| 54 |
This model was trained with [NeMo Megatron](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/nemo_megatron/intro.html).
|
| 55 |
|
|
|
|
|
|
|
| 56 |
## List of Languages
|
| 57 |
|
| 58 |
We pre-trained our mT5 model on the following languages from the [mC4](https://github.com/allenai/allennlp/discussions/5265) dataset.
|
|
|
|
| 53 |
|
| 54 |
This model was trained with [NeMo Megatron](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/nemo_megatron/intro.html).
|
| 55 |
|
| 56 |
+
**NOTE**: Weights are distributed in bfloat16.
|
| 57 |
+
|
| 58 |
## List of Languages
|
| 59 |
|
| 60 |
We pre-trained our mT5 model on the following languages from the [mC4](https://github.com/allenai/allennlp/discussions/5265) dataset.
|