Dr. Jorge Abreu Vicente
commited on
Commit
·
1001acb
1
Parent(s):
26d020f
Update README.md
Browse files
README.md
CHANGED
|
@@ -26,7 +26,7 @@ In this implementation we have followed the commands of the [`nvidia/megatron-be
|
|
| 26 |
|
| 27 |
However, the file [`convert_megatron_bert_checkpoint.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py) needed a modification. The reason is that the Megatron model shown in [`nvidia/megatron-bert-uncased-345m`](https://huggingface.co/nvidia/megatron-bert-uncased-345m) has included head layers, while the weights of the BioMegatron model that we upload to this repository do not contain a head.
|
| 28 |
|
| 29 |
-
We provide in the repository an alternative version of the python script in order to any user to cross-check the validity of the model replicated in this repository.
|
| 30 |
|
| 31 |
|
| 32 |
The code below is a modification of the original [`convert_megatron_bert_checkpoint.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py).
|
|
|
|
| 26 |
|
| 27 |
However, the file [`convert_megatron_bert_checkpoint.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py) needed a modification. The reason is that the Megatron model shown in [`nvidia/megatron-bert-uncased-345m`](https://huggingface.co/nvidia/megatron-bert-uncased-345m) has included head layers, while the weights of the BioMegatron model that we upload to this repository do not contain a head.
|
| 28 |
|
| 29 |
+
We provide in the repository an alternative version of the [python script](https://huggingface.co/EMBO/BioMegatron345mUncased/blob/main/convert_biomegatron_checkpoint.py) in order to any user to cross-check the validity of the model replicated in this repository.
|
| 30 |
|
| 31 |
|
| 32 |
The code below is a modification of the original [`convert_megatron_bert_checkpoint.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py).
|