Update README.md
Browse files
README.md
CHANGED
@@ -9,4 +9,5 @@ I used the Mega source code (namely the `MegaEncoderLayer` class) and created wr
|
|
9 |
was pretrained for 5 epochs (11.3k gradient steps) on wikitext-103, which took roughly 5 hours on a single T4 (in Colab's free tier).
|
10 |
|
11 |
See [the Colab notebook](https://colab.research.google.com/drive/1qfUO6o5HRdxBblWlw058HVyvaEPhPpH8?usp=sharing)
|
12 |
-
for further training details
|
|
|
|
9 |
was pretrained for 5 epochs (11.3k gradient steps) on wikitext-103, which took roughly 5 hours on a single T4 (in Colab's free tier).
|
10 |
|
11 |
See [the Colab notebook](https://colab.research.google.com/drive/1qfUO6o5HRdxBblWlw058HVyvaEPhPpH8?usp=sharing)
|
12 |
+
for further training details. In order to load the pretrained weights for this model, you'll need to use the
|
13 |
+
[Mega repo](https://github.com/facebookresearch/mega) along with the example code at the end of the Colab notebook.
|