mnaylor commited on
Commit
09f4545
·
1 Parent(s): 9787fc0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -9,4 +9,5 @@ I used the Mega source code (namely the `MegaEncoderLayer` class) and created wr
9
  was pretrained for 5 epochs (11.3k gradient steps) on wikitext-103, which took roughly 5 hours on a single T4 (in Colab's free tier).
10
 
11
  See [the Colab notebook](https://colab.research.google.com/drive/1qfUO6o5HRdxBblWlw058HVyvaEPhPpH8?usp=sharing)
12
- for further training details and example code for reuse.
 
 
9
  was pretrained for 5 epochs (11.3k gradient steps) on wikitext-103, which took roughly 5 hours on a single T4 (in Colab's free tier).
10
 
11
  See [the Colab notebook](https://colab.research.google.com/drive/1qfUO6o5HRdxBblWlw058HVyvaEPhPpH8?usp=sharing)
12
+ for further training details. In order to load the pretrained weights for this model, you'll need to use the
13
+ [Mega repo](https://github.com/facebookresearch/mega) along with the example code at the end of the Colab notebook.