venkatasg commited on
Commit
ff8c391
1 Parent(s): 9c99b7e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -14,9 +14,9 @@ Lil-Bevo is UT Austin's submission to the BabyLM challenge, specifically the *st
14
 
15
  ## TLDR:
16
  - Unigram tokenizer trained on 10M BabyLM tokens plus MAESTRO dataset for a vocab size of 16k.
17
- - `deberta-small-v3` trained on mixture of MAESTRO and 10M tokens for 3 epochs.
18
  - Model continues training for 50 epochs on 10M tokens with 128 sequence length.
19
- - Model continues training for 200 epochs on 10M tokens with 512 sequence length.
20
  - Model is trained with targeted linguistic masking for 10 epochs.
21
 
22
 
 
14
 
15
  ## TLDR:
16
  - Unigram tokenizer trained on 10M BabyLM tokens plus MAESTRO dataset for a vocab size of 16k.
17
+ - `deberta-small-v3` trained on mixture of MAESTRO and 10M tokens for 5 epochs.
18
  - Model continues training for 50 epochs on 10M tokens with 128 sequence length.
19
+ - Model continues training for 2 epochs on 10M tokens with 512 sequence length.
20
  - Model is trained with targeted linguistic masking for 10 epochs.
21
 
22