Tristan commited on
Commit
0805050
·
1 Parent(s): 62003c9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -8,7 +8,7 @@ language: en
8
 
9
  This is a more up-to-date version of the [original BERT](https://huggingface.co/bert-base-cased) and [original RoBERTa](https://huggingface.co/roberta-base).
10
  In addition to being more up-to-date, it also tends to perform better than the original BERT on standard benchmarks.
11
- We think it is more fair to directly compare our model to the original BERT because our model was trained with about the same level of compute as the original BERT, and the architecture of BERT and RoBERTa are basically the same.
12
  The original RoBERTa takes an order of magnitude more compute, although our model is also not that different in performance from RoBERTa on standard benchmarks.
13
  Our model was trained on a cleaned October 2022 snapshot of Common Crawl and Wikipedia.
14
 
 
8
 
9
  This is a more up-to-date version of the [original BERT](https://huggingface.co/bert-base-cased) and [original RoBERTa](https://huggingface.co/roberta-base).
10
  In addition to being more up-to-date, it also tends to perform better than the original BERT on standard benchmarks.
11
+ We think it is fair to directly compare our model to the original BERT because our model was trained with about the same level of compute as the original BERT, and the architecture of BERT and RoBERTa are basically the same.
12
  The original RoBERTa takes an order of magnitude more compute, although our model is also not that different in performance from RoBERTa on standard benchmarks.
13
  Our model was trained on a cleaned October 2022 snapshot of Common Crawl and Wikipedia.
14