Update README.md
Browse files
README.md
CHANGED
|
@@ -10,7 +10,7 @@ base_model:
|
|
| 10 |
|
| 11 |
## KidLM Model
|
| 12 |
|
| 13 |
-
We continue pre-train the [RoBERTa (base)
|
| 14 |
|
| 15 |
## How to use
|
| 16 |
|
|
|
|
| 10 |
|
| 11 |
## KidLM Model
|
| 12 |
|
| 13 |
+
We continue pre-train the [RoBERTa (base)](https://huggingface.co/FacebookAI/roberta-base) model on our [KidLM corpus](https://huggingface.co/datasets/tafseer-nayeem/KidLM-corpus) using a masked language modeling (MLM) objective. This approach involves randomly masking 15% of the words in each input sequence, allowing the model to predict the masked words based on their surrounding context.
|
| 14 |
|
| 15 |
## How to use
|
| 16 |
|