File size: 490 Bytes
31019d3 17bbb49 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
---
license: mit
datasets:
- omarmomen/babylm_10M
language:
- en
metrics:
- perplexity
library_name: transformers
---
# Model Card for omarmomen/ptb_bpe_tokenizer_10k
This model is part of the experiments in my master's thesis titled "Linguistic Structure Induction from Language Models" (https://arxiv.org/abs/2403.09714).
"omarmomen/ptb_bpe_tokenizer_10k" is a RobertaTokenizer pretrained on the Penn Tree Bank Training dataset (cased) with 10K tokens.
https://arxiv.org/abs/2403.09714 |