|
--- |
|
language: |
|
- uk |
|
tags: |
|
- ukrainian |
|
widget: |
|
- text: "Тарас Шевченко – великий український <mask>." |
|
license: mit |
|
--- |
|
This is a smaller version of the [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base) model with only Ukrainian and some English embeddings left. |
|
|
|
* The original model has 470M parameters, with 384M of them being input and output embeddings. |
|
* After shrinking the `sentencepiece` vocabulary from 250K to 31K (top 25K Ukrainian tokens and top English tokens) the number of model parameters reduced to 134M parameters, and model size reduced from 1GB to 400MB. |
|
|
