metadata
license: mit
tags:
- generated_from_trainer
model-index:
- name: finetuned-xlm-r-masakhaner-swa-whole-word-phonetic
results: []
finetuned-xlm-r-masakhaner-swa-whole-word-phonetic
This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 15.5961
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 7e-08
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 1.0 | 62 | 39.6873 |
| No log | 2.0 | 124 | 39.5168 |
| No log | 3.0 | 186 | 38.9323 |
| No log | 4.0 | 248 | 37.7971 |
| No log | 5.0 | 310 | 36.8967 |
| No log | 6.0 | 372 | 35.3844 |
| No log | 7.0 | 434 | 34.6435 |
| No log | 8.0 | 496 | 33.6444 |
| 39.4735 | 9.0 | 558 | 32.8915 |
| 39.4735 | 10.0 | 620 | 31.1389 |
| 39.4735 | 11.0 | 682 | 30.0209 |
| 39.4735 | 12.0 | 744 | 28.7981 |
| 39.4735 | 13.0 | 806 | 27.6728 |
| 39.4735 | 14.0 | 868 | 26.6111 |
| 39.4735 | 15.0 | 930 | 25.5562 |
| 39.4735 | 16.0 | 992 | 24.4283 |
| 29.1909 | 17.0 | 1054 | 23.2272 |
| 29.1909 | 18.0 | 1116 | 22.8566 |
| 29.1909 | 19.0 | 1178 | 22.1741 |
| 29.1909 | 20.0 | 1240 | 21.4192 |
| 29.1909 | 21.0 | 1302 | 20.4911 |
| 29.1909 | 22.0 | 1364 | 20.2457 |
| 29.1909 | 23.0 | 1426 | 19.5857 |
| 29.1909 | 24.0 | 1488 | 19.2878 |
| 21.7917 | 25.0 | 1550 | 18.8065 |
| 21.7917 | 26.0 | 1612 | 18.3365 |
| 21.7917 | 27.0 | 1674 | 18.1690 |
| 21.7917 | 28.0 | 1736 | 17.5599 |
| 21.7917 | 29.0 | 1798 | 17.4295 |
| 21.7917 | 30.0 | 1860 | 17.1918 |
| 21.7917 | 31.0 | 1922 | 16.9913 |
| 21.7917 | 32.0 | 1984 | 16.7378 |
| 18.0017 | 33.0 | 2046 | 16.5858 |
| 18.0017 | 34.0 | 2108 | 16.4158 |
| 18.0017 | 35.0 | 2170 | 16.2602 |
| 18.0017 | 36.0 | 2232 | 16.0334 |
| 18.0017 | 37.0 | 2294 | 16.0809 |
| 18.0017 | 38.0 | 2356 | 15.7176 |
| 18.0017 | 39.0 | 2418 | 15.8248 |
| 18.0017 | 40.0 | 2480 | 15.8528 |
| 16.124 | 41.0 | 2542 | 15.7772 |
| 16.124 | 42.0 | 2604 | 15.7973 |
| 16.124 | 43.0 | 2666 | 15.4572 |
| 16.124 | 44.0 | 2728 | 15.6225 |
| 16.124 | 45.0 | 2790 | 15.4301 |
| 16.124 | 46.0 | 2852 | 15.7354 |
| 16.124 | 47.0 | 2914 | 15.5296 |
| 16.124 | 48.0 | 2976 | 15.4213 |
| 15.3787 | 49.0 | 3038 | 15.5128 |
| 15.3787 | 50.0 | 3100 | 15.5961 |
Framework versions
- Transformers 4.21.2
- Pytorch 1.12.1+cu113
- Datasets 2.4.0
- Tokenizers 0.12.1