|
--- |
|
base_model: facebook/w2v-bert-2.0 |
|
datasets: |
|
- common_voice_17_0 |
|
license: mit |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: wav2vec2-bert-turkish |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# wav2vec2-bert-turkish |
|
|
|
This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on the common_voice_17_0 dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.3552 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0001 |
|
- train_batch_size: 4 |
|
- eval_batch_size: 4 |
|
- seed: 42 |
|
- distributed_type: multi-GPU |
|
- num_devices: 2 |
|
- total_train_batch_size: 8 |
|
- total_eval_batch_size: 8 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 1000 |
|
- num_epochs: 50 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-------:|:-----:|:---------------:| |
|
| 1.0927 | 0.1724 | 1000 | 0.6278 | |
|
| 0.4967 | 0.3448 | 2000 | 0.5884 | |
|
| 0.3964 | 0.5172 | 3000 | 0.4851 | |
|
| 0.355 | 0.6895 | 4000 | 0.5371 | |
|
| 0.3264 | 0.8619 | 5000 | 0.4579 | |
|
| 0.2979 | 1.0343 | 6000 | 0.4308 | |
|
| 0.2568 | 1.2067 | 7000 | 0.4136 | |
|
| 0.2495 | 1.3791 | 8000 | 0.4711 | |
|
| 0.2422 | 1.5515 | 9000 | 0.4280 | |
|
| 0.2357 | 1.7238 | 10000 | 0.4045 | |
|
| 0.2193 | 1.8962 | 11000 | 0.4194 | |
|
| 0.2087 | 2.0686 | 12000 | 0.4427 | |
|
| 0.1819 | 2.2410 | 13000 | 0.4155 | |
|
| 0.1772 | 2.4134 | 14000 | 0.4012 | |
|
| 0.1739 | 2.5858 | 15000 | 0.3651 | |
|
| 0.172 | 2.7581 | 16000 | 0.4081 | |
|
| 0.1676 | 2.9305 | 17000 | 0.3948 | |
|
| 0.1498 | 3.1029 | 18000 | 0.3587 | |
|
| 0.1299 | 3.2753 | 19000 | 0.4106 | |
|
| 0.1319 | 3.4477 | 20000 | 0.3624 | |
|
| 0.1425 | 3.6201 | 21000 | 0.3551 | |
|
| 0.1362 | 3.7924 | 22000 | 0.3504 | |
|
| 0.1386 | 3.9648 | 23000 | 0.3454 | |
|
| 0.1106 | 4.1372 | 24000 | 0.3632 | |
|
| 0.1069 | 4.3096 | 25000 | 0.3404 | |
|
| 0.1155 | 4.4820 | 26000 | 0.3517 | |
|
| 0.1162 | 4.6544 | 27000 | 0.3315 | |
|
| 0.1121 | 4.8268 | 28000 | 0.3521 | |
|
| 0.1109 | 4.9991 | 29000 | 0.3456 | |
|
| 0.0875 | 5.1715 | 30000 | 0.3507 | |
|
| 0.0963 | 5.3439 | 31000 | 0.3878 | |
|
| 0.0933 | 5.5163 | 32000 | 0.3653 | |
|
| 0.0988 | 5.6887 | 33000 | 0.3427 | |
|
| 0.0912 | 5.8611 | 34000 | 0.3582 | |
|
| 0.0889 | 6.0334 | 35000 | 0.3262 | |
|
| 0.0769 | 6.2058 | 36000 | 0.3548 | |
|
| 0.08 | 6.3782 | 37000 | 0.4327 | |
|
| 0.0821 | 6.5506 | 38000 | 0.3374 | |
|
| 0.0841 | 6.7230 | 39000 | 0.3522 | |
|
| 0.0826 | 6.8954 | 40000 | 0.3499 | |
|
| 0.0773 | 7.0677 | 41000 | 0.3434 | |
|
| 0.07 | 7.2401 | 42000 | 0.3453 | |
|
| 0.0695 | 7.4125 | 43000 | 0.3455 | |
|
| 0.073 | 7.5849 | 44000 | 0.3614 | |
|
| 0.0705 | 7.7573 | 45000 | 0.3209 | |
|
| 0.0759 | 7.9297 | 46000 | 0.3455 | |
|
| 0.0599 | 8.1021 | 47000 | 0.3237 | |
|
| 0.0617 | 8.2744 | 48000 | 0.3298 | |
|
| 0.0605 | 8.4468 | 49000 | 0.3684 | |
|
| 0.0594 | 8.6192 | 50000 | 0.3623 | |
|
| 0.0631 | 8.7916 | 51000 | 0.3582 | |
|
| 0.0625 | 8.9640 | 52000 | 0.3469 | |
|
| 0.0504 | 9.1364 | 53000 | 0.3462 | |
|
| 0.0502 | 9.3087 | 54000 | 0.3417 | |
|
| 0.0551 | 9.4811 | 55000 | 0.3526 | |
|
| 0.0548 | 9.6535 | 56000 | 0.3359 | |
|
| 0.0563 | 9.8259 | 57000 | 0.3581 | |
|
| 0.056 | 9.9983 | 58000 | 0.3421 | |
|
| 0.042 | 10.1707 | 59000 | 0.3349 | |
|
| 0.05 | 10.3430 | 60000 | 0.3552 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.44.0 |
|
- Pytorch 2.4.0+cu121 |
|
- Datasets 2.18.0 |
|
- Tokenizers 0.19.1 |
|
|
