---
library_name: transformers
license: mit
base_model: facebook/w2v-bert-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: w2v-bert-2.0-CV_Fleurs-lg-20hrs-v5
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# w2v-bert-2.0-CV_Fleurs-lg-20hrs-v5

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4606
- Wer: 0.3643
- Cer: 0.0782

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 2
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 80
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|
| 1.4915        | 1.0   | 2058  | 0.3507          | 0.4166 | 0.0815 |
| 0.3195        | 2.0   | 4116  | 0.3251          | 0.3885 | 0.0772 |
| 0.2817        | 3.0   | 6174  | 0.3035          | 0.3921 | 0.0787 |
| 0.27          | 4.0   | 8232  | 0.3337          | 0.4144 | 0.0824 |
| 0.2645        | 5.0   | 10290 | 0.3604          | 0.4144 | 0.0849 |
| 0.2579        | 6.0   | 12348 | 0.3396          | 0.4502 | 0.0933 |
| 0.2609        | 7.0   | 14406 | 0.3439          | 0.3976 | 0.0830 |
| 0.2557        | 8.0   | 16464 | 0.3807          | 0.4361 | 0.0953 |
| 0.242         | 9.0   | 18522 | 0.3477          | 0.3997 | 0.0841 |
| 0.2198        | 10.0  | 20580 | 0.3354          | 0.3986 | 0.0845 |
| 0.1912        | 11.0  | 22638 | 0.3337          | 0.3951 | 0.0837 |
| 0.1716        | 12.0  | 24696 | 0.3179          | 0.3646 | 0.0779 |
| 0.1566        | 13.0  | 26754 | 0.3486          | 0.3747 | 0.0797 |
| 0.1422        | 14.0  | 28812 | 0.3320          | 0.3838 | 0.0808 |
| 0.1284        | 15.0  | 30870 | 0.3482          | 0.3668 | 0.0807 |
| 0.1142        | 16.0  | 32928 | 0.3330          | 0.3721 | 0.0780 |
| 0.1005        | 17.0  | 34986 | 0.3272          | 0.3539 | 0.0738 |
| 0.0897        | 18.0  | 37044 | 0.3906          | 0.3732 | 0.0763 |
| 0.0787        | 19.0  | 39102 | 0.3827          | 0.3597 | 0.0755 |
| 0.0697        | 20.0  | 41160 | 0.3883          | 0.3586 | 0.0770 |
| 0.0632        | 21.0  | 43218 | 0.3923          | 0.3798 | 0.0797 |
| 0.0544        | 22.0  | 45276 | 0.4401          | 0.3689 | 0.0803 |
| 0.0503        | 23.0  | 47334 | 0.4111          | 0.3704 | 0.0790 |
| 0.0438        | 24.0  | 49392 | 0.4019          | 0.3599 | 0.0762 |
| 0.0392        | 25.0  | 51450 | 0.4198          | 0.3625 | 0.0774 |
| 0.0372        | 26.0  | 53508 | 0.4374          | 0.3650 | 0.0794 |
| 0.0333        | 27.0  | 55566 | 0.4606          | 0.3643 | 0.0782 |


### Framework versions

- Transformers 4.46.1
- Pytorch 2.1.0+cu118
- Datasets 3.1.0
- Tokenizers 0.20.1