---
library_name: transformers
license: mit
base_model: facebook/w2v-bert-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: w2v-bert-2.0-CV_Fleurs-lg-100hrs-v4
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# w2v-bert-2.0-CV_Fleurs-lg-100hrs-v4

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2835
- Wer: 0.2749
- Cer: 0.0546

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 8
- eval_batch_size: 4
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:------:|:---------------:|:------:|:------:|
| 1.3102        | 1.0   | 7057   | 0.3038          | 0.3739 | 0.0787 |
| 0.1944        | 2.0   | 14114  | 0.2340          | 0.3188 | 0.0634 |
| 0.1558        | 3.0   | 21171  | 0.2157          | 0.3021 | 0.0598 |
| 0.1373        | 4.0   | 28228  | 0.2037          | 0.2926 | 0.0574 |
| 0.1237        | 5.0   | 35285  | 0.2079          | 0.3025 | 0.0591 |
| 0.1152        | 6.0   | 42342  | 0.2044          | 0.2899 | 0.0583 |
| 0.1091        | 7.0   | 49399  | 0.2025          | 0.3012 | 0.0567 |
| 0.1042        | 8.0   | 56456  | 0.2033          | 0.2728 | 0.0552 |
| 0.0985        | 9.0   | 63513  | 0.2035          | 0.2760 | 0.0543 |
| 0.0949        | 10.0  | 70570  | 0.2054          | 0.2730 | 0.0541 |
| 0.0885        | 11.0  | 77627  | 0.2071          | 0.2758 | 0.0557 |
| 0.08          | 12.0  | 84684  | 0.2039          | 0.2743 | 0.0540 |
| 0.0719        | 13.0  | 91741  | 0.2045          | 0.2648 | 0.0531 |
| 0.063         | 14.0  | 98798  | 0.2133          | 0.2682 | 0.0544 |
| 0.0555        | 15.0  | 105855 | 0.2107          | 0.2658 | 0.0524 |
| 0.0471        | 16.0  | 112912 | 0.2408          | 0.2687 | 0.0529 |
| 0.0399        | 17.0  | 119969 | 0.2419          | 0.2721 | 0.0535 |
| 0.0341        | 18.0  | 127026 | 0.2587          | 0.2833 | 0.0551 |
| 0.0293        | 19.0  | 134083 | 0.2518          | 0.2735 | 0.0534 |
| 0.0255        | 20.0  | 141140 | 0.2795          | 0.2669 | 0.0530 |
| 0.022         | 21.0  | 148197 | 0.2748          | 0.2820 | 0.0543 |
| 0.0196        | 22.0  | 155254 | 0.2837          | 0.2785 | 0.0529 |
| 0.017         | 23.0  | 162311 | 0.2835          | 0.2749 | 0.0546 |


### Framework versions

- Transformers 4.46.0
- Pytorch 2.1.0+cu118
- Datasets 3.0.2
- Tokenizers 0.20.1