---
library_name: transformers
license: mit
base_model: facebook/w2v-bert-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: W2V2_Bert_BIG_C_Bemba_20hr_v1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# W2V2_Bert_BIG_C_Bemba_20hr_v1

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6286
- Wer: 0.4136
- Cer: 0.1049

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.025
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|
| 2.1779        | 1.0   | 2546  | 0.8771          | 0.6175 | 0.1688 |
| 0.8215        | 2.0   | 5092  | 0.7074          | 0.5268 | 0.1557 |
| 0.7007        | 3.0   | 7638  | 0.6562          | 0.4932 | 0.1365 |
| 0.6419        | 4.0   | 10184 | 0.6431          | 0.4636 | 0.1263 |
| 0.5961        | 5.0   | 12730 | 0.6088          | 0.4746 | 0.1309 |
| 0.5581        | 6.0   | 15276 | 0.7341          | 0.4436 | 0.1240 |
| 0.5237        | 7.0   | 17822 | 0.6851          | 0.4406 | 0.1218 |
| 0.4935        | 8.0   | 20368 | 0.6648          | 0.4368 | 0.1206 |
| 0.4622        | 9.0   | 22914 | 0.6347          | 0.4320 | 0.1203 |
| 0.4284        | 10.0  | 25460 | 0.6552          | 0.4380 | 0.1206 |
| 0.4001        | 11.0  | 28006 | 0.6317          | 0.448  | 0.1259 |
| 0.3686        | 12.0  | 30552 | 0.6987          | 0.4329 | 0.1201 |
| 0.3397        | 13.0  | 33098 | 0.7235          | 0.4236 | 0.1183 |
| 0.3103        | 14.0  | 35644 | 0.7169          | 0.4386 | 0.1205 |
| 0.2839        | 15.0  | 38190 | 0.7628          | 0.4372 | 0.1226 |
| 0.2569        | 16.0  | 40736 | 0.7728          | 0.4476 | 0.1263 |
| 0.2316        | 17.0  | 43282 | 0.8605          | 0.4384 | 0.1255 |
| 0.2112        | 18.0  | 45828 | 0.8648          | 0.4619 | 0.1291 |
| 0.1889        | 19.0  | 48374 | 0.9461          | 0.4420 | 0.1280 |
| 0.1698        | 20.0  | 50920 | 1.0526          | 0.4482 | 0.1262 |
| 0.1529        | 21.0  | 53466 | 1.1109          | 0.4454 | 0.1249 |
| 0.136         | 22.0  | 56012 | 1.1421          | 0.4521 | 0.1283 |
| 0.1218        | 23.0  | 58558 | 1.1437          | 0.4585 | 0.1296 |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.2.0+cu121
- Datasets 3.0.0
- Tokenizers 0.19.1