---
library_name: transformers
language:
- sn
license: cc-by-nc-4.0
base_model: facebook/mms-300m
tags:
- generated_from_trainer
datasets:
- DigitalUmuganda_Afrivoice/Shona
metrics:
- wer
model-index:
- name: facebook/mms-300m
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: DigitalUmuganda
      type: DigitalUmuganda_Afrivoice/Shona
    metrics:
    - name: Wer
      type: wer
      value: 0.4128480489819719
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# facebook/mms-300m

This model is a fine-tuned version of [facebook/mms-300m](https://huggingface.co/facebook/mms-300m) on the DigitalUmuganda dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7396
- Wer: 0.4128
- Cer: 0.0953

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 150
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step | Validation Loss | Wer    | Cer    |
|:-------------:|:-------:|:----:|:---------------:|:------:|:------:|
| 23.8246       | 0.9954  | 109  | 2.9644          | 1.0    | 1.0    |
| 11.6666       | 2.0     | 219  | 2.8975          | 1.0    | 1.0    |
| 11.5956       | 2.9954  | 328  | 2.8467          | 1.0    | 1.0    |
| 10.3096       | 4.0     | 438  | 1.9758          | 1.0    | 0.6253 |
| 5.5486        | 4.9954  | 547  | 0.7431          | 0.9025 | 0.2232 |
| 2.263         | 6.0     | 657  | 0.3758          | 0.5049 | 0.0977 |
| 1.4016        | 6.9954  | 766  | 0.3167          | 0.4258 | 0.0805 |
| 1.0646        | 8.0     | 876  | 0.2821          | 0.3744 | 0.0702 |
| 0.8883        | 8.9954  | 985  | 0.2754          | 0.3581 | 0.0679 |
| 0.7279        | 10.0    | 1095 | 0.2653          | 0.3543 | 0.0669 |
| 0.644         | 10.9954 | 1204 | 0.2632          | 0.3417 | 0.0610 |
| 0.5561        | 12.0    | 1314 | 0.2647          | 0.3336 | 0.0597 |
| 0.5029        | 12.9954 | 1423 | 0.2868          | 0.3368 | 0.0614 |
| 0.434         | 14.0    | 1533 | 0.2981          | 0.3239 | 0.0583 |
| 0.3766        | 14.9954 | 1642 | 0.2986          | 0.3327 | 0.0604 |
| 0.3356        | 16.0    | 1752 | 0.3170          | 0.3150 | 0.0562 |
| 0.3046        | 16.9954 | 1861 | 0.3350          | 0.3278 | 0.0579 |
| 0.2801        | 18.0    | 1971 | 0.3359          | 0.3358 | 0.0598 |
| 0.2601        | 18.9954 | 2080 | 0.3392          | 0.3341 | 0.0599 |
| 0.2339        | 20.0    | 2190 | 0.3526          | 0.3282 | 0.0584 |
| 0.2175        | 20.9954 | 2299 | 0.3517          | 0.3317 | 0.0582 |
| 0.2031        | 22.0    | 2409 | 0.3547          | 0.3275 | 0.0564 |
| 0.197         | 22.9954 | 2518 | 0.3688          | 0.3164 | 0.0552 |
| 0.1751        | 24.0    | 2628 | 0.3724          | 0.3212 | 0.0570 |
| 0.1738        | 24.9954 | 2737 | 0.3778          | 0.3035 | 0.0539 |
| 0.1656        | 26.0    | 2847 | 0.3946          | 0.3120 | 0.0547 |
| 0.163         | 26.9954 | 2956 | 0.3770          | 0.3089 | 0.0542 |
| 0.1484        | 28.0    | 3066 | 0.3871          | 0.3151 | 0.0568 |
| 0.1452        | 28.9954 | 3175 | 0.3712          | 0.3038 | 0.0535 |
| 0.1335        | 30.0    | 3285 | 0.3989          | 0.3085 | 0.0544 |
| 0.1337        | 30.9954 | 3394 | 0.3880          | 0.3064 | 0.0543 |
| 0.126         | 32.0    | 3504 | 0.4123          | 0.3079 | 0.0540 |
| 0.1252        | 32.9954 | 3613 | 0.3959          | 0.3070 | 0.0544 |
| 0.1229        | 34.0    | 3723 | 0.3975          | 0.2989 | 0.0533 |
| 0.1198        | 34.9954 | 3832 | 0.4194          | 0.3020 | 0.0531 |
| 0.1136        | 36.0    | 3942 | 0.3916          | 0.3056 | 0.0534 |
| 0.1186        | 36.9954 | 4051 | 0.3939          | 0.3040 | 0.0535 |
| 0.1078        | 38.0    | 4161 | 0.4042          | 0.3011 | 0.0532 |
| 0.108         | 38.9954 | 4270 | 0.4044          | 0.3000 | 0.0526 |
| 0.1011        | 40.0    | 4380 | 0.4309          | 0.2997 | 0.0522 |
| 0.106         | 40.9954 | 4489 | 0.4245          | 0.2941 | 0.0517 |
| 0.0994        | 42.0    | 4599 | 0.4268          | 0.3059 | 0.0528 |
| 0.0982        | 42.9954 | 4708 | 0.4239          | 0.2996 | 0.0524 |
| 0.0956        | 44.0    | 4818 | 0.4175          | 0.2974 | 0.0525 |
| 0.0931        | 44.9954 | 4927 | 0.4677          | 0.2934 | 0.0518 |
| 0.0886        | 46.0    | 5037 | 0.4262          | 0.2979 | 0.0523 |
| 0.0874        | 46.9954 | 5146 | 0.4274          | 0.2927 | 0.0514 |
| 0.0882        | 48.0    | 5256 | 0.4230          | 0.2988 | 0.0524 |
| 0.0856        | 48.9954 | 5365 | 0.4169          | 0.2872 | 0.0505 |
| 0.0826        | 50.0    | 5475 | 0.4092          | 0.3034 | 0.0522 |
| 0.0828        | 50.9954 | 5584 | 0.4383          | 0.2869 | 0.0507 |
| 0.0798        | 52.0    | 5694 | 0.4201          | 0.2968 | 0.0512 |
| 0.0761        | 52.9954 | 5803 | 0.3999          | 0.2871 | 0.0498 |
| 0.0753        | 54.0    | 5913 | 0.4303          | 0.2914 | 0.0502 |
| 0.0743        | 54.9954 | 6022 | 0.4424          | 0.2937 | 0.0507 |
| 0.0788        | 56.0    | 6132 | 0.4214          | 0.2877 | 0.0495 |
| 0.0752        | 56.9954 | 6241 | 0.4293          | 0.2925 | 0.0507 |
| 0.0713        | 58.0    | 6351 | 0.4494          | 0.2919 | 0.0504 |
| 0.0693        | 58.9954 | 6460 | 0.4297          | 0.2882 | 0.0498 |


### Framework versions

- Transformers 4.47.0.dev0
- Pytorch 2.1.0+cu118
- Datasets 3.0.2
- Tokenizers 0.20.1