---
library_name: transformers
license: apache-2.0
base_model: openai/whisper-large-v3
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: Moroccan-Darija-STT-large-v1.6.10
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Moroccan-Darija-STT-large-v1.6.10

This model is a fine-tuned version of [openai/whisper-large-v3](https://huggingface.co/openai/whisper-large-v3) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5413
- Wer: 78.1961
- Cer: 38.0589

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 128
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10
- num_epochs: 6

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Wer      | Cer      |
|:-------------:|:------:|:----:|:---------------:|:--------:|:--------:|
| 0.8531        | 0.1473 | 30   | 0.3501          | 188.0940 | 155.4803 |
| 0.666         | 0.2947 | 60   | 0.3092          | 104.0495 | 61.4107  |
| 0.578         | 0.4420 | 90   | 0.3034          | 79.7356  | 38.7565  |
| 0.5255        | 0.5893 | 120  | 0.3006          | 77.0164  | 35.8951  |
| 0.4326        | 0.7366 | 150  | 0.3038          | 75.2175  | 32.5845  |
| 0.4941        | 0.8840 | 180  | 0.3091          | 71.1094  | 29.9579  |
| 0.3748        | 1.0344 | 210  | 0.3131          | 79.5766  | 39.2294  |
| 0.3613        | 1.1817 | 240  | 0.3339          | 72.2557  | 31.5373  |
| 0.3328        | 1.3290 | 270  | 0.3273          | 70.8835  | 30.4174  |
| 0.3032        | 1.4764 | 300  | 0.3430          | 69.4863  | 29.2451  |
| 0.2933        | 1.6237 | 330  | 0.3484          | 74.7406  | 34.7466  |
| 0.2593        | 1.7710 | 360  | 0.3523          | 75.3263  | 34.0287  |
| 0.2487        | 1.9184 | 390  | 0.3500          | 83.5843  | 41.7783  |
| 0.1815        | 2.0688 | 420  | 0.3783          | 81.2249  | 40.5233  |
| 0.1648        | 2.2161 | 450  | 0.3847          | 78.9324  | 38.8409  |
| 0.1533        | 2.3634 | 480  | 0.3979          | 78.7149  | 37.9981  |
| 0.1358        | 2.5107 | 510  | 0.4050          | 77.1670  | 36.0556  |
| 0.1261        | 2.6581 | 540  | 0.4099          | 80.7982  | 41.2479  |
| 0.1274        | 2.8054 | 570  | 0.4234          | 81.8106  | 41.7834  |
| 0.1141        | 2.9527 | 600  | 0.4223          | 79.1499  | 38.9389  |
| 0.0713        | 3.1031 | 630  | 0.4516          | 93.3233  | 52.0920  |
| 0.0624        | 3.2505 | 660  | 0.4555          | 81.7185  | 40.3071  |
| 0.066         | 3.3978 | 690  | 0.4652          | 82.4799  | 42.5181  |
| 0.063         | 3.5451 | 720  | 0.4673          | 79.5850  | 39.5909  |
| 0.059         | 3.6924 | 750  | 0.4690          | 86.5462  | 46.5010  |
| 0.0519        | 3.8398 | 780  | 0.4795          | 86.8725  | 47.4554  |
| 0.0514        | 3.9871 | 810  | 0.4876          | 86.3286  | 45.5213  |
| 0.0367        | 4.1375 | 840  | 0.5028          | 83.6931  | 43.3796  |
| 0.0353        | 4.2848 | 870  | 0.5070          | 80.6225  | 40.6888  |
| 0.0367        | 4.4322 | 900  | 0.5084          | 78.1961  | 38.5470  |
| 0.0309        | 4.5795 | 930  | 0.5146          | 79.6854  | 39.2396  |
| 0.0328        | 4.7268 | 960  | 0.5134          | 78.2965  | 39.1517  |
| 0.0314        | 4.8742 | 990  | 0.5284          | 78.4137  | 38.3274  |
| 0.0257        | 5.0246 | 1020 | 0.5273          | 76.3554  | 37.1214  |
| 0.0229        | 5.1719 | 1050 | 0.5342          | 84.5298  | 43.8677  |
| 0.0245        | 5.3192 | 1080 | 0.5354          | 82.0783  | 42.5080  |
| 0.0203        | 5.4665 | 1110 | 0.5407          | 86.2533  | 46.4622  |
| 0.0237        | 5.6139 | 1140 | 0.5427          | 81.3922  | 40.7547  |
| 0.021         | 5.7612 | 1170 | 0.5413          | 77.2005  | 37.2566  |
| 0.0229        | 5.9085 | 1200 | 0.5413          | 78.1961  | 38.0589  |


### Framework versions

- Transformers 4.48.0
- Pytorch 2.5.1+cu124
- Datasets 3.1.0
- Tokenizers 0.21.0