Moroccan-Darija-STT-large-v1.6.13

This model is a fine-tuned version of openai/whisper-large-v3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2877
• Wer: 95.9672
• Cer: 51.6612

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.377	0.3553	60	0.2993	80.8986	39.0082
0.3482	0.7106	120	0.2687	85.6509	44.0940
0.2801	1.0711	180	0.2702	100.8450	54.9651
0.2925	1.4264	240	0.2717	83.8019	39.5098
0.3023	1.7816	300	0.2620	109.8143	62.4951
0.268	2.1421	360	0.2787	92.6623	49.6867
0.2583	2.4974	420	0.2800	85.2242	42.9488
0.2721	2.8527	480	0.2771	105.8986	59.2858
0.2384	3.2132	540	0.2897	95.5991	49.2306
0.2231	3.5685	600	0.2843	101.2132	58.2673
0.2495	3.9238	660	0.2877	95.9672	51.6612

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0