Moroccan-Darija-STT-large-v1.6.2

This model is a fine-tuned version of openai/whisper-large-v3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3740
• Wer: 126.2718
• Cer: 95.6202

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.0178	0.3067	50	0.5184	161.0693	141.2192
1.4866	0.6135	100	0.4190	258.3584	229.4698
1.2886	0.9202	150	0.3359	137.6171	99.2230
1.1267	1.2270	200	0.3335	126.2969	83.9535
1.0166	1.5337	250	0.3309	122.8999	84.7508
1.0861	1.8405	300	0.3269	118.5492	79.1902
0.8318	2.1472	350	0.3260	140.9639	104.9508
0.8914	2.4540	400	0.3262	121.7035	83.6613
0.7825	2.7607	450	0.3261	119.2185	81.6226
0.6106	3.0675	500	0.3404	109.5131	69.9458
0.683	3.3742	550	0.3300	122.5234	88.4803
0.6084	3.6810	600	0.3341	119.3775	82.7492
0.6353	3.9877	650	0.3423	120.9337	85.5041
0.6988	4.2945	700	0.3470	114.6670	79.1598
0.6906	4.6012	750	0.3498	121.4692	88.6695
0.5259	4.9080	800	0.3451	114.5917	79.7274
0.4837	5.2147	850	0.3565	124.7490	93.2807
0.569	5.5215	900	0.3603	122.3561	90.2995
0.5982	5.8282	950	0.3616	115.5539	82.9316
0.4429	6.1350	1000	0.3716	120.1724	88.0563
0.4223	6.4417	1050	0.3727	121.1931	90.5630
0.4495	6.7485	1100	0.3740	126.2718	95.6202

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0