Moroccan-Darija-STT-large-v1.6.10

This model is a fine-tuned version of openai/whisper-large-v3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5413
• Wer: 78.1961
• Cer: 38.0589

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10
• num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.8531	0.1473	30	0.3501	188.0940	155.4803
0.666	0.2947	60	0.3092	104.0495	61.4107
0.578	0.4420	90	0.3034	79.7356	38.7565
0.5255	0.5893	120	0.3006	77.0164	35.8951
0.4326	0.7366	150	0.3038	75.2175	32.5845
0.4941	0.8840	180	0.3091	71.1094	29.9579
0.3748	1.0344	210	0.3131	79.5766	39.2294
0.3613	1.1817	240	0.3339	72.2557	31.5373
0.3328	1.3290	270	0.3273	70.8835	30.4174
0.3032	1.4764	300	0.3430	69.4863	29.2451
0.2933	1.6237	330	0.3484	74.7406	34.7466
0.2593	1.7710	360	0.3523	75.3263	34.0287
0.2487	1.9184	390	0.3500	83.5843	41.7783
0.1815	2.0688	420	0.3783	81.2249	40.5233
0.1648	2.2161	450	0.3847	78.9324	38.8409
0.1533	2.3634	480	0.3979	78.7149	37.9981
0.1358	2.5107	510	0.4050	77.1670	36.0556
0.1261	2.6581	540	0.4099	80.7982	41.2479
0.1274	2.8054	570	0.4234	81.8106	41.7834
0.1141	2.9527	600	0.4223	79.1499	38.9389
0.0713	3.1031	630	0.4516	93.3233	52.0920
0.0624	3.2505	660	0.4555	81.7185	40.3071
0.066	3.3978	690	0.4652	82.4799	42.5181
0.063	3.5451	720	0.4673	79.5850	39.5909
0.059	3.6924	750	0.4690	86.5462	46.5010
0.0519	3.8398	780	0.4795	86.8725	47.4554
0.0514	3.9871	810	0.4876	86.3286	45.5213
0.0367	4.1375	840	0.5028	83.6931	43.3796
0.0353	4.2848	870	0.5070	80.6225	40.6888
0.0367	4.4322	900	0.5084	78.1961	38.5470
0.0309	4.5795	930	0.5146	79.6854	39.2396
0.0328	4.7268	960	0.5134	78.2965	39.1517
0.0314	4.8742	990	0.5284	78.4137	38.3274
0.0257	5.0246	1020	0.5273	76.3554	37.1214
0.0229	5.1719	1050	0.5342	84.5298	43.8677
0.0245	5.3192	1080	0.5354	82.0783	42.5080
0.0203	5.4665	1110	0.5407	86.2533	46.4622
0.0237	5.6139	1140	0.5427	81.3922	40.7547
0.021	5.7612	1170	0.5413	77.2005	37.2566
0.0229	5.9085	1200	0.5413	78.1961	38.0589

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0