Moroccan-Darija-STT-large-turbo-v1.6.9

This model is a fine-tuned version of openai/whisper-large-v3-turbo on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4080
• Wer: 102.7276
• Cer: 65.9426

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.375e-06
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10
• num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.9217	0.1472	30	0.3522	111.8976	91.8703
0.8127	0.2945	60	0.3268	86.7721	43.5214
0.7222	0.4417	90	0.3221	117.1268	73.3527
0.6824	0.5890	120	0.3134	91.8842	47.1969
0.5771	0.7362	150	0.3122	87.0649	41.8475
0.6673	0.8834	180	0.3114	69.1600	27.4192
0.5596	1.0294	210	0.3160	71.5445	29.5627
0.5592	1.1767	240	0.3179	70.0218	28.3364
0.5145	1.3239	270	0.3183	74.7490	31.6825
0.5074	1.4712	300	0.3225	73.7533	31.7467
0.4832	1.6184	330	0.3277	74.7406	32.9848
0.487	1.7656	360	0.3281	81.7436	38.9693
0.4738	1.9129	390	0.3338	90.0937	49.8134
0.428	2.0589	420	0.3396	83.3166	42.1786
0.3742	2.2061	450	0.3448	85.0569	43.1870
0.421	2.3534	480	0.3526	87.7259	46.7172
0.3903	2.5006	510	0.3529	85.4501	43.4657
0.4019	2.6479	540	0.3490	84.9481	44.2275
0.4212	2.7951	570	0.3596	87.6088	44.9369
0.4261	2.9423	600	0.3524	99.9749	61.7182
0.3387	3.0883	630	0.3663	77.3427	37.6822
0.3472	3.2356	660	0.3659	79.1667	38.2768
0.3667	3.3828	690	0.3718	104.2754	62.9022
0.3392	3.5301	720	0.3742	88.8303	50.9805
0.3674	3.6773	750	0.3764	88.0438	45.4977
0.3445	3.8245	780	0.3770	97.0633	58.8957
0.3134	3.9718	810	0.3813	101.7821	60.5983
0.342	4.1178	840	0.3875	107.3628	67.6655
0.3285	4.2650	870	0.3908	105.2711	65.4291
0.301	4.4123	900	0.3899	95.9337	57.7217
0.3269	4.5595	930	0.3952	126.8407	90.2758
0.3114	4.7067	960	0.3946	90.6376	52.7676
0.3126	4.8540	990	0.3970	92.5033	55.7387
0.2979	5.0	1020	0.4006	97.6155	58.8923
0.2771	5.1472	1050	0.4027	111.2199	72.1636
0.3053	5.2945	1080	0.4067	91.8591	56.8890
0.3138	5.4417	1110	0.4069	97.4314	60.4496
0.3056	5.5890	1140	0.4078	95.2393	59.5426
0.3026	5.7362	1170	0.4077	90.2025	55.9431
0.3068	5.8834	1200	0.4080	102.7276	65.9426

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0