Moroccan-Darija-STT-large-v1.5.3

This model is a fine-tuned version of openai/whisper-large-v3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4505
• Wer: 160.7702
• Cer: 139.4884

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 256
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
20.3476	1.1963	25	0.8233	92.5391	49.6955
22.8276	2.3926	50	0.8013	92.4188	49.2814
22.0032	3.5890	75	0.7578	93.0205	49.5006
21.1859	4.7853	100	0.6853	92.9001	48.9403
19.2673	5.9816	125	0.6291	91.0951	49.7686
16.5596	7.1472	150	0.5911	91.5764	49.6468
17.2825	8.3436	175	0.5518	91.6968	50.4994
16.7783	9.5399	200	0.5374	93.8628	55.2010
16.5996	10.7362	225	0.5287	93.8628	56.5652
15.928	11.9325	250	0.5216	94.3442	59.3423
13.6163	13.0982	275	0.5148	95.1865	61.6565
15.5083	14.2945	300	0.5082	95.1865	62.2412
15.1642	15.4908	325	0.5034	97.1119	64.6285
14.5258	16.6871	350	0.4967	100.4813	73.5932
14.3617	17.8834	375	0.4929	101.0830	74.7868
12.645	19.0491	400	0.4877	101.4440	75.6882
13.9231	20.2454	425	0.4837	101.4440	75.9074
13.9187	21.4417	450	0.4805	101.2034	75.7369
13.7615	22.6380	475	0.4767	102.7677	81.5104
13.2719	23.8344	500	0.4741	110.1083	82.4604
11.9294	25.0	525	0.4704	145.0060	117.5883
11.6505	26.1963	550	0.4689	145.3670	121.9976
13.275	27.3926	575	0.4661	146.4501	123.2887
13.0894	28.5890	600	0.4638	145.8484	123.2156
12.7894	29.7853	625	0.4619	138.7485	122.9476
13.1045	30.9816	650	0.4600	137.9061	119.5128
11.3755	32.1472	675	0.4583	145.0060	120.8283
12.7722	33.3436	700	0.4575	145.1264	122.4117
12.7683	34.5399	725	0.4559	136.3418	119.8051
12.9925	35.7362	750	0.4547	144.4043	120.1218
12.6664	36.9325	775	0.4541	143.2010	120.6577
10.8082	38.0982	800	0.4539	144.5247	121.4373
12.806	39.2945	825	0.4527	143.5620	119.3179
12.305	40.4908	850	0.4525	135.6197	118.7089
12.2581	41.6871	875	0.4518	142.7196	120.6821
12.4142	42.8834	900	0.4514	143.2010	118.3922
10.7696	44.0491	925	0.4514	143.6823	121.0475
12.3338	45.2454	950	0.4505	134.6570	118.7089
12.1262	46.4417	975	0.4506	160.2888	139.1230
12.3791	47.6380	1000	0.4504	160.0481	138.7333
12.7156	48.8344	1025	0.4503	160.6498	139.2935
10.5394	50.0	1050	0.4502	134.7774	118.4409
10.764	51.1963	1075	0.4504	159.9278	138.7333
12.4898	52.3926	1100	0.4504	168.3514	140.7308
11.9754	53.5890	1125	0.4501	161.0108	137.1742
12.3212	54.7853	1150	0.4502	168.5921	141.1693
12.6654	55.9816	1175	0.4502	160.4091	139.3666
11.0194	57.1472	1200	0.4505	160.7702	139.4884

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0