MMS-Wolof-20-hour-Mixed-dataset

This model is a fine-tuned version of facebook/mms-1b-all on the fleurs dataset. It achieves the following results on the evaluation set:

• Loss: 1.3654
• Wer: 0.4749
• Cer: 0.1708

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
9.5604	0.7369	500	0.7373	0.5227	0.1732
2.6975	1.4738	1000	0.6973	0.5301	0.1779
2.726	2.2108	1500	0.8814	0.5966	0.2092
2.8093	2.9477	2000	0.8858	0.6151	0.2303
2.78	3.6846	2500	0.8017	0.5918	0.2146
2.8066	4.4215	3000	1.0143	0.6655	0.2453
2.8916	5.1584	3500	0.8609	0.6299	0.2351
2.7853	5.8954	4000	0.8444	0.6100	0.2304
2.6374	6.6323	4500	0.9584	0.6420	0.2505
2.5197	7.3692	5000	0.9859	0.6703	0.2562
2.4943	8.1061	5500	0.8587	0.6267	0.2405
2.39	8.8430	6000	0.9456	0.6221	0.2388
2.2478	9.5800	6500	0.9030	0.6598	0.2536
2.1688	10.3169	7000	0.8462	0.6527	0.2567
2.0468	11.0538	7500	0.8851	0.6080	0.2302
1.9318	11.7907	8000	0.8898	0.6276	0.2301
1.8937	12.5276	8500	0.7846	0.5826	0.2191
1.7943	13.2646	9000	0.8560	0.6137	0.2326
1.7337	14.0015	9500	0.8789	0.6005	0.2271
1.608	14.7384	10000	0.8736	0.6092	0.2270
1.5533	15.4753	10500	0.9045	0.5951	0.2287
1.4501	16.2122	11000	0.8505	0.6288	0.2415
1.4074	16.9492	11500	0.8023	0.5853	0.2227
1.3339	17.6861	12000	0.8177	0.5967	0.2211
1.2208	18.4230	12500	0.8922	0.5761	0.2126
1.1803	19.1599	13000	0.8207	0.5637	0.2076
1.1159	19.8968	13500	0.8114	0.5473	0.2013
1.0415	20.6338	14000	0.8646	0.5533	0.2036
0.9767	21.3707	14500	0.9001	0.5569	0.2088
0.9803	22.1076	15000	0.8485	0.5696	0.2100
0.8933	22.8445	15500	0.8164	0.5452	0.2041
0.8509	23.5814	16000	0.9136	0.5545	0.2037
0.8398	24.3183	16500	0.8095	0.5350	0.1950
0.7741	25.0553	17000	0.9116	0.5448	0.1998
0.7303	25.7922	17500	0.9380	0.5339	0.1948
0.7132	26.5291	18000	0.8357	0.5143	0.1888
0.6655	27.2660	18500	0.9127	0.5495	0.2049
0.6452	28.0029	19000	0.8722	0.5258	0.1933
0.5913	28.7399	19500	0.9262	0.5227	0.1929
0.5792	29.4768	20000	0.9722	0.5239	0.1883
0.5528	30.2137	20500	0.9868	0.5259	0.1937
0.5488	30.9506	21000	0.9860	0.5268	0.1945
0.5023	31.6875	21500	0.9549	0.5134	0.1874
0.4668	32.4245	22000	1.0188	0.5200	0.1943
0.4751	33.1614	22500	1.0139	0.5112	0.1852
0.434	33.8983	23000	1.0354	0.5073	0.1815
0.4149	34.6352	23500	0.9920	0.5170	0.1874
0.4044	35.3721	24000	1.1387	0.5051	0.1840
0.3839	36.1091	24500	1.1052	0.5034	0.1848
0.3576	36.8460	25000	1.0593	0.4889	0.1811
0.3379	37.5829	25500	1.0930	0.5007	0.1823
0.336	38.3198	26000	1.1091	0.4968	0.1808
0.3148	39.0567	26500	1.1871	0.4993	0.1810
0.3005	39.7937	27000	1.1890	0.4993	0.1801
0.2964	40.5306	27500	1.1436	0.4899	0.1758
0.2731	41.2675	28000	1.1677	0.4940	0.1780
0.2641	42.0044	28500	1.1943	0.4943	0.1783
0.2435	42.7413	29000	1.2838	0.4906	0.1787
0.2486	43.4783	29500	1.2935	0.4938	0.1767
0.2259	44.2152	30000	1.3013	0.4898	0.1749
0.218	44.9521	30500	1.2965	0.4856	0.1749
0.2143	45.6890	31000	1.2891	0.4823	0.1739
0.195	46.4259	31500	1.3284	0.4791	0.1724
0.1943	47.1629	32000	1.3182	0.4805	0.1734
0.1851	47.8998	32500	1.3429	0.4769	0.1719
0.1833	48.6367	33000	1.3515	0.4723	0.1708
0.1739	49.3736	33500	1.3654	0.4749	0.1708

Framework versions

• Transformers 4.46.1
• Pytorch 2.1.0+cu118
• Datasets 2.17.0
• Tokenizers 0.20.3