mms-1b-all-sw-CV_Fleurs_AMMI_ALFFA-10hrs-v1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.2298
• Cer: 0.0792

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
6.1301	1.0	367	inf	0.3014	0.1016
0.706	2.0	734	inf	0.2576	0.0893
0.6107	3.0	1101	inf	0.2494	0.0863
0.5664	4.0	1468	inf	0.2467	0.0848
0.5353	5.0	1835	inf	0.2430	0.0836
0.5086	6.0	2202	inf	0.2414	0.0830
0.4963	7.0	2569	inf	0.2404	0.0822
0.4773	8.0	2936	inf	0.2391	0.0817
0.4619	9.0	3303	inf	0.2396	0.0814
0.4566	10.0	3670	inf	0.2379	0.0810
0.4474	11.0	4037	inf	0.2376	0.0807
0.4356	12.0	4404	inf	0.2374	0.0804
0.4331	13.0	4771	inf	0.2361	0.0801
0.4297	14.0	5138	inf	0.2364	0.0800
0.4219	15.0	5505	inf	0.2356	0.0798
0.4067	16.0	5872	inf	0.2341	0.0793
0.4158	17.0	6239	inf	0.2348	0.0792
0.4071	18.0	6606	inf	0.2337	0.0788
0.4018	19.0	6973	inf	0.2334	0.0787
0.3907	20.0	7340	inf	0.2329	0.0787
0.3937	21.0	7707	inf	0.2318	0.0784
0.3895	22.0	8074	inf	0.2329	0.0787
0.3851	23.0	8441	inf	0.2322	0.0784
0.3844	24.0	8808	inf	0.2313	0.0780
0.3799	25.0	9175	inf	0.2333	0.0791
0.3754	26.0	9542	inf	0.2344	0.0794
0.3733	27.0	9909	inf	0.2290	0.0779
0.3761	28.0	10276	inf	0.2321	0.0793
0.3678	29.0	10643	inf	0.2329	0.0789
0.3639	30.0	11010	inf	0.2327	0.0795
0.361	31.0	11377	inf	0.2313	0.0791
0.3559	32.0	11744	inf	0.2303	0.0788
0.3596	33.0	12111	inf	0.2310	0.0797
0.358	34.0	12478	inf	0.2422	0.0874
0.3501	35.0	12845	inf	0.2309	0.0802
0.3533	36.0	13212	inf	0.2293	0.0793
0.349	37.0	13579	inf	0.2298	0.0792

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0