mms-1b-all-sw-CV_Fleurs_AMMI_ALFFA-1hrs-v1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.2505
• Cer: 0.0866

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
16.1327	1.0	37	inf	1.1533	2.6305
14.5509	2.0	74	inf	1.4415	1.7715
11.4505	3.0	111	inf	1.1493	0.8650
7.3358	4.0	148	inf	1.0074	0.8544
4.2752	5.0	185	inf	1.0009	0.8517
3.2479	6.0	222	inf	1.0628	0.6094
2.322	7.0	259	inf	0.7326	0.2337
1.5058	8.0	296	inf	0.4135	0.1305
1.0304	9.0	333	inf	0.3108	0.1034
0.8109	10.0	370	inf	0.2843	0.0967
0.7432	11.0	407	inf	0.2771	0.0948
0.6892	12.0	444	inf	0.2725	0.0935
0.6813	13.0	481	inf	0.2694	0.0928
0.6618	14.0	518	inf	0.2683	0.0925
0.6347	15.0	555	inf	0.2661	0.0917
0.6194	16.0	592	inf	0.2648	0.0913
0.6234	17.0	629	inf	0.2643	0.0909
0.6042	18.0	666	inf	0.2610	0.0904
0.6092	19.0	703	inf	0.2608	0.0900
0.6032	20.0	740	inf	0.2588	0.0896
0.5886	21.0	777	inf	0.2578	0.0892
0.5548	22.0	814	inf	0.2559	0.0888
0.5701	23.0	851	inf	0.2557	0.0887
0.591	24.0	888	inf	0.2556	0.0885
0.5481	25.0	925	inf	0.2545	0.0882
0.5437	26.0	962	inf	0.2541	0.0879
0.537	27.0	999	inf	0.2531	0.0877
0.555	28.0	1036	inf	0.2526	0.0875
0.5391	29.0	1073	inf	0.2518	0.0873
0.531	30.0	1110	inf	0.2523	0.0873
0.5248	31.0	1147	inf	0.2517	0.0872
0.5125	32.0	1184	inf	0.2514	0.0871
0.5133	33.0	1221	inf	0.2513	0.0870
0.4936	34.0	1258	inf	0.2519	0.0870
0.5095	35.0	1295	inf	0.2514	0.0868
0.5121	36.0	1332	inf	0.2509	0.0867
0.4983	37.0	1369	inf	0.2505	0.0866

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0