asr-africa/mms-1B_all_BIG_C_Bemba_100hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the BIG_C dataset. It achieves the following results on the evaluation set:

• Loss: 0.3475
• Model Preparation Time: 0.0112
• Wer: 0.4171
• Cer: 0.0777

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
0.9189	1.0	3096	0.5510	0.0112	0.4860	0.1248
0.5994	2.0	6192	0.5258	0.0112	0.4870	0.1264
0.5721	3.0	9288	0.5079	0.0112	0.4638	0.1264
0.5573	4.0	12384	0.4963	0.0112	0.4410	0.1128
0.5442	5.0	15480	0.4938	0.0112	0.4449	0.1202
0.5347	6.0	18576	0.4837	0.0112	0.4348	0.1165
0.5261	7.0	21672	0.4795	0.0112	0.4205	0.1092
0.5203	8.0	24768	0.4791	0.0112	0.4237	0.1076
0.5132	9.0	27864	0.4745	0.0112	0.4159	0.1073
0.5073	10.0	30960	0.4696	0.0112	0.4162	0.1072
0.5037	11.0	34056	0.4696	0.0112	0.4227	0.1136
0.4974	12.0	37152	0.4702	0.0112	0.4042	0.1047
0.4924	13.0	40248	0.4761	0.0112	0.3916	0.1028
0.4873	14.0	43344	0.4617	0.0112	0.3977	0.1046
0.4846	15.0	46440	0.4756	0.0112	0.3926	0.1025
0.4798	16.0	49536	0.4614	0.0112	0.3980	0.1066
0.4767	17.0	52632	0.4606	0.0112	0.3940	0.1031
0.473	18.0	55728	0.4712	0.0112	0.3919	0.1030
0.4685	19.0	58824	0.4586	0.0112	0.3913	0.1032
0.4656	20.0	61920	0.4713	0.0112	0.3892	0.1034
0.4618	21.0	65016	0.4627	0.0112	0.3854	0.1020
0.4577	22.0	68112	0.4565	0.0112	0.3880	0.1015
0.4554	23.0	71208	0.4593	0.0112	0.3862	0.1034
0.4523	24.0	74304	0.4624	0.0112	0.3818	0.1015
0.4486	25.0	77400	0.4638	0.0112	0.3855	0.1024
0.446	26.0	80496	0.4574	0.0112	0.3929	0.1029
0.4426	27.0	83592	0.4662	0.0112	0.3851	0.1033
0.4407	28.0	86688	0.4633	0.0112	0.3898	0.1032
0.4378	29.0	89784	0.4614	0.0112	0.3800	0.1017
0.4337	30.0	92880	0.4595	0.0112	0.3809	0.1009
0.4312	31.0	95976	0.4681	0.0112	0.3841	0.1046
0.4277	32.0	99072	0.4575	0.0112	0.3749	0.0999
0.4257	33.0	102168	0.4583	0.0112	0.3760	0.1021
0.4224	34.0	105264	0.4566	0.0112	0.3830	0.1012
0.4205	35.0	108360	0.4718	0.0112	0.3698	0.1004
0.4159	36.0	111456	0.4642	0.0112	0.3734	0.1008
0.4141	37.0	114552	0.4601	0.0112	0.3802	0.1024
0.4118	38.0	117648	0.4665	0.0112	0.3718	0.1004
0.408	39.0	120744	0.4665	0.0112	0.3745	0.1041
0.4062	40.0	123840	0.4743	0.0112	0.3708	0.0994
0.4033	41.0	126936	0.4700	0.0112	0.3720	0.1003
0.4017	42.0	130032	0.4755	0.0112	0.3841	0.1057
0.3982	43.0	133128	0.4798	0.0112	0.3671	0.0984
0.3966	44.0	136224	0.4738	0.0112	0.3752	0.1010
0.3944	45.0	139320	0.4749	0.0112	0.3727	0.1010
0.3923	46.0	142416	0.4702	0.0112	0.3792	0.1033
0.389	47.0	145512	0.4723	0.0112	0.3714	0.1014
0.3874	48.0	148608	0.4750	0.0112	0.3733	0.1004
0.3851	49.0	151704	0.4708	0.0112	0.3747	0.1024
0.3831	50.0	154800	0.4768	0.0112	0.3688	0.0996
0.3803	51.0	157896	0.4807	0.0112	0.3673	0.0996
0.379	52.0	160992	0.4823	0.0112	0.3648	0.0991
0.3772	53.0	164088	0.4809	0.0112	0.3723	0.1002
0.3741	54.0	167184	0.4807	0.0112	0.3705	0.0996
0.3726	55.0	170280	0.4866	0.0112	0.3669	0.0990
0.371	56.0	173376	0.4793	0.0112	0.3738	0.1027
0.3695	57.0	176472	0.4789	0.0112	0.3768	0.1025
0.3668	58.0	179568	0.4774	0.0112	0.3681	0.1015
0.3656	59.0	182664	0.4783	0.0112	0.3684	0.1000
0.3639	60.0	185760	0.4848	0.0112	0.3714	0.1002
0.3615	61.0	188856	0.4893	0.0112	0.3721	0.0996
0.3596	62.0	191952	0.4808	0.0112	0.3654	0.1008

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.1