asr-africa/mms-1B_all_BIG_C_Bemba_50hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the BIG_C dataset. It achieves the following results on the evaluation set:

• Loss: 0.3664
• Model Preparation Time: 0.0118
• Wer: 0.4282
• Cer: 0.0810

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
1.1174	1.0	1544	0.6021	0.0118	0.5160	0.1324
0.6138	2.0	3088	0.5427	0.0118	0.4778	0.1229
0.5739	3.0	4632	0.5287	0.0118	0.4618	0.1194
0.5536	4.0	6176	0.5322	0.0118	0.4502	0.1169
0.5399	5.0	7720	0.5151	0.0118	0.4503	0.1168
0.5291	6.0	9264	0.5201	0.0118	0.4560	0.1224
0.52	7.0	10808	0.5058	0.0118	0.4622	0.1198
0.5133	8.0	12352	0.5017	0.0118	0.4466	0.1175
0.5054	9.0	13896	0.4975	0.0118	0.4438	0.1134
0.4994	10.0	15440	0.4987	0.0118	0.4373	0.1138
0.4933	11.0	16984	0.4932	0.0118	0.4211	0.1098
0.4868	12.0	18528	0.4977	0.0118	0.4460	0.1193
0.4827	13.0	20072	0.4907	0.0118	0.4229	0.1110
0.4767	14.0	21616	0.4876	0.0118	0.4169	0.1088
0.4712	15.0	23160	0.4875	0.0118	0.4306	0.1149
0.4714	16.0	24704	0.4862	0.0118	0.4191	0.1084
0.4631	17.0	26248	0.4873	0.0118	0.4171	0.1114
0.4578	18.0	27792	0.4848	0.0118	0.4153	0.1114
0.4535	19.0	29336	0.4789	0.0118	0.4105	0.1094
0.4491	20.0	30880	0.4874	0.0118	0.4301	0.1145
0.4453	21.0	32424	0.4847	0.0118	0.4174	0.1092
0.4395	22.0	33968	0.4861	0.0118	0.4080	0.1061
0.4345	23.0	35512	0.4903	0.0118	0.4021	0.1055
0.4307	24.0	37056	0.4919	0.0118	0.4115	0.1097
0.4261	25.0	38600	0.4820	0.0118	0.4036	0.1082
0.4218	26.0	40144	0.4921	0.0118	0.4101	0.1107
0.4198	27.0	41688	0.4892	0.0118	0.4068	0.1097
0.4149	28.0	43232	0.4898	0.0118	0.4070	0.1090
0.4097	29.0	44776	0.4870	0.0118	0.3914	0.1039
0.4061	30.0	46320	0.4886	0.0118	0.4029	0.1105
0.4027	31.0	47864	0.4872	0.0118	0.4058	0.1071
0.4002	32.0	49408	0.5048	0.0118	0.4004	0.1045
0.3957	33.0	50952	0.4955	0.0118	0.3950	0.1040
0.3935	34.0	52496	0.4999	0.0118	0.4083	0.1127
0.3906	35.0	54040	0.4966	0.0118	0.4075	0.1082
0.3867	36.0	55584	0.4977	0.0118	0.4169	0.1173
0.3837	37.0	57128	0.4920	0.0118	0.3964	0.1042
0.3795	38.0	58672	0.4911	0.0118	0.3938	0.1060
0.3769	39.0	60216	0.5098	0.0118	0.3870	0.1023
0.3745	40.0	61760	0.5026	0.0118	0.3926	0.1053
0.3719	41.0	63304	0.4950	0.0118	0.3979	0.1064
0.3685	42.0	64848	0.5065	0.0118	0.3902	0.1036
0.3654	43.0	66392	0.4997	0.0118	0.3933	0.1075
0.3624	44.0	67936	0.5080	0.0118	0.3856	0.1021
0.3612	45.0	69480	0.4999	0.0118	0.3920	0.1057
0.3583	46.0	71024	0.5161	0.0118	0.3823	0.1019
0.3548	47.0	72568	0.5025	0.0118	0.3877	0.1036
0.3528	48.0	74112	0.5079	0.0118	0.3928	0.1052
0.3489	49.0	75656	0.5063	0.0118	0.3956	0.1048
0.3475	50.0	77200	0.5052	0.0118	0.3862	0.1032
0.3454	51.0	78744	0.5066	0.0118	0.3847	0.1024
0.3441	52.0	80288	0.5166	0.0118	0.3848	0.1028
0.3412	53.0	81832	0.5055	0.0118	0.3895	0.1041
0.3399	54.0	83376	0.5160	0.0118	0.3871	0.1030
0.3365	55.0	84920	0.5082	0.0118	0.3881	0.1042
0.3345	56.0	86464	0.5137	0.0118	0.3873	0.1044

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.1