nhi_heldout-speaker-exp_GGN505_mms-1b-nhi-adapterft

This model is a fine-tuned version of facebook/mms-1b-all on the audiofolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5168
• Wer: 0.3564
• Cer: 0.0963

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.0628	1.3072	200	0.6601	0.6203	0.1672
0.8658	2.6144	400	0.5592	0.5526	0.1436
0.7815	3.9216	600	0.5179	0.5286	0.1345
0.7157	5.2288	800	0.5074	0.5108	0.1281
0.6834	6.5359	1000	0.4945	0.4801	0.1259
0.6746	7.8431	1200	0.4748	0.4699	0.1229
0.6337	9.1503	1400	0.4826	0.4655	0.1198
0.6218	10.4575	1600	0.4772	0.4584	0.1205
0.5927	11.7647	1800	0.4740	0.4490	0.1155
0.5927	13.0719	2000	0.4637	0.4293	0.1121
0.5776	14.3791	2200	0.4621	0.4423	0.1153
0.5432	15.6863	2400	0.4752	0.4289	0.1131
0.5259	16.9935	2600	0.4516	0.3966	0.1059
0.5152	18.3007	2800	0.4510	0.4210	0.1109
0.4901	19.6078	3000	0.4602	0.4191	0.1095
0.4926	20.9150	3200	0.4607	0.4057	0.1068
0.4742	22.2222	3400	0.4569	0.3856	0.1037
0.4813	23.5294	3600	0.4538	0.4163	0.1080
0.4598	24.8366	3800	0.4672	0.4147	0.1098
0.4418	26.1438	4000	0.4656	0.4013	0.1054
0.4561	27.4510	4200	0.4737	0.4002	0.1054
0.4358	28.7582	4400	0.4560	0.3931	0.1059
0.4343	30.0654	4600	0.4644	0.3954	0.1062
0.4148	31.3725	4800	0.4510	0.3966	0.1072
0.4208	32.6797	5000	0.4687	0.3840	0.1026
0.4208	33.9869	5200	0.4805	0.3856	0.1062
0.4189	35.2941	5400	0.4624	0.3765	0.1032
0.3899	36.6013	5600	0.4741	0.3844	0.1051
0.3835	37.9085	5800	0.4721	0.3876	0.1040
0.4017	39.2157	6000	0.4733	0.3915	0.1056
0.3928	40.5229	6200	0.4644	0.3742	0.1029
0.373	41.8301	6400	0.4628	0.3848	0.1027
0.372	43.1373	6600	0.4805	0.3899	0.1034
0.3473	44.4444	6800	0.4637	0.3769	0.1012
0.3419	45.7516	7000	0.4687	0.3753	0.1008
0.3607	47.0588	7200	0.4642	0.3777	0.1007
0.3474	48.3660	7400	0.4610	0.3690	0.0989
0.3464	49.6732	7600	0.4631	0.3757	0.1001
0.3398	50.9804	7800	0.4571	0.3588	0.0982
0.3182	52.2876	8000	0.4868	0.3659	0.0999
0.3158	53.5948	8200	0.4821	0.3718	0.1004
0.3368	54.9020	8400	0.4712	0.3777	0.1015
0.3312	56.2092	8600	0.4918	0.3820	0.1022
0.3175	57.5163	8800	0.4969	0.3761	0.1023
0.3081	58.8235	9000	0.4717	0.3742	0.0996
0.3144	60.1307	9200	0.4901	0.3753	0.1029
0.3085	61.4379	9400	0.4793	0.3655	0.0992
0.3033	62.7451	9600	0.4726	0.3623	0.0985
0.291	64.0523	9800	0.4792	0.3750	0.1015
0.3022	65.3595	10000	0.4942	0.3761	0.1018
0.2949	66.6667	10200	0.5000	0.3809	0.1028
0.2744	67.9739	10400	0.5011	0.3773	0.1010
0.2648	69.2810	10600	0.5171	0.3809	0.1028
0.2793	70.5882	10800	0.5050	0.3738	0.1004
0.2664	71.8954	11000	0.4973	0.3663	0.1000
0.2515	73.2026	11200	0.5010	0.3675	0.1004
0.2421	74.5098	11400	0.5130	0.3560	0.0973
0.2638	75.8170	11600	0.5044	0.3679	0.0995
0.2444	77.1242	11800	0.4933	0.3627	0.0986
0.2399	78.4314	12000	0.4950	0.3620	0.0984
0.2532	79.7386	12200	0.4971	0.3588	0.0974
0.242	81.0458	12400	0.5043	0.3718	0.1012
0.2351	82.3529	12600	0.5112	0.3722	0.0990
0.2344	83.6601	12800	0.4991	0.3651	0.0986
0.2274	84.9673	13000	0.5089	0.3533	0.0959
0.2394	86.2745	13200	0.5069	0.3588	0.0973
0.2336	87.5817	13400	0.5152	0.3631	0.0983
0.2323	88.8889	13600	0.5168	0.3600	0.0975
0.2427	90.1961	13800	0.5017	0.3620	0.0979
0.2296	91.5033	14000	0.5121	0.3596	0.0975
0.2289	92.8105	14200	0.5106	0.3545	0.0956
0.2153	94.1176	14400	0.5133	0.3584	0.0959
0.244	95.4248	14600	0.5134	0.3553	0.0959
0.2277	96.7320	14800	0.5166	0.3580	0.0964
0.2224	98.0392	15000	0.5136	0.3568	0.0963
0.2218	99.3464	15200	0.5168	0.3564	0.0963

Framework versions

• Transformers 4.41.2
• Pytorch 2.4.0
• Datasets 3.2.0
• Tokenizers 0.19.1