wav2vec2-large-xls-r-300m-finetune-dali

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.2332
• Wer: 0.7088

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
9.1246	0.49	100	4.0976	1.0
4.5372	0.97	200	3.2580	1.0
3.2467	1.46	300	3.0922	1.0001
3.4683	1.94	400	2.7944	0.9588
2.56	2.43	500	2.7701	0.9228
3.5665	2.91	600	2.7017	0.9356
3.5163	3.4	700	2.6731	0.9019
2.7201	3.88	800	2.7024	0.9067
3.1927	4.37	900	2.7681	0.9083
2.6796	4.85	1000	2.6577	0.8902
2.7204	5.34	1100	2.5810	0.8899
2.8474	5.83	1200	2.6795	0.9008
3.4242	6.31	1300	2.5315	0.8699
2.6685	6.8	1400	2.6477	0.8743
2.8734	7.28	1500	2.6630	0.8772
3.0146	7.77	1600	2.5337	0.8667
2.1542	8.25	1700	2.5623	0.8345
2.8927	8.74	1800	2.4624	0.8185
2.4501	9.22	1900	2.5193	0.8196
2.5283	9.71	2000	2.5110	0.8231
2.7019	10.19	2100	2.5571	0.7776
1.8019	10.68	2200	2.4725	0.7735
1.8982	11.17	2300	2.5294	0.7662
1.553	11.65	2400	2.5773	0.7662
1.6364	12.14	2500	2.5883	0.7568
1.9175	12.62	2600	2.4496	0.7405
1.5186	13.11	2700	2.4917	0.7410
1.911	13.59	2800	2.4941	0.7280
1.389	14.08	2900	2.4738	0.7187
1.2499	14.56	3000	2.5277	0.7236
1.3069	15.05	3100	2.5051	0.7291
1.1218	15.53	3200	2.6532	0.7207
1.2423	16.02	3300	2.5690	0.7197
1.0828	16.5	3400	2.6145	0.7216
1.0926	16.99	3500	2.5524	0.7114
1.0012	17.48	3600	2.5685	0.7108
0.9849	17.96	3700	2.6362	0.7081
0.9141	18.45	3800	2.6395	0.7151
0.9115	18.93	3900	2.6901	0.7051
0.8355	19.42	4000	2.6799	0.7150
0.8134	19.9	4100	2.8110	0.7091
0.7593	20.39	4200	2.8204	0.7123
0.7528	20.87	4300	2.8369	0.7067
0.6899	21.36	4400	2.7883	0.7138
0.6799	21.84	4500	2.8745	0.7088
0.6268	22.33	4600	2.9128	0.7137
0.5877	22.82	4700	2.9422	0.7111
0.5789	23.3	4800	2.9755	0.7143
0.5338	23.79	4900	2.9797	0.7115
0.5143	24.27	5000	3.0170	0.7230
0.5019	24.76	5100	3.0441	0.7161
0.4791	25.24	5200	3.1360	0.7084
0.4557	25.73	5300	3.1516	0.7090
0.4293	26.21	5400	3.1535	0.7195
0.4116	26.7	5500	3.1685	0.7177
0.4108	27.18	5600	3.1756	0.7081
0.4012	27.67	5700	3.1828	0.7093
0.3753	28.16	5800	3.2102	0.7057
0.3605	28.64	5900	3.2137	0.7044
0.3678	29.13	6000	3.2202	0.7107
0.3541	29.61	6100	3.2332	0.7088

Framework versions

• Transformers 4.38.1
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.15.2