wav2vec2-large-xls-r-300m-lg-1hr-v2

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.6799
• Cer: 0.1655

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 60
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
11.4137	1.4085	100	inf	1.0	1.0
3.6423	2.8169	200	inf	1.0	1.0
3.0132	4.2254	300	inf	1.0	1.0
2.897	5.6338	400	inf	1.0	1.0
2.1486	7.0423	500	inf	0.9473	0.2781
0.9601	8.4507	600	inf	0.8429	0.2259
0.629	9.8592	700	inf	0.7806	0.2010
0.439	11.2676	800	inf	0.7712	0.1977
0.3285	12.6761	900	inf	0.7742	0.1980
0.274	14.0845	1000	inf	0.7378	0.1893
0.2205	15.4930	1100	inf	0.7700	0.1980
0.2006	16.9014	1200	inf	0.7410	0.1855
0.1582	18.3099	1300	inf	0.7450	0.1876
0.1403	19.7183	1400	inf	0.7339	0.1843
0.1315	21.1268	1500	inf	0.7442	0.1856
0.1207	22.5352	1600	inf	0.7329	0.1824
0.1158	23.9437	1700	inf	0.7351	0.1823
0.1094	25.3521	1800	inf	0.7314	0.1816
0.0987	26.7606	1900	inf	0.7138	0.1787
0.0918	28.1690	2000	inf	0.7393	0.1797
0.091	29.5775	2100	inf	0.7450	0.1844
0.0821	30.9859	2200	inf	0.7153	0.1807
0.0876	32.3944	2300	inf	0.7012	0.1722
0.0792	33.8028	2400	inf	0.7175	0.1742
0.071	35.2113	2500	inf	0.7168	0.1767
0.0705	36.6197	2600	inf	0.7054	0.1704
0.0681	38.0282	2700	inf	0.7111	0.1724
0.0619	39.4366	2800	inf	0.7086	0.1737
0.06	40.8451	2900	inf	0.7331	0.1791
0.0596	42.2535	3000	inf	0.7012	0.1703
0.0527	43.6620	3100	inf	0.7044	0.1723
0.0592	45.0704	3200	inf	0.6948	0.1710
0.0496	46.4789	3300	inf	0.6975	0.1710
0.0452	47.8873	3400	inf	0.6987	0.1706
0.0488	49.2958	3500	inf	0.6918	0.1691
0.0449	50.7042	3600	inf	0.6869	0.1685
0.0419	52.1127	3700	inf	0.6827	0.1665
0.0384	53.5211	3800	inf	0.6824	0.1670
0.0432	54.9296	3900	inf	0.6817	0.1658
0.0376	56.3380	4000	inf	0.6785	0.1659
0.0395	57.7465	4100	inf	0.6809	0.1659
0.0437	59.1549	4200	inf	0.6812	0.1668

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1