wav2vec2-large-xls-r-300m-ln-BibleTTS-1hr-v1

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6740
• Wer: 0.7140
• Cer: 0.1802

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 500
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
17.416	0.9091	5	17.5456	1.0	1.5562
14.5068	2.0	11	17.5120	1.0	1.4789
17.3472	2.9091	16	17.4189	1.0	1.1780
14.3268	4.0	22	17.2016	1.0	0.9674
17.0088	4.9091	27	17.0130	1.0	1.0293
13.9609	6.0	33	16.7075	1.0	0.8481
16.4587	6.9091	38	16.4284	1.0	0.8363
13.3164	8.0	44	15.9681	1.0	0.9944
15.3216	8.9091	49	15.3243	1.0	1.0
11.5548	10.0	55	14.0370	1.0	1.0
11.5967	10.9091	60	12.5247	1.0	1.0
8.2049	12.0	66	11.1587	1.0	1.0
8.4152	12.9091	71	10.0872	1.0	1.0
6.0687	14.0	77	8.8445	1.0	1.0
6.4176	14.9091	82	8.1147	1.0	1.0
4.8424	16.0	88	7.4897	1.0	1.0
5.3723	16.9091	93	6.9360	1.0	1.0
4.1999	18.0	99	6.3607	1.0	1.0
4.7866	18.9091	104	5.9263	1.0	1.0
3.8236	20.0	110	5.5028	1.0	1.0
4.4302	20.9091	115	5.0334	1.0	1.0
3.591	22.0	121	4.6869	1.0	1.0
4.1949	22.9091	126	4.4762	1.0	1.0
3.4191	24.0	132	4.2646	1.0	1.0
4.0108	24.9091	137	4.1448	1.0	1.0
3.28	26.0	143	3.9637	1.0	1.0
3.8588	26.9091	148	3.8742	1.0	1.0
3.158	28.0	154	3.7669	1.0	1.0
3.7243	28.9091	159	3.7039	1.0	1.0
3.0502	30.0	165	3.5948	1.0	1.0
3.6	30.9091	170	3.5306	1.0	1.0
2.9515	32.0	176	3.4187	1.0	1.0
3.4903	32.9091	181	3.3450	1.0	1.0
2.8763	34.0	187	3.2715	1.0	1.0
3.3998	34.9091	192	3.2082	1.0	1.0
2.7964	36.0	198	3.2021	1.0	1.0
3.3113	36.9091	203	3.1217	1.0	1.0
2.7362	38.0	209	3.0453	1.0	1.0
3.2274	38.9091	214	3.0072	1.0	1.0
2.6544	40.0	220	2.9557	1.0	1.0
3.1479	40.9091	225	2.9229	1.0	1.0
2.5983	42.0	231	2.8882	1.0	1.0
3.0854	42.9091	236	2.8638	1.0	1.0
2.5509	44.0	242	2.8354	1.0	1.0
3.0388	44.9091	247	2.8169	1.0	1.0
2.517	46.0	253	2.8000	1.0	1.0
3.0042	46.9091	258	2.7845	1.0	1.0
2.4924	48.0	264	2.7746	1.0	1.0
2.9774	48.9091	269	2.7673	1.0	1.0
2.4696	50.0	275	2.7573	1.0	1.0
2.9528	50.9091	280	2.7507	1.0	1.0
2.4516	52.0	286	2.7300	1.0	1.0
2.9192	52.9091	291	2.7201	1.0	1.0
2.4026	54.0	297	2.6965	1.0	1.0
2.8441	54.9091	302	2.6793	1.0	1.0
2.3344	56.0	308	2.6544	1.0	1.0
2.7453	56.9091	313	2.6112	1.0	1.0
2.223	58.0	319	2.5443	1.0	1.0
2.5596	58.9091	324	2.4781	1.0	0.9985
2.0074	60.0	330	2.3645	1.0	0.8402
2.2236	60.9091	335	2.2414	1.0	0.7652
1.703	62.0	341	2.0875	1.0	0.6468
1.8108	62.9091	346	2.0012	1.0	0.6570
1.2854	64.0	352	1.7562	0.9994	0.5339
1.289	64.9091	357	1.5530	0.9984	0.4409
0.8914	66.0	363	1.5710	0.9998	0.5147
0.8887	66.9091	368	1.3824	0.9547	0.4007
0.6132	68.0	374	1.3746	0.9759	0.4337
0.6176	68.9091	379	1.2084	0.9326	0.3478
0.4483	70.0	385	1.3181	0.9215	0.3787
0.4877	70.9091	390	1.1160	0.8970	0.3005
0.3614	72.0	396	1.1371	0.9054	0.3223
0.3889	72.9091	401	1.0581	0.8344	0.2827
0.2835	74.0	407	0.8882	0.7899	0.2292
0.3203	74.9091	412	0.9902	0.8447	0.2651
0.2494	76.0	418	1.0597	0.8338	0.2928
0.2693	76.9091	423	0.9066	0.8115	0.2517
0.2094	78.0	429	0.8811	0.7823	0.2311
0.2332	78.9091	434	0.9556	0.8330	0.2648
0.1874	80.0	440	0.9001	0.8103	0.2374
0.2155	80.9091	445	0.9119	0.8338	0.2499
0.1691	82.0	451	0.8158	0.7254	0.2066
0.1941	82.9091	456	0.8922	0.8010	0.2407
0.1537	84.0	462	0.8880	0.8137	0.2343
0.1801	84.9091	467	0.8078	0.7503	0.2021
0.141	86.0	473	0.7358	0.7340	0.1920
0.161	86.9091	478	0.8496	0.7928	0.2307
0.1317	88.0	484	0.7789	0.7491	0.2017
0.1435	88.9091	489	0.9110	0.8074	0.2435
0.1168	90.0	495	0.7392	0.7249	0.1908
0.1223	90.9091	500	0.7573	0.7177	0.1904

Framework versions

• Transformers 4.42.4
• Pytorch 2.2.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1