w2v-bert-2.0-Fleurs_AMMI_AFRIVOICE_LRSC-ln-20hrs-v2

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7994
• Wer: 0.2042
• Cer: 0.0647

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 80
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.2265	1.0	356	0.4952	0.3304	0.1008
0.5303	2.0	712	0.3928	0.2891	0.0862
0.4278	3.0	1068	0.3568	0.2574	0.0769
0.3675	4.0	1424	0.3322	0.2385	0.0735
0.3271	5.0	1780	0.3118	0.2411	0.0776
0.2848	6.0	2136	0.3208	0.2092	0.0659
0.2556	7.0	2492	0.3289	0.2090	0.0658
0.2265	8.0	2848	0.3280	0.2324	0.0812
0.2043	9.0	3204	0.3279	0.2206	0.0700
0.1865	10.0	3560	0.3551	0.2132	0.0676
0.1658	11.0	3916	0.3360	0.2108	0.0697
0.1521	12.0	4272	0.3539	0.2347	0.0785
0.1408	13.0	4628	0.3642	0.2124	0.0708
0.128	14.0	4984	0.3877	0.2141	0.0689
0.1159	15.0	5340	0.4239	0.1999	0.0635
0.1073	16.0	5696	0.4337	0.2142	0.0700
0.0975	17.0	6052	0.4401	0.2022	0.0653
0.0899	18.0	6408	0.4767	0.2101	0.0662
0.0857	19.0	6764	0.4927	0.2041	0.0677
0.0783	20.0	7120	0.5006	0.2091	0.0677
0.0714	21.0	7476	0.5030	0.2140	0.0709
0.0667	22.0	7832	0.5110	0.2144	0.0691
0.0665	23.0	8188	0.5129	0.2136	0.0669
0.0586	24.0	8544	0.5906	0.1980	0.0632
0.0527	25.0	8900	0.6139	0.2019	0.0631
0.0471	26.0	9256	0.5897	0.2127	0.0677
0.0425	27.0	9612	0.6123	0.2037	0.0655
0.0374	28.0	9968	0.5954	0.2154	0.0697
0.035	29.0	10324	0.6249	0.2017	0.0634
0.0311	30.0	10680	0.6401	0.2056	0.0649
0.0274	31.0	11036	0.6688	0.2105	0.0668
0.0243	32.0	11392	0.6783	0.2122	0.0661
0.0223	33.0	11748	0.6974	0.2062	0.0652
0.0188	34.0	12104	0.7393	0.2069	0.0662
0.0177	35.0	12460	0.7598	0.2065	0.0645
0.0168	36.0	12816	0.7584	0.2058	0.0640
0.0137	37.0	13172	0.8009	0.2052	0.0645
0.0122	38.0	13528	0.7671	0.2045	0.0644
0.0105	39.0	13884	0.7994	0.2042	0.0647

Framework versions

• Transformers 4.46.3
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.3