w2v-bert-2.0-CV_Fleurs-lg-20hrs-v5

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4606
• Wer: 0.3643
• Cer: 0.0782

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 80
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.4915	1.0	2058	0.3507	0.4166	0.0815
0.3195	2.0	4116	0.3251	0.3885	0.0772
0.2817	3.0	6174	0.3035	0.3921	0.0787
0.27	4.0	8232	0.3337	0.4144	0.0824
0.2645	5.0	10290	0.3604	0.4144	0.0849
0.2579	6.0	12348	0.3396	0.4502	0.0933
0.2609	7.0	14406	0.3439	0.3976	0.0830
0.2557	8.0	16464	0.3807	0.4361	0.0953
0.242	9.0	18522	0.3477	0.3997	0.0841
0.2198	10.0	20580	0.3354	0.3986	0.0845
0.1912	11.0	22638	0.3337	0.3951	0.0837
0.1716	12.0	24696	0.3179	0.3646	0.0779
0.1566	13.0	26754	0.3486	0.3747	0.0797
0.1422	14.0	28812	0.3320	0.3838	0.0808
0.1284	15.0	30870	0.3482	0.3668	0.0807
0.1142	16.0	32928	0.3330	0.3721	0.0780
0.1005	17.0	34986	0.3272	0.3539	0.0738
0.0897	18.0	37044	0.3906	0.3732	0.0763
0.0787	19.0	39102	0.3827	0.3597	0.0755
0.0697	20.0	41160	0.3883	0.3586	0.0770
0.0632	21.0	43218	0.3923	0.3798	0.0797
0.0544	22.0	45276	0.4401	0.3689	0.0803
0.0503	23.0	47334	0.4111	0.3704	0.0790
0.0438	24.0	49392	0.4019	0.3599	0.0762
0.0392	25.0	51450	0.4198	0.3625	0.0774
0.0372	26.0	53508	0.4374	0.3650	0.0794
0.0333	27.0	55566	0.4606	0.3643	0.0782

Framework versions

• Transformers 4.46.1
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.1