xls-r-1b-bem-genbed-f-model

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the GENBED - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.3137
• Wer: 0.5529

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.2740	100	3.0378	1.0
No log	0.5479	200	0.8302	0.9818
No log	0.8219	300	0.6783	0.9103
No log	1.0959	400	0.5512	0.8721
1.8782	1.3699	500	0.5296	0.8568
1.8782	1.6438	600	0.4413	0.7333
1.8782	1.9178	700	0.4747	0.7614
1.8782	2.1918	800	0.3884	0.6667
1.8782	2.4658	900	0.3577	0.6355
0.5114	2.7397	1000	0.3585	0.6321
0.5114	3.0137	1100	0.3641	0.6607
0.5114	3.2877	1200	0.3813	0.7282
0.5114	3.5616	1300	0.3829	0.7086
0.5114	3.8356	1400	0.3682	0.6413
0.3931	4.1096	1500	0.3527	0.6221
0.3931	4.3836	1600	0.3481	0.6297
0.3931	4.6575	1700	0.3541	0.6193
0.3931	4.9315	1800	0.3355	0.6242
0.3931	5.2055	1900	0.3339	0.5801
0.3293	5.4795	2000	0.3137	0.5529
0.3293	5.7534	2100	0.3132	0.5822
0.3293	6.0274	2200	0.3145	0.5676
0.3293	6.3014	2300	0.3283	0.5961
0.3293	6.5753	2400	0.3247	0.5988

Framework versions

• Transformers 4.46.0.dev0
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0