w2v2_bert-Wolof-10-hours-kallaama-dataset

This model is a fine-tuned version of facebook/w2v-bert-2.0 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.8806
• Wer: 0.4798
• Cer: 0.2414

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.3424	2.9091	400	1.7309	0.7395	0.3439
1.768	5.8182	800	1.8090	0.8050	0.4380
1.6715	8.7273	1200	1.7011	0.7347	0.3999
1.4881	11.6364	1600	1.5775	0.6741	0.3379
1.3706	14.5455	2000	1.4897	0.6109	0.3018
1.2601	17.4545	2400	1.5302	0.6046	0.3022
1.1786	20.3636	2800	1.4589	0.5783	0.2958
1.062	23.2727	3200	1.3950	0.6128	0.3097
0.9823	26.1818	3600	1.5242	0.5797	0.2872
0.9022	29.0909	4000	1.5189	0.5256	0.2630
0.8182	32.0	4400	1.4356	0.5787	0.3096
0.733	34.9091	4800	1.4985	0.5232	0.2601
0.6641	37.8182	5200	1.4687	0.5023	0.2586
0.5742	40.7273	5600	1.7385	0.4952	0.2456
0.5221	43.6364	6000	1.4670	0.4990	0.2596
0.4468	46.5455	6400	1.7551	0.4868	0.2429
0.3835	49.4545	6800	1.8806	0.4798	0.2414

Framework versions

• Transformers 4.44.1
• Pytorch 2.1.0+cu118
• Datasets 2.17.0
• Tokenizers 0.19.1