wav2vec2-xls-r-1b-faroese-100h-5k-steps

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2200
• Wer: 38.3783
• Cer: 11.6507

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.006	0.4640	1000	0.6140	71.3811	22.6529
0.7219	0.9281	2000	0.4564	60.0262	18.6149
0.6682	1.3921	3000	0.3286	47.9666	14.9048
0.3491	1.8561	4000	0.2586	42.2237	12.8636
0.2501	2.3202	5000	0.2200	38.3783	11.6507

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0