bach-arb

This model is a fine-tuned version of jonatasgrosman/wav2vec2-large-xlsr-53-german on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.9404
• Wer: 0.6130

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 115
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
27.8653	7.14	100	3.1369	1.0
2.5975	14.28	200	2.1223	0.9976
1.2001	21.41	300	1.7455	0.8774
0.5938	28.55	400	1.8534	0.7981
0.4001	35.69	500	2.3318	0.7740
0.2895	42.83	600	2.2214	0.7163
0.1853	49.97	700	2.4841	0.7043
0.1318	57.14	800	2.9749	0.7139
0.1067	64.28	900	2.4759	0.7115
0.0635	71.41	1000	2.6708	0.6635
0.0515	78.55	1100	3.0593	0.6923
0.0455	85.69	1200	2.9637	0.6587
0.0329	92.83	1300	2.9837	0.6346
0.0232	99.97	1400	2.9361	0.6178
0.021	107.14	1500	2.9221	0.6010
0.0193	114.28	1600	2.9404	0.6130

Framework versions

• Transformers 4.11.3
• Pytorch 1.10.0+cu113
• Datasets 1.18.3
• Tokenizers 0.10.3