wav2vec2-large-xls-r-300m-korean-g

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 16.2060
• Cer: 0.9336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
11.1605	1.11	300	17.8697	1.1350
10.9986	2.22	600	17.7379	1.0831
10.747	3.33	900	17.4821	0.9742
10.82	4.44	1200	17.1100	0.8829
9.7866	5.56	1500	16.7928	0.8976
10.3298	6.67	1800	16.5434	0.9105
10.4446	7.78	2100	16.3516	0.9222
10.0077	8.89	2400	16.2427	0.9305
9.7931	10.0	2700	16.2060	0.9336

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3