whisper-small-zhTW-miltilang-test-4090

This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1771
• Cer: 34.9612
• Wer: 40.1830
• Both Er: 38.1632

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 12000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer	Wer	Both Er
0.299	0.5340	1000	0.2359	37.8038	55.5369	48.6777
0.139	1.0681	2000	0.1909	33.7847	49.9751	43.7126
0.1452	1.6021	3000	0.1768	31.8054	47.2183	41.2565
0.0417	2.1362	4000	0.1662	29.6447	40.6133	36.3706
0.0476	2.6702	5000	0.1564	28.2215	41.5191	36.3755
0.0207	3.2043	6000	0.1629	33.8760	44.8345	40.5957
0.0194	3.7383	7000	0.1610	31.7575	39.8086	36.6944
0.0061	4.2724	8000	0.1675	31.8979	40.4427	37.1375
0.008	4.8064	9000	0.1683	32.3771	38.3019	36.0101
0.0025	5.3405	10000	0.1760	35.2427	44.1596	40.7105
0.0031	5.8745	11000	0.1754	37.8861	43.1949	41.1414
0.0019	6.4085	12000	0.1771	34.9612	40.1830	38.1632

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1