whisper-small-ga2en-v3.2-r

This model is a fine-tuned version of openai/whisper-small on the IWSLT-2023, FLEURS, BiteSize, SpokenWords, Tatoeba, and Wikimedia datasets. The best model checkpoint (this version) based on ChrF is at step 2700, epoch 3.5433, and it achieves the following results on the evaluation set:

• Loss: 1.4313
• Bleu: 30.87
• Chrf: 47.72
• Wer: 64.2954

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Hardware

1 NVIDIA A100-SXM4-80GB

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 3000
• mixed_precision_training: Native AMP

Training results

Step	Training Loss	Validation Loss	Bleu	Chrf	Wer
100	2.311900	1.773697	9.20	28.23	120.486267
200	1.870000	1.479052	16.90	33.01	83.701036
300	1.627700	1.372679	20.33	38.68	84.061234
400	1.460400	1.309611	24.52	40.37	74.696083
500	1.336300	1.283173	20.35	40.77	86.537596
600	1.193300	1.255632	20.63	41.37	95.632598
700	1.015600	1.251285	21.24	41.42	82.170194
800	0.518300	1.292586	28.08	44.76	66.951824
900	0.467300	1.329429	25.16	42.93	76.316974
1000	0.438700	1.330984	28.29	46.08	67.672220
1100	0.403600	1.300828	27.43	46.32	68.977938
1200	0.379500	1.323791	30.02	45.48	63.800090
1300	0.337100	1.327949	30.40	47.61	61.999100
1400	0.288000	1.359497	28.13	44.60	66.501576
1500	0.265100	1.355470	26.58	45.51	71.319226
1600	0.100800	1.400149	26.19	46.02	72.985142
1700	0.092300	1.383455	24.83	46.18	77.532643
1800	0.103900	1.404863	22.19	43.19	88.743809
1900	0.090100	1.402833	29.73	45.85	66.186403
2000	0.084200	1.418717	28.18	45.29	73.570464
2100	0.074800	1.461650	26.58	44.66	74.020711
2200	0.072000	1.400547	31.01	47.30	61.143629
2300	0.042900	1.424147	28.72	45.53	65.511031
2400	0.025200	1.412174	27.18	47.19	74.020711
2500	0.026500	1.438945	30.01	46.73	65.105808
2600	0.023300	1.454140	30.93	46.65	62.404322
2700	0.021600	1.431275	30.87	47.72	64.295362
2800	0.019200	1.439022	30.50	46.98	65.150833
2900	0.018200	1.439916	31.09	47.27	63.529941
3000	0.019000	1.444545	30.83	47.35	64.205313

Framework versions

• Transformers 4.40.2
• Pytorch 2.2.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1