Hubert-common_voice-ja-demo-kana-only-cosine

This model is a fine-tuned version of rinna/japanese-hubert-base on the MOZILLA-FOUNDATION/COMMON_VOICE_13_0 - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.6322
• Wer: 1.0
• Cer: 0.3308

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 25.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
No log	0.2660	100	43.2251	1.5295	5.8210
No log	0.5319	200	42.4452	1.5322	5.2849
No log	0.7979	300	40.4154	1.1262	1.8126
No log	1.0638	400	32.8021	1.0	0.9999
31.884	1.3298	500	20.8133	1.0	0.9999
31.884	1.5957	600	17.5824	1.0	0.9999
31.884	1.8617	700	16.8682	1.0	0.9999
31.884	2.1277	800	16.4466	1.0	0.9999
31.884	2.3936	900	16.0037	1.0	0.9999
14.4701	2.6596	1000	15.5409	1.0	0.9999
14.4701	2.9255	1100	15.0446	1.0	0.9999
14.4701	3.1915	1200	14.5023	1.0	0.9999
14.4701	3.4574	1300	13.9298	1.0	0.9999
14.4701	3.7234	1400	13.3212	1.0	0.9999
12.1626	3.9894	1500	12.6814	1.0	0.9999
12.1626	4.2553	1600	12.0099	1.0	0.9999
12.1626	4.5213	1700	11.3179	1.0	0.9999
12.1626	4.7872	1800	10.6017	1.0	0.9999
12.1626	5.0532	1900	9.8810	1.0	0.9999
9.5127	5.3191	2000	9.1567	1.0	0.9999
9.5127	5.5851	2100	8.4445	1.0	0.9999
9.5127	5.8511	2200	7.7573	1.0	0.9999
9.5127	6.1170	2300	7.1049	1.0	0.9999
9.5127	6.3830	2400	6.5016	1.0	0.9999
6.6873	6.6489	2500	5.9565	1.0	0.9999
6.6873	6.9149	2600	5.4853	1.0	0.9999
6.6873	7.1809	2700	5.0997	1.0	0.9999
6.6873	7.4468	2800	4.7889	1.0	0.9999
6.6873	7.7128	2900	4.5573	1.0	0.9999
4.7448	7.9787	3000	4.3889	1.0	0.9999
4.7448	8.2447	3100	4.2614	1.0	0.9999
4.7448	8.5106	3200	4.1960	1.0	0.9999
4.7448	8.7766	3300	4.1398	1.0	0.9999
4.7448	9.0426	3400	4.1092	1.0	0.9999
4.1253	9.3085	3500	4.0911	1.0	0.9999
4.1253	9.5745	3600	4.0851	1.0	0.9999
4.1253	9.8404	3700	4.0707	1.0	0.9999
4.1253	10.1064	3800	4.0630	1.0	0.9999
4.1253	10.3723	3900	4.0589	1.0	0.9999
4.0399	10.6383	4000	4.0574	1.0	0.9999
4.0399	10.9043	4100	4.0495	1.0	0.9999
4.0399	11.1702	4200	4.0367	1.0	0.9999
4.0399	11.4362	4300	4.0297	1.0	0.9999
4.0399	11.7021	4400	4.0168	1.0	0.9999
4.0102	11.9681	4500	4.0002	1.0	0.9999
4.0102	12.2340	4600	3.9823	1.0	0.9999
4.0102	12.5	4700	3.9474	1.0	0.9999
4.0102	12.7660	4800	3.8870	1.0	0.9999
4.0102	13.0319	4900	3.7933	1.0	0.9999
3.8616	13.2979	5000	3.6576	1.0	0.9999
3.8616	13.5638	5100	3.4925	1.0	0.9999
3.8616	13.8298	5200	3.2550	1.0	0.9999
3.8616	14.0957	5300	2.8836	1.0	0.8301
3.8616	14.3617	5400	2.5211	1.0	0.6171
3.023	14.6277	5500	2.2902	1.0	0.5481
3.023	14.8936	5600	2.1006	1.0	0.5079
3.023	15.1596	5700	1.9464	1.0	0.4784
3.023	15.4255	5800	1.8196	1.0	0.4597
3.023	15.6915	5900	1.6975	1.0	0.4238
1.9348	15.9574	6000	1.6040	1.0	0.4093
1.9348	16.2234	6100	1.5035	1.0	0.4021
1.9348	16.4894	6200	1.4211	1.0	0.3930
1.9348	16.7553	6300	1.3529	1.0	0.3802
1.9348	17.0213	6400	1.2795	1.0	0.3791
1.4128	17.2872	6500	1.2193	1.0	0.3711
1.4128	17.5532	6600	1.1646	1.0	0.3674
1.4128	17.8191	6700	1.1193	1.0	0.3706
1.4128	18.0851	6800	1.0665	1.0	0.3606
1.4128	18.3511	6900	1.0244	0.9998	0.3590
1.1012	18.6170	7000	0.9864	1.0	0.3540
1.1012	18.8830	7100	0.9578	1.0	0.3554
1.1012	19.1489	7200	0.9309	0.9998	0.3509
1.1012	19.4149	7300	0.9070	1.0	0.3495
1.1012	19.6809	7400	0.8693	0.9998	0.3470
0.9083	19.9468	7500	0.8492	1.0	0.3449
0.9083	20.2128	7600	0.8214	1.0	0.3449
0.9083	20.4787	7700	0.8211	1.0	0.3500
0.9083	20.7447	7800	0.7964	1.0	0.3452
0.9083	21.0106	7900	0.7797	1.0	0.3429
0.7546	21.2766	8000	0.7634	1.0	0.3400
0.7546	21.5426	8100	0.7471	1.0	0.3384
0.7546	21.8085	8200	0.7400	1.0	0.3378
0.7546	22.0745	8300	0.7214	1.0	0.3390
0.7546	22.3404	8400	0.7062	0.9998	0.3375
0.651	22.6064	8500	0.6973	1.0	0.3344
0.651	22.8723	8600	0.6930	0.9998	0.3344
0.651	23.1383	8700	0.6829	1.0	0.3350
0.651	23.4043	8800	0.6683	1.0	0.3332
0.651	23.6702	8900	0.6596	0.9998	0.3322
0.5868	23.9362	9000	0.6764	1.0	0.3321
0.5868	24.2021	9100	0.6635	0.9998	0.3308
0.5868	24.4681	9200	0.6560	1.0	0.3324
0.5868	24.7340	9300	0.6412	1.0	0.3290
0.5868	25.0	9400	0.6323	1.0	0.3307

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3