ft_0112_korean

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6163
• Cer: 0.1655

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
66.0473	0.03	100	126.2500	1.0
39.2751	0.05	200	76.4439	1.0
24.2617	0.07	300	36.6274	1.0
10.2253	0.1	400	7.8025	1.0
4.9219	0.12	500	5.8257	1.0
4.7709	0.15	600	5.2597	1.0
4.7545	0.17	700	5.3516	1.0
4.701	0.2	800	5.2238	1.0
4.6753	0.23	900	5.1713	1.0
4.6339	0.25	1000	5.1546	1.0
4.6107	0.28	1100	5.0488	1.0
4.6086	0.3	1200	4.8149	1.0
4.5324	0.33	1300	4.7533	1.0
4.4797	0.35	1400	4.6892	1.0
4.4485	0.38	1500	4.5327	1.0
4.3794	0.4	1600	4.3797	0.9999
4.1549	0.42	1700	4.2075	0.9838
3.9647	0.45	1800	3.8729	0.9647
3.621	0.47	1900	3.3229	0.6854
3.3163	0.5	2000	2.9646	0.5646
3.0668	0.53	2100	2.7178	0.5608
2.8248	0.55	2200	2.4843	0.4937
2.7238	0.57	2300	2.3321	0.4736
2.614	0.6	2400	2.2513	0.4650
2.4994	0.62	2500	2.1655	0.4538
2.4431	0.65	2600	2.0785	0.4355
2.3307	0.68	2700	1.9603	0.4169
2.2495	0.7	2800	1.9026	0.4134
2.1647	0.72	2900	1.8152	0.4009
2.1075	0.75	3000	1.7521	0.3849
2.0577	0.78	3100	1.7004	0.3781
1.9935	0.8	3200	1.6226	0.3666
1.9391	0.82	3300	1.6097	0.3604
1.9295	0.85	3400	1.5416	0.3526
1.8759	0.88	3500	1.5227	0.3583
1.8316	0.9	3600	1.4791	0.3484
1.7531	0.93	3700	1.4472	0.3415
1.7413	0.95	3800	1.4178	0.3363
1.6609	0.97	3900	1.3587	0.3256
1.6986	1.0	4000	1.3396	0.3208
1.6189	1.02	4100	1.3253	0.3187
1.5853	1.05	4200	1.2929	0.3109
1.5153	1.07	4300	1.2691	0.3106
1.5259	1.1	4400	1.2500	0.3012
1.4916	1.12	4500	1.2151	0.2977
1.4113	1.15	4600	1.1796	0.2930
1.452	1.18	4700	1.1857	0.2928
1.3879	1.2	4800	1.1830	0.2915
1.4164	1.23	4900	1.1725	0.2920
1.4692	1.25	5000	1.1171	0.2794
1.346	1.27	5100	1.0858	0.2745
1.3964	1.3	5200	1.0644	0.2712
1.3359	1.32	5300	1.0585	0.2694
1.2769	1.35	5400	1.0290	0.2614
1.2741	1.38	5500	1.0356	0.2604
1.2257	1.4	5600	1.0167	0.2607
1.2416	1.43	5700	1.0074	0.2558
1.2376	1.45	5800	0.9889	0.2524
1.2048	1.48	5900	0.9649	0.2464
1.1335	1.5	6000	0.9580	0.2488
1.1946	1.52	6100	0.9503	0.2471
1.1926	1.55	6200	0.9467	0.2494
1.1451	1.57	6300	0.9202	0.2408
1.1426	1.6	6400	0.9018	0.2359
1.1569	1.62	6500	0.9216	0.2362
1.1093	1.65	6600	0.9433	0.2414
1.1258	1.68	6700	0.8986	0.2291
1.1024	1.7	6800	0.8838	0.2305
1.0567	1.73	6900	0.8916	0.2298
1.0928	1.75	7000	0.8855	0.2294
1.0526	1.77	7100	0.8592	0.2237
1.0236	1.8	7200	0.8433	0.2209
1.0454	1.82	7300	0.8382	0.2214
1.0252	1.85	7400	0.8252	0.2173
1.0404	1.88	7500	0.8190	0.2148
1.0326	1.9	7600	0.8067	0.2155
1.0008	1.93	7700	0.8081	0.2161
0.9814	1.95	7800	0.8061	0.2152
0.9664	1.98	7900	0.8147	0.2155
1.0032	2.0	8000	0.8232	0.2128
0.9274	2.02	8100	0.7951	0.2118
0.9115	2.05	8200	0.7857	0.2105
0.9339	2.08	8300	0.7722	0.2069
0.8553	2.1	8400	0.7603	0.2070
0.8671	2.12	8500	0.7927	0.2099
0.9067	2.15	8600	0.7511	0.2013
0.8507	2.17	8700	0.7763	0.2029
0.899	2.2	8800	0.7579	0.2026
0.8061	2.23	8900	0.7561	0.2014
0.8191	2.25	9000	0.7590	0.2024
0.8084	2.27	9100	0.7394	0.1972
0.8163	2.3	9200	0.7404	0.1941
0.8189	2.33	9300	0.7340	0.1955
0.8639	2.35	9400	0.7331	0.1950
0.8218	2.38	9500	0.7347	0.1959
0.8221	2.4	9600	0.7098	0.1922
0.7725	2.42	9700	0.7264	0.1923
0.7882	2.45	9800	0.7079	0.1875
0.7786	2.48	9900	0.7131	0.1913
0.7734	2.5	10000	0.7079	0.1912
0.7834	2.52	10100	0.6944	0.1896
0.78	2.55	10200	0.6980	0.1879
0.7602	2.58	10300	0.7076	0.1894
0.7415	2.6	10400	0.6946	0.1857
0.7791	2.62	10500	0.7025	0.1887
0.7357	2.65	10600	0.6949	0.1885
0.7102	2.67	10700	0.6978	0.1895
0.7395	2.7	10800	0.6893	0.1859
0.7301	2.73	10900	0.6847	0.1857
0.7492	2.75	11000	0.7063	0.1863
0.7372	2.77	11100	0.6917	0.1857
0.7474	2.8	11200	0.6843	0.1845
0.6727	2.83	11300	0.6628	0.1775
0.7342	2.85	11400	0.6729	0.1797
0.6599	2.88	11500	0.6631	0.1797
0.7209	2.9	11600	0.6658	0.1795
0.7222	2.92	11700	0.6741	0.1807
0.7124	2.95	11800	0.6722	0.1828
0.7304	2.98	11900	0.6606	0.1782
0.7234	3.0	12000	0.6499	0.1753
0.6857	3.02	12100	0.6547	0.1751
0.6238	3.05	12200	0.6615	0.1771
0.6495	3.08	12300	0.6499	0.1764
0.6219	3.1	12400	0.6558	0.1752
0.6684	3.12	12500	0.6479	0.1752
0.6455	3.15	12600	0.6574	0.1741
0.6414	3.17	12700	0.6489	0.1755
0.6619	3.2	12800	0.6527	0.1754
0.6303	3.23	12900	0.6462	0.1743
0.6525	3.25	13000	0.6505	0.1731
0.6347	3.27	13100	0.6432	0.1713
0.6206	3.3	13200	0.6495	0.1746
0.6445	3.33	13300	0.6328	0.1706
0.6097	3.35	13400	0.6329	0.1689
0.6151	3.38	13500	0.6473	0.1730
0.5948	3.4	13600	0.6413	0.1714
0.5949	3.42	13700	0.6377	0.1712
0.6402	3.45	13800	0.6295	0.1692
0.6607	3.48	13900	0.6287	0.1694
0.6219	3.5	14000	0.6357	0.1704
0.61	3.52	14100	0.6392	0.1715
0.5974	3.55	14200	0.6315	0.1687
0.5839	3.58	14300	0.6359	0.1689
0.6017	3.6	14400	0.6316	0.1673
0.6091	3.62	14500	0.6284	0.1686
0.6565	3.65	14600	0.6304	0.1684
0.6179	3.67	14700	0.6259	0.1661
0.5813	3.7	14800	0.6310	0.1672
0.5802	3.73	14900	0.6250	0.1667
0.6035	3.75	15000	0.6284	0.1666
0.5569	3.77	15100	0.6203	0.1651
0.5712	3.8	15200	0.6207	0.1660
0.546	3.83	15300	0.6246	0.1661
0.5602	3.85	15400	0.6206	0.1656
0.591	3.88	15500	0.6179	0.1650
0.5972	3.9	15600	0.6164	0.1653
0.6168	3.92	15700	0.6174	0.1660
0.5957	3.95	15800	0.6164	0.1657
0.5754	3.98	15900	0.6163	0.1657
0.5686	4.0	16000	0.6163	0.1655

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu121
• Datasets 2.13.0
• Tokenizers 0.15.0