finetuned-xlm-r-masakhaner-swa-whole-word-phonetic

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 11.0418

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-08
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 500

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	62	39.5330
No log	2.0	124	39.2832
No log	3.0	186	39.7275
No log	4.0	248	38.7389
No log	5.0	310	38.1088
No log	6.0	372	37.6541
No log	7.0	434	36.2860
No log	8.0	496	36.0252
40.88	9.0	558	35.2045
40.88	10.0	620	34.1314
40.88	11.0	682	32.9068
40.88	12.0	744	32.0917
40.88	13.0	806	31.3414
40.88	14.0	868	30.1522
40.88	15.0	930	29.0864
40.88	16.0	992	27.7657
32.3449	17.0	1054	26.6795
32.3449	18.0	1116	25.4307
32.3449	19.0	1178	24.4571
32.3449	20.0	1240	24.0340
32.3449	21.0	1302	22.6889
32.3449	22.0	1364	21.9149
32.3449	23.0	1426	20.9471
32.3449	24.0	1488	20.7152
24.114	25.0	1550	19.9579
24.114	26.0	1612	19.4215
24.114	27.0	1674	18.7025
24.114	28.0	1736	18.0530
24.114	29.0	1798	17.7290
24.114	30.0	1860	17.2821
24.114	31.0	1922	16.8666
24.114	32.0	1984	16.4210
18.4015	33.0	2046	16.2339
18.4015	34.0	2108	15.7990
18.4015	35.0	2170	15.7727
18.4015	36.0	2232	15.2343
18.4015	37.0	2294	15.1427
18.4015	38.0	2356	14.8736
18.4015	39.0	2418	14.7042
18.4015	40.0	2480	14.5392
14.946	41.0	2542	14.5356
14.946	42.0	2604	14.1603
14.946	43.0	2666	14.1452
14.946	44.0	2728	14.1982
14.946	45.0	2790	13.9763
14.946	46.0	2852	13.9216
14.946	47.0	2914	13.7671
14.946	48.0	2976	13.6343
13.1518	49.0	3038	13.6241
13.1518	50.0	3100	13.4163
13.1518	51.0	3162	13.4401
13.1518	52.0	3224	13.5609
13.1518	53.0	3286	13.2858
13.1518	54.0	3348	13.1582
13.1518	55.0	3410	13.2456
13.1518	56.0	3472	13.2341
12.1359	57.0	3534	13.0155
12.1359	58.0	3596	13.0293
12.1359	59.0	3658	12.9712
12.1359	60.0	3720	12.9446
12.1359	61.0	3782	12.8301
12.1359	62.0	3844	12.7574
12.1359	63.0	3906	12.6967
12.1359	64.0	3968	12.6533
11.5584	65.0	4030	12.5547
11.5584	66.0	4092	12.5866
11.5584	67.0	4154	12.7563
11.5584	68.0	4216	12.6487
11.5584	69.0	4278	12.6015
11.5584	70.0	4340	12.5097
11.5584	71.0	4402	12.5312
11.5584	72.0	4464	12.4679
11.224	73.0	4526	12.4805
11.224	74.0	4588	12.4431
11.224	75.0	4650	12.3025
11.224	76.0	4712	12.1543
11.224	77.0	4774	12.3555
11.224	78.0	4836	12.2642
11.224	79.0	4898	12.3555
11.224	80.0	4960	12.2913
10.9897	81.0	5022	12.2483
10.9897	82.0	5084	12.1817
10.9897	83.0	5146	12.3593
10.9897	84.0	5208	12.0717
10.9897	85.0	5270	12.2478
10.9897	86.0	5332	12.0652
10.9897	87.0	5394	12.1249
10.9897	88.0	5456	12.0900
10.8156	89.0	5518	11.9840
10.8156	90.0	5580	12.1210
10.8156	91.0	5642	12.1809
10.8156	92.0	5704	12.0218
10.8156	93.0	5766	12.1439
10.8156	94.0	5828	11.9183
10.8156	95.0	5890	12.0674
10.8156	96.0	5952	12.0093
10.6906	97.0	6014	12.0089
10.6906	98.0	6076	11.9560
10.6906	99.0	6138	11.7673
10.6906	100.0	6200	12.0341
10.6906	101.0	6262	12.0382
10.6906	102.0	6324	11.8758
10.6906	103.0	6386	11.9393
10.6906	104.0	6448	11.8785
10.5767	105.0	6510	11.8156
10.5767	106.0	6572	11.7506
10.5767	107.0	6634	11.9005
10.5767	108.0	6696	11.8800
10.5767	109.0	6758	11.9346
10.5767	110.0	6820	11.8334
10.5767	111.0	6882	11.6163
10.5767	112.0	6944	11.8294
10.5013	113.0	7006	11.7866
10.5013	114.0	7068	11.7243
10.5013	115.0	7130	11.5645
10.5013	116.0	7192	11.7551
10.5013	117.0	7254	11.6259
10.5013	118.0	7316	11.7235
10.5013	119.0	7378	11.6999
10.5013	120.0	7440	11.4851
10.4493	121.0	7502	11.4951
10.4493	122.0	7564	11.5214
10.4493	123.0	7626	11.7396
10.4493	124.0	7688	11.6963
10.4493	125.0	7750	11.5305
10.4493	126.0	7812	11.7711
10.4493	127.0	7874	11.8031
10.4493	128.0	7936	11.5951
10.4493	129.0	7998	11.4736
10.3973	130.0	8060	11.6156
10.3973	131.0	8122	11.5447
10.3973	132.0	8184	11.3567
10.3973	133.0	8246	11.4839
10.3973	134.0	8308	11.4416
10.3973	135.0	8370	11.5395
10.3973	136.0	8432	11.4376
10.3973	137.0	8494	11.3781
10.3569	138.0	8556	11.4179
10.3569	139.0	8618	11.6747
10.3569	140.0	8680	11.6111
10.3569	141.0	8742	11.5093
10.3569	142.0	8804	11.4756
10.3569	143.0	8866	11.3498
10.3569	144.0	8928	11.5626
10.3569	145.0	8990	11.3474
10.3565	146.0	9052	11.3776
10.3565	147.0	9114	11.3628
10.3565	148.0	9176	11.4859
10.3565	149.0	9238	11.4221
10.3565	150.0	9300	11.4199
10.3565	151.0	9362	11.5701
10.3565	152.0	9424	11.4637
10.3565	153.0	9486	11.6586
10.3122	154.0	9548	11.4878
10.3122	155.0	9610	11.5903
10.3122	156.0	9672	11.3811
10.3122	157.0	9734	11.4550
10.3122	158.0	9796	11.5699
10.3122	159.0	9858	11.5154
10.3122	160.0	9920	11.5016
10.3122	161.0	9982	11.5712
10.2888	162.0	10044	11.2899
10.2888	163.0	10106	11.2509
10.2888	164.0	10168	11.4525
10.2888	165.0	10230	11.3640
10.2888	166.0	10292	11.3375
10.2888	167.0	10354	11.3969
10.2888	168.0	10416	11.4781
10.2888	169.0	10478	11.3980
10.2642	170.0	10540	11.2877
10.2642	171.0	10602	11.3369
10.2642	172.0	10664	11.3854
10.2642	173.0	10726	11.5050
10.2642	174.0	10788	11.4826
10.2642	175.0	10850	11.3425
10.2642	176.0	10912	11.2417
10.2642	177.0	10974	11.1263
10.2586	178.0	11036	11.4087
10.2586	179.0	11098	11.3919
10.2586	180.0	11160	11.2514
10.2586	181.0	11222	11.2570
10.2586	182.0	11284	11.4419
10.2586	183.0	11346	11.4110
10.2586	184.0	11408	11.3919
10.2586	185.0	11470	11.2426
10.2479	186.0	11532	11.3869
10.2479	187.0	11594	11.2953
10.2479	188.0	11656	11.3316
10.2479	189.0	11718	11.1373
10.2479	190.0	11780	11.1654
10.2479	191.0	11842	11.3204
10.2479	192.0	11904	11.3129
10.2479	193.0	11966	11.2715
10.2172	194.0	12028	11.2646
10.2172	195.0	12090	11.3356
10.2172	196.0	12152	11.3013
10.2172	197.0	12214	11.5378
10.2172	198.0	12276	11.3641
10.2172	199.0	12338	11.3698
10.2172	200.0	12400	11.2677
10.2172	201.0	12462	11.2973
10.2023	202.0	12524	11.1405
10.2023	203.0	12586	11.3286
10.2023	204.0	12648	11.2122
10.2023	205.0	12710	11.2324
10.2023	206.0	12772	11.3512
10.2023	207.0	12834	11.0882
10.2023	208.0	12896	11.3575
10.2023	209.0	12958	11.2502
10.1926	210.0	13020	11.3278
10.1926	211.0	13082	11.2199
10.1926	212.0	13144	11.2567
10.1926	213.0	13206	11.2855
10.1926	214.0	13268	11.1893
10.1926	215.0	13330	11.2600
10.1926	216.0	13392	11.2850
10.1926	217.0	13454	11.2274
10.2075	218.0	13516	11.1108
10.2075	219.0	13578	11.2756
10.2075	220.0	13640	11.2839
10.2075	221.0	13702	11.2161
10.2075	222.0	13764	11.1866
10.2075	223.0	13826	11.1222
10.2075	224.0	13888	11.2648
10.2075	225.0	13950	11.2936
10.1822	226.0	14012	11.1362
10.1822	227.0	14074	11.2598
10.1822	228.0	14136	11.3352
10.1822	229.0	14198	11.1183
10.1822	230.0	14260	11.2075
10.1822	231.0	14322	11.1908
10.1822	232.0	14384	11.1331
10.1822	233.0	14446	11.0929
10.1863	234.0	14508	11.1061
10.1863	235.0	14570	11.0546
10.1863	236.0	14632	11.1765
10.1863	237.0	14694	11.2513
10.1863	238.0	14756	11.0919
10.1863	239.0	14818	11.2739
10.1863	240.0	14880	11.0342
10.1863	241.0	14942	11.2368
10.1724	242.0	15004	11.2552
10.1724	243.0	15066	11.1515
10.1724	244.0	15128	11.0491
10.1724	245.0	15190	11.0184
10.1724	246.0	15252	11.3787
10.1724	247.0	15314	11.2329
10.1724	248.0	15376	11.0729
10.1724	249.0	15438	11.1472
10.1671	250.0	15500	11.0619
10.1671	251.0	15562	11.2197
10.1671	252.0	15624	11.2241
10.1671	253.0	15686	11.3102
10.1671	254.0	15748	11.2900
10.1671	255.0	15810	11.3001
10.1671	256.0	15872	11.2206
10.1671	257.0	15934	11.1027
10.1671	258.0	15996	11.3067
10.1579	259.0	16058	11.3017
10.1579	260.0	16120	11.0980
10.1579	261.0	16182	11.2158
10.1579	262.0	16244	10.9408
10.1579	263.0	16306	11.1086
10.1579	264.0	16368	11.2858
10.1579	265.0	16430	11.2805
10.1579	266.0	16492	11.0887
10.1552	267.0	16554	11.1715
10.1552	268.0	16616	11.1470
10.1552	269.0	16678	11.1067
10.1552	270.0	16740	11.1639
10.1552	271.0	16802	11.1470
10.1552	272.0	16864	11.2063
10.1552	273.0	16926	11.1647
10.1552	274.0	16988	11.0061
10.1468	275.0	17050	11.3822
10.1468	276.0	17112	10.9467
10.1468	277.0	17174	11.2735
10.1468	278.0	17236	11.0132
10.1468	279.0	17298	11.1604
10.1468	280.0	17360	11.1620
10.1468	281.0	17422	11.0650
10.1468	282.0	17484	10.9920
10.1402	283.0	17546	11.0450
10.1402	284.0	17608	10.9830
10.1402	285.0	17670	11.0135
10.1402	286.0	17732	11.2642
10.1402	287.0	17794	11.1124
10.1402	288.0	17856	11.1928
10.1402	289.0	17918	11.1284
10.1402	290.0	17980	11.2088
10.147	291.0	18042	11.0756
10.147	292.0	18104	11.0514
10.147	293.0	18166	11.1475
10.147	294.0	18228	11.0448
10.147	295.0	18290	11.2417
10.147	296.0	18352	11.1070
10.147	297.0	18414	11.0149
10.147	298.0	18476	11.1680
10.1331	299.0	18538	11.1006
10.1331	300.0	18600	10.9447
10.1331	301.0	18662	10.9882
10.1331	302.0	18724	11.0269
10.1331	303.0	18786	10.9764
10.1331	304.0	18848	11.0889
10.1331	305.0	18910	11.0243
10.1331	306.0	18972	11.0250
10.1244	307.0	19034	11.1133
10.1244	308.0	19096	11.0134
10.1244	309.0	19158	10.9951
10.1244	310.0	19220	11.2620
10.1244	311.0	19282	11.1107
10.1244	312.0	19344	11.0597
10.1244	313.0	19406	10.9608
10.1244	314.0	19468	10.9975
10.1251	315.0	19530	11.0170
10.1251	316.0	19592	11.1525
10.1251	317.0	19654	11.0108
10.1251	318.0	19716	11.0247
10.1251	319.0	19778	11.1131
10.1251	320.0	19840	11.0462
10.1251	321.0	19902	11.0606
10.1251	322.0	19964	11.0049
10.1178	323.0	20026	11.1101
10.1178	324.0	20088	11.0348
10.1178	325.0	20150	11.2864
10.1178	326.0	20212	10.9508
10.1178	327.0	20274	10.9956
10.1178	328.0	20336	10.9561
10.1178	329.0	20398	10.9534
10.1178	330.0	20460	11.3653
10.1215	331.0	20522	11.1188
10.1215	332.0	20584	11.0908
10.1215	333.0	20646	10.9856
10.1215	334.0	20708	10.9873
10.1215	335.0	20770	11.0489
10.1215	336.0	20832	11.1130
10.1215	337.0	20894	10.8774
10.1215	338.0	20956	10.7878
10.1267	339.0	21018	11.1740
10.1267	340.0	21080	11.0039
10.1267	341.0	21142	11.0325
10.1267	342.0	21204	11.0498
10.1267	343.0	21266	11.0237
10.1267	344.0	21328	11.0935
10.1267	345.0	21390	10.8161
10.1267	346.0	21452	11.0701
10.1105	347.0	21514	10.8763
10.1105	348.0	21576	11.0623
10.1105	349.0	21638	10.9502
10.1105	350.0	21700	11.0760
10.1105	351.0	21762	11.1472
10.1105	352.0	21824	11.1567
10.1105	353.0	21886	11.0119
10.1105	354.0	21948	11.0740
10.1124	355.0	22010	11.0351
10.1124	356.0	22072	10.9959
10.1124	357.0	22134	10.9829
10.1124	358.0	22196	11.1624
10.1124	359.0	22258	11.1082
10.1124	360.0	22320	11.0496
10.1124	361.0	22382	11.1115
10.1124	362.0	22444	10.8872
10.1057	363.0	22506	10.8992
10.1057	364.0	22568	11.0268
10.1057	365.0	22630	10.9615
10.1057	366.0	22692	10.9675
10.1057	367.0	22754	11.1775
10.1057	368.0	22816	10.9949
10.1057	369.0	22878	11.0816
10.1057	370.0	22940	11.0516
10.1217	371.0	23002	11.0672
10.1217	372.0	23064	10.9670
10.1217	373.0	23126	11.2001
10.1217	374.0	23188	11.0178
10.1217	375.0	23250	10.8959
10.1217	376.0	23312	11.1671
10.1217	377.0	23374	11.1813
10.1217	378.0	23436	11.1560
10.1217	379.0	23498	11.0848
10.1098	380.0	23560	11.0467
10.1098	381.0	23622	11.1489
10.1098	382.0	23684	11.0066
10.1098	383.0	23746	10.9771
10.1098	384.0	23808	10.8953
10.1098	385.0	23870	10.9032
10.1098	386.0	23932	10.8922
10.1098	387.0	23994	10.9931
10.1262	388.0	24056	11.1278
10.1262	389.0	24118	11.1011
10.1262	390.0	24180	11.0422
10.1262	391.0	24242	11.0631
10.1262	392.0	24304	10.9703
10.1262	393.0	24366	10.9536
10.1262	394.0	24428	11.0767
10.1262	395.0	24490	10.9228
10.1038	396.0	24552	11.1420
10.1038	397.0	24614	11.1451
10.1038	398.0	24676	11.1663
10.1038	399.0	24738	11.0756
10.1038	400.0	24800	11.1149
10.1038	401.0	24862	11.0656
10.1038	402.0	24924	11.0752
10.1038	403.0	24986	11.1417
10.1159	404.0	25048	10.9629
10.1159	405.0	25110	10.9380
10.1159	406.0	25172	11.0099
10.1159	407.0	25234	11.1378
10.1159	408.0	25296	10.8511
10.1159	409.0	25358	11.0122
10.1159	410.0	25420	11.0101
10.1159	411.0	25482	10.8735
10.0848	412.0	25544	11.0082
10.0848	413.0	25606	11.0128
10.0848	414.0	25668	11.1373
10.0848	415.0	25730	11.1380
10.0848	416.0	25792	10.9388
10.0848	417.0	25854	10.9308
10.0848	418.0	25916	11.0828
10.0848	419.0	25978	11.2567
10.0923	420.0	26040	11.0402
10.0923	421.0	26102	11.0285
10.0923	422.0	26164	11.0028
10.0923	423.0	26226	10.9785
10.0923	424.0	26288	11.0468
10.0923	425.0	26350	11.0595
10.0923	426.0	26412	11.0987
10.0923	427.0	26474	10.8764
10.1	428.0	26536	10.9643
10.1	429.0	26598	10.9864
10.1	430.0	26660	11.0019
10.1	431.0	26722	11.1037
10.1	432.0	26784	10.9560
10.1	433.0	26846	10.9808
10.1	434.0	26908	11.0447
10.1	435.0	26970	10.9433
10.0944	436.0	27032	11.0408
10.0944	437.0	27094	10.8298
10.0944	438.0	27156	11.1691
10.0944	439.0	27218	11.0291
10.0944	440.0	27280	10.9908
10.0944	441.0	27342	10.8890
10.0944	442.0	27404	11.0247
10.0944	443.0	27466	11.1023
10.0965	444.0	27528	11.1252
10.0965	445.0	27590	11.0374
10.0965	446.0	27652	11.0327
10.0965	447.0	27714	10.8393
10.0965	448.0	27776	10.9528
10.0965	449.0	27838	11.0124
10.0965	450.0	27900	10.8483
10.0965	451.0	27962	10.8977
10.1023	452.0	28024	11.0330
10.1023	453.0	28086	10.9282
10.1023	454.0	28148	11.0282
10.1023	455.0	28210	10.9465
10.1023	456.0	28272	10.9338
10.1023	457.0	28334	10.9981
10.1023	458.0	28396	11.0046
10.1023	459.0	28458	11.0581
10.0928	460.0	28520	11.0699
10.0928	461.0	28582	11.0084
10.0928	462.0	28644	10.9690
10.0928	463.0	28706	11.0585
10.0928	464.0	28768	11.1586
10.0928	465.0	28830	11.0082
10.0928	466.0	28892	11.0495
10.0928	467.0	28954	10.8531
10.0925	468.0	29016	10.8587
10.0925	469.0	29078	11.1600
10.0925	470.0	29140	10.9920
10.0925	471.0	29202	11.0200
10.0925	472.0	29264	11.0570
10.0925	473.0	29326	10.9316
10.0925	474.0	29388	11.0597
10.0925	475.0	29450	10.9941
10.0956	476.0	29512	11.0770
10.0956	477.0	29574	11.0068
10.0956	478.0	29636	10.9080
10.0956	479.0	29698	10.8656
10.0956	480.0	29760	11.0350
10.0956	481.0	29822	11.0673
10.0956	482.0	29884	11.0344
10.0956	483.0	29946	10.9819
10.112	484.0	30008	11.0667
10.112	485.0	30070	10.9373
10.112	486.0	30132	10.9777
10.112	487.0	30194	10.9832
10.112	488.0	30256	11.1426
10.112	489.0	30318	11.0771
10.112	490.0	30380	10.9515
10.112	491.0	30442	10.9617
10.1007	492.0	30504	10.9523
10.1007	493.0	30566	10.8797
10.1007	494.0	30628	10.9816
10.1007	495.0	30690	10.9526
10.1007	496.0	30752	11.0078
10.1007	497.0	30814	11.1568
10.1007	498.0	30876	10.9862
10.1007	499.0	30938	11.0537
10.0953	500.0	31000	11.0418

Framework versions

• Transformers 4.20.1
• Pytorch 1.11.0
• Datasets 2.1.0
• Tokenizers 0.12.1