vit-pretraining-2024_04_02

This model is a fine-tuned version of facebook/vit-mae-base on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.0132

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.6875e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 400.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.0282	1.0	3583	0.0275
0.0271	2.0	7166	0.0234
0.0203	3.0	10749	0.0218
0.0218	4.0	14332	0.0203
0.0195	5.0	17915	0.0191
0.0174	6.0	21498	0.0186
0.0173	7.0	25081	0.0181
0.0163	8.0	28664	0.0177
0.0172	9.0	32247	0.0173
0.0165	10.0	35830	0.0173
0.0173	11.0	39413	0.0169
0.0156	12.0	42996	0.0166
0.0168	13.0	46579	0.0165
0.0156	14.0	50162	0.0163
0.0152	15.0	53745	0.0161
0.0153	16.0	57328	0.0159
0.0163	17.0	60911	0.0158
0.0148	18.0	64494	0.0160
0.0144	19.0	68077	0.0159
0.0148	20.0	71660	0.0155
0.0169	21.0	75243	0.0156
0.0154	22.0	78826	0.0155
0.0151	23.0	82409	0.0154
0.0135	24.0	85992	0.0153
0.0157	25.0	89575	0.0153
0.0165	26.0	93158	0.0152
0.0146	27.0	96741	0.0151
0.0171	28.0	100324	0.0150
0.015	29.0	103907	0.0151
0.0165	30.0	107490	0.0152
0.0136	31.0	111073	0.0150
0.0147	32.0	114656	0.0147
0.0153	33.0	118239	0.0149
0.0135	34.0	121822	0.0147
0.0144	35.0	125405	0.0147
0.0155	36.0	128988	0.0148
0.0172	37.0	132571	0.0147
0.0141	38.0	136154	0.0149
0.0152	39.0	139737	0.0146
0.014	40.0	143320	0.0148
0.0158	41.0	146903	0.0147
0.0151	42.0	150486	0.0145
0.0155	43.0	154069	0.0147
0.0152	44.0	157652	0.0148
0.014	45.0	161235	0.0147
0.0142	46.0	164818	0.0145
0.014	47.0	168401	0.0147
0.0156	48.0	171984	0.0147
0.0138	49.0	175567	0.0145
0.0142	50.0	179150	0.0147
0.0142	51.0	182733	0.0143
0.014	52.0	186316	0.0143
0.0139	53.0	189899	0.0145
0.0133	54.0	193482	0.0146
0.0135	55.0	197065	0.0144
0.0137	56.0	200648	0.0145
0.0138	57.0	204231	0.0144
0.014	58.0	207814	0.0144
0.0139	59.0	211397	0.0144
0.0163	60.0	214980	0.0144
0.0163	61.0	218563	0.0142
0.016	62.0	222146	0.0144
0.0136	63.0	225729	0.0141
0.0152	64.0	229312	0.0142
0.0134	65.0	232895	0.0143
0.0146	66.0	236478	0.0143
0.0149	67.0	240061	0.0141
0.0148	68.0	243644	0.0144
0.0141	69.0	247227	0.0143
0.0145	70.0	250810	0.0142
0.0137	71.0	254393	0.0141
0.0137	72.0	257976	0.0141
0.0134	73.0	261559	0.0143
0.0186	74.0	265142	0.0141
0.0143	75.0	268725	0.0141
0.0155	76.0	272308	0.0142
0.0164	77.0	275891	0.0142
0.0137	78.0	279474	0.0141
0.0148	79.0	283057	0.0141
0.0136	80.0	286640	0.0142
0.0151	81.0	290223	0.0142
0.0137	82.0	293806	0.0141
0.0149	83.0	297389	0.0142
0.0145	84.0	300972	0.0141
0.0129	85.0	304555	0.0140
0.0144	86.0	308138	0.0142
0.0145	87.0	311721	0.0141
0.0125	88.0	315304	0.0140
0.0136	89.0	318887	0.0139
0.0138	90.0	322470	0.0141
0.0138	91.0	326053	0.0139
0.0144	92.0	329636	0.0139
0.0145	93.0	333219	0.0140
0.0136	94.0	336802	0.0140
0.0126	95.0	340385	0.0139
0.0154	96.0	343968	0.0140
0.0139	97.0	347551	0.0141
0.0135	98.0	351134	0.0138
0.0131	99.0	354717	0.0138
0.0123	100.0	358300	0.0139
0.014	101.0	361883	0.0140
0.0118	102.0	365466	0.0138
0.0151	103.0	369049	0.0140
0.0138	104.0	372632	0.0140
0.0156	105.0	376215	0.0139
0.0137	106.0	379798	0.0139
0.0136	107.0	383381	0.0140
0.0137	108.0	386964	0.0139
0.0151	109.0	390547	0.0138
0.0154	110.0	394130	0.0137
0.0142	111.0	397713	0.0140
0.0121	112.0	401296	0.0139
0.0131	113.0	404879	0.0139
0.0127	114.0	408462	0.0139
0.0131	115.0	412045	0.0140
0.014	116.0	415628	0.0137
0.0126	117.0	419211	0.0140
0.0135	118.0	422794	0.0140
0.0131	119.0	426377	0.0138
0.013	120.0	429960	0.0138
0.0145	121.0	433543	0.0139
0.0141	122.0	437126	0.0139
0.0134	123.0	440709	0.0140
0.0134	124.0	444292	0.0139
0.0138	125.0	447875	0.0139
0.0143	126.0	451458	0.0139
0.0127	127.0	455041	0.0139
0.0126	128.0	458624	0.0139
0.0141	129.0	462207	0.0139
0.0129	130.0	465790	0.0138
0.0138	131.0	469373	0.0139
0.0137	132.0	472956	0.0140
0.0143	133.0	476539	0.0138
0.0136	134.0	480122	0.0138
0.0142	135.0	483705	0.0138
0.0133	136.0	487288	0.0138
0.0162	137.0	490871	0.0136
0.0128	138.0	494454	0.0140
0.0137	139.0	498037	0.0139
0.0131	140.0	501620	0.0139
0.0137	141.0	505203	0.0138
0.0122	142.0	508786	0.0138
0.0132	143.0	512369	0.0137
0.0131	144.0	515952	0.0136
0.0139	145.0	519535	0.0138
0.0141	146.0	523118	0.0139
0.0148	147.0	526701	0.0137
0.0138	148.0	530284	0.0136
0.0139	149.0	533867	0.0139
0.0136	150.0	537450	0.0137
0.0121	151.0	541033	0.0139
0.014	152.0	544616	0.0138
0.013	153.0	548199	0.0136
0.0146	154.0	551782	0.0136
0.0138	155.0	555365	0.0136
0.0144	156.0	558948	0.0137
0.0141	157.0	562531	0.0138
0.0141	158.0	566114	0.0136
0.0116	159.0	569697	0.0137
0.012	160.0	573280	0.0136
0.0131	161.0	576863	0.0138
0.0131	162.0	580446	0.0136
0.0125	163.0	584029	0.0137
0.0145	164.0	587612	0.0137
0.0135	165.0	591195	0.0139
0.012	166.0	594778	0.0138
0.0144	167.0	598361	0.0137
0.0136	168.0	601944	0.0137
0.0123	169.0	605527	0.0136
0.0134	170.0	609110	0.0136
0.0128	171.0	612693	0.0134
0.0134	172.0	616276	0.0137
0.0129	173.0	619859	0.0137
0.014	174.0	623442	0.0136
0.0152	175.0	627025	0.0138
0.0139	176.0	630608	0.0136
0.0125	177.0	634191	0.0138
0.0142	178.0	637774	0.0136
0.0137	179.0	641357	0.0136
0.0168	180.0	644940	0.0134
0.0143	181.0	648523	0.0135
0.0118	182.0	652106	0.0137
0.0144	183.0	655689	0.0136
0.0125	184.0	659272	0.0136
0.014	185.0	662855	0.0136
0.0132	186.0	666438	0.0136
0.013	187.0	670021	0.0137
0.0128	188.0	673604	0.0136
0.0134	189.0	677187	0.0136
0.0138	190.0	680770	0.0137
0.013	191.0	684353	0.0135
0.0137	192.0	687936	0.0136
0.0124	193.0	691519	0.0135
0.0133	194.0	695102	0.0135
0.0135	195.0	698685	0.0136
0.0142	196.0	702268	0.0136
0.0122	197.0	705851	0.0136
0.0124	198.0	709434	0.0137
0.0154	199.0	713017	0.0136
0.0137	200.0	716600	0.0135
0.0128	201.0	720183	0.0135
0.0133	202.0	723766	0.0136
0.0139	203.0	727349	0.0136
0.0136	204.0	730932	0.0136
0.0143	205.0	734515	0.0134
0.0129	206.0	738098	0.0134
0.0142	207.0	741681	0.0136
0.0135	208.0	745264	0.0136
0.014	209.0	748847	0.0136
0.0132	210.0	752430	0.0135
0.0142	211.0	756013	0.0135
0.0136	212.0	759596	0.0134
0.0131	213.0	763179	0.0135
0.014	214.0	766762	0.0135
0.0131	215.0	770345	0.0135
0.0122	216.0	773928	0.0135
0.0128	217.0	777511	0.0135
0.0135	218.0	781094	0.0136
0.0136	219.0	784677	0.0135
0.0151	220.0	788260	0.0135
0.013	221.0	791843	0.0135
0.0123	222.0	795426	0.0135
0.013	223.0	799009	0.0134
0.0146	224.0	802592	0.0134
0.0144	225.0	806175	0.0135
0.0137	226.0	809758	0.0135
0.0115	227.0	813341	0.0134
0.0132	228.0	816924	0.0135
0.0141	229.0	820507	0.0135
0.013	230.0	824090	0.0136
0.0128	231.0	827673	0.0134
0.0138	232.0	831256	0.0135
0.0136	233.0	834839	0.0135
0.013	234.0	838422	0.0135
0.0137	235.0	842005	0.0134
0.0147	236.0	845588	0.0135
0.0134	237.0	849171	0.0135
0.013	238.0	852754	0.0133
0.0146	239.0	856337	0.0135
0.0145	240.0	859920	0.0136
0.0149	241.0	863503	0.0136
0.0131	242.0	867086	0.0134
0.0128	243.0	870669	0.0134
0.0139	244.0	874252	0.0133
0.0128	245.0	877835	0.0132
0.0138	246.0	881418	0.0135
0.0126	247.0	885001	0.0133
0.0144	248.0	888584	0.0134
0.0138	249.0	892167	0.0134
0.0137	250.0	895750	0.0134
0.0146	251.0	899333	0.0136
0.0135	252.0	902916	0.0134
0.0124	253.0	906499	0.0134
0.013	254.0	910082	0.0135
0.0136	255.0	913665	0.0134
0.0146	256.0	917248	0.0133
0.0129	257.0	920831	0.0136
0.013	258.0	924414	0.0134
0.0135	259.0	927997	0.0134
0.0134	260.0	931580	0.0131
0.0151	261.0	935163	0.0133
0.0126	262.0	938746	0.0133
0.0124	263.0	942329	0.0134
0.013	264.0	945912	0.0134
0.0139	265.0	949495	0.0133
0.0142	266.0	953078	0.0133
0.0129	267.0	956661	0.0134
0.0128	268.0	960244	0.0134
0.0177	269.0	963827	0.0133
0.0124	270.0	967410	0.0134
0.0165	271.0	970993	0.0133
0.0131	272.0	974576	0.0135
0.0119	273.0	978159	0.0133
0.0125	274.0	981742	0.0133
0.0127	275.0	985325	0.0133
0.013	276.0	988908	0.0134
0.0131	277.0	992491	0.0136
0.0131	278.0	996074	0.0134
0.0128	279.0	999657	0.0133
0.0146	280.0	1003240	0.0133
0.0138	281.0	1006823	0.0134
0.0125	282.0	1010406	0.0135
0.0121	283.0	1013989	0.0133
0.0137	284.0	1017572	0.0135
0.0144	285.0	1021155	0.0135
0.0131	286.0	1024738	0.0134
0.0133	287.0	1028321	0.0134
0.013	288.0	1031904	0.0133
0.013	289.0	1035487	0.0132
0.0133	290.0	1039070	0.0134
0.0127	291.0	1042653	0.0133
0.0121	292.0	1046236	0.0133
0.013	293.0	1049819	0.0132
0.0116	294.0	1053402	0.0131
0.0151	295.0	1056985	0.0133
0.013	296.0	1060568	0.0133
0.0121	297.0	1064151	0.0133
0.0127	298.0	1067734	0.0132
0.0125	299.0	1071317	0.0134
0.0132	300.0	1074900	0.0131
0.0121	301.0	1078483	0.0134
0.0147	302.0	1082066	0.0134
0.0127	303.0	1085649	0.0133
0.0131	304.0	1089232	0.0132
0.0144	305.0	1092815	0.0133
0.0145	306.0	1096398	0.0135
0.014	307.0	1099981	0.0132
0.0162	308.0	1103564	0.0132
0.0125	309.0	1107147	0.0133
0.0146	310.0	1110730	0.0134
0.0145	311.0	1114313	0.0132
0.0133	312.0	1117896	0.0133
0.0146	313.0	1121479	0.0133
0.0137	314.0	1125062	0.0133
0.0131	315.0	1128645	0.0132
0.014	316.0	1132228	0.0131
0.0134	317.0	1135811	0.0132
0.0138	318.0	1139394	0.0133
0.0133	319.0	1142977	0.0131
0.0131	320.0	1146560	0.0132
0.0127	321.0	1150143	0.0132
0.0137	322.0	1153726	0.0133
0.0129	323.0	1157309	0.0133
0.0123	324.0	1160892	0.0132
0.0135	325.0	1164475	0.0133
0.0132	326.0	1168058	0.0132
0.0125	327.0	1171641	0.0132
0.0135	328.0	1175224	0.0132
0.0135	329.0	1178807	0.0135
0.0139	330.0	1182390	0.0133
0.0133	331.0	1185973	0.0130
0.0123	332.0	1189556	0.0134
0.0141	333.0	1193139	0.0131
0.013	334.0	1196722	0.0130
0.015	335.0	1200305	0.0132
0.0139	336.0	1203888	0.0132
0.0125	337.0	1207471	0.0133
0.0133	338.0	1211054	0.0132
0.0131	339.0	1214637	0.0133
0.0136	340.0	1218220	0.0131
0.0114	341.0	1221803	0.0132
0.0146	342.0	1225386	0.0132
0.014	343.0	1228969	0.0132
0.0138	344.0	1232552	0.0131
0.0123	345.0	1236135	0.0133
0.0129	346.0	1239718	0.0132
0.0135	347.0	1243301	0.0133
0.0142	348.0	1246884	0.0132
0.0134	349.0	1250467	0.0132
0.0121	350.0	1254050	0.0132
0.0124	351.0	1257633	0.0132
0.0152	352.0	1261216	0.0133
0.0126	353.0	1264799	0.0134
0.012	354.0	1268382	0.0132
0.0138	355.0	1271965	0.0133
0.0136	356.0	1275548	0.0132
0.012	357.0	1279131	0.0132
0.0123	358.0	1282714	0.0134
0.0131	359.0	1286297	0.0131
0.0127	360.0	1289880	0.0133
0.0132	361.0	1293463	0.0131
0.0135	362.0	1297046	0.0131
0.0119	363.0	1300629	0.0132
0.0132	364.0	1304212	0.0132
0.013	365.0	1307795	0.0131
0.0124	366.0	1311378	0.0132
0.0122	367.0	1314961	0.0132
0.0146	368.0	1318544	0.0132
0.0121	369.0	1322127	0.0134
0.0119	370.0	1325710	0.0131
0.0138	371.0	1329293	0.0134
0.0132	372.0	1332876	0.0134
0.0146	373.0	1336459	0.0133
0.0126	374.0	1340042	0.0132
0.0125	375.0	1343625	0.0133
0.0129	376.0	1347208	0.0132
0.0117	377.0	1350791	0.0133
0.0114	378.0	1354374	0.0132
0.0123	379.0	1357957	0.0132
0.0119	380.0	1361540	0.0132
0.0131	381.0	1365123	0.0133
0.0135	382.0	1368706	0.0132
0.0154	383.0	1372289	0.0131
0.0151	384.0	1375872	0.0133
0.0125	385.0	1379455	0.0132
0.0137	386.0	1383038	0.0132
0.0125	387.0	1386621	0.0132
0.012	388.0	1390204	0.0132
0.0155	389.0	1393787	0.0132
0.013	390.0	1397370	0.0133
0.0131	391.0	1400953	0.0131
0.013	392.0	1404536	0.0132
0.0141	393.0	1408119	0.0132
0.0123	394.0	1411702	0.0132
0.0126	395.0	1415285	0.0132
0.0122	396.0	1418868	0.0132
0.0126	397.0	1422451	0.0131
0.0135	398.0	1426034	0.0131
0.0126	399.0	1429617	0.0133
0.0136	400.0	1433200	0.0131

Framework versions

• Transformers 4.39.0.dev0
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2