jialicheng/cifar100-vit-base

vit-base

This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the cifar100 dataset. It achieves the following results on the evaluation set:

• Loss: 0.3142
• Accuracy: 0.9197

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 128
• eval_batch_size: 256
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
4.0904	1.0	333	4.0142	0.6663
3.5092	2.0	666	3.3563	0.7659
3.0987	3.0	999	2.9060	0.8043
2.7858	4.0	1332	2.5428	0.827
2.4372	5.0	1665	2.2353	0.8428
2.2157	6.0	1998	1.9597	0.8568
1.9573	7.0	2331	1.7241	0.8685
1.841	8.0	2664	1.5210	0.8736
1.6085	9.0	2997	1.3363	0.8832
1.4188	10.0	3330	1.1857	0.8861
1.3424	11.0	3663	1.0522	0.8923
1.1924	12.0	3996	0.9380	0.8983
1.1764	13.0	4329	0.8405	0.8999
1.0548	14.0	4662	0.7641	0.9024
0.9714	15.0	4995	0.6897	0.9069
0.9141	16.0	5328	0.6327	0.9047
0.8937	17.0	5661	0.5862	0.9065
0.79	18.0	5994	0.5389	0.9104
0.6761	19.0	6327	0.5092	0.9075
0.7064	20.0	6660	0.4760	0.9162
0.7224	21.0	6993	0.4502	0.9127
0.712	22.0	7326	0.4289	0.913
0.6541	23.0	7659	0.4088	0.913
0.6338	24.0	7992	0.3914	0.9172
0.6097	25.0	8325	0.3776	0.9182
0.6369	26.0	8658	0.3676	0.9155
0.6007	27.0	8991	0.3632	0.9149
0.6059	28.0	9324	0.3552	0.9187
0.5227	29.0	9657	0.3454	0.9178
0.6712	30.0	9990	0.3375	0.9183
0.5053	31.0	10323	0.3355	0.9171
0.5432	32.0	10656	0.3328	0.917
0.4617	33.0	10989	0.3295	0.9191
0.4784	34.0	11322	0.3250	0.918
0.5088	35.0	11655	0.3188	0.9195
0.5121	36.0	11988	0.3188	0.9172
0.4734	37.0	12321	0.3174	0.9193
0.5554	38.0	12654	0.3108	0.9196
0.4573	39.0	12987	0.3111	0.9203
0.4692	40.0	13320	0.3074	0.9203
0.481	41.0	13653	0.3042	0.922
0.4888	42.0	13986	0.3058	0.921
0.4032	43.0	14319	0.3025	0.9211
0.4731	44.0	14652	0.3063	0.9202
0.4574	45.0	14985	0.3052	0.92
0.3993	46.0	15318	0.3098	0.9215
0.4631	47.0	15651	0.3078	0.9201
0.409	48.0	15984	0.3056	0.9197
0.4584	49.0	16317	0.3060	0.9208
0.3853	50.0	16650	0.3061	0.9208
0.3836	51.0	16983	0.3072	0.9216
0.3969	52.0	17316	0.3070	0.9197
0.453	53.0	17649	0.3060	0.9188
0.3802	54.0	17982	0.3046	0.9204
0.4191	55.0	18315	0.3075	0.9208
0.4245	56.0	18648	0.3018	0.9205
0.4356	57.0	18981	0.3033	0.9214
0.348	58.0	19314	0.3081	0.9208
0.4232	59.0	19647	0.3058	0.9198
0.3363	60.0	19980	0.3066	0.9195
0.3537	61.0	20313	0.3067	0.9197
0.3613	62.0	20646	0.3065	0.9192
0.4121	63.0	20979	0.3086	0.9211
0.3939	64.0	21312	0.3095	0.9207
0.3616	65.0	21645	0.3061	0.9215
0.3645	66.0	21978	0.3085	0.9197
0.42	67.0	22311	0.3088	0.9191
0.3862	68.0	22644	0.3083	0.9193
0.3519	69.0	22977	0.3103	0.9187
0.4464	70.0	23310	0.3111	0.9192
0.3852	71.0	23643	0.3116	0.919
0.3406	72.0	23976	0.3082	0.9194
0.3785	73.0	24309	0.3071	0.9191
0.3559	74.0	24642	0.3101	0.9194
0.3298	75.0	24975	0.3099	0.9187
0.3596	76.0	25308	0.3099	0.9208
0.3419	77.0	25641	0.3120	0.9201
0.3918	78.0	25974	0.3077	0.9201
0.3571	79.0	26307	0.3119	0.9195
0.3609	80.0	26640	0.3120	0.9195
0.3324	81.0	26973	0.3120	0.9194
0.3387	82.0	27306	0.3118	0.9199
0.441	83.0	27639	0.3117	0.92
0.359	84.0	27972	0.3132	0.9195
0.3106	85.0	28305	0.3131	0.9204
0.3191	86.0	28638	0.3130	0.9201
0.3987	87.0	28971	0.3141	0.9202
0.3327	88.0	29304	0.3138	0.9194
0.3464	89.0	29637	0.3142	0.9207
0.3634	90.0	29970	0.3145	0.9207
0.3123	91.0	30303	0.3133	0.9197
0.3029	92.0	30636	0.3138	0.92
0.3814	93.0	30969	0.3124	0.9192
0.2953	94.0	31302	0.3126	0.9203
0.3475	95.0	31635	0.3141	0.9206
0.3406	96.0	31968	0.3141	0.9197
0.3448	97.0	32301	0.3141	0.9198
0.3687	98.0	32634	0.3137	0.9205
0.345	99.0	32967	0.3144	0.92
0.3582	100.0	33300	0.3142	0.9197

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.2+cu118
• Datasets 2.18.0
• Tokenizers 0.15.2