swiftformer-xs-ve-U13-b-80d

This model is a fine-tuned version of MBZUAI/swiftformer-xs on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.7669
• Accuracy: 0.8261

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 70

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	6	1.3848	0.2174
1.3838	2.0	13	1.3723	0.1957
1.3838	2.92	19	1.3540	0.1739
1.3023	4.0	26	1.3327	0.2391
1.1398	4.92	32	1.2555	0.2391
1.1398	6.0	39	1.3010	0.3913
1.0076	6.92	45	1.1957	0.5
0.8823	8.0	52	1.0565	0.5870
0.8823	8.92	58	0.9410	0.7391
0.7637	10.0	65	0.9274	0.7391
0.6688	10.92	71	0.8492	0.7826
0.6688	12.0	78	0.8906	0.6739
0.5855	12.92	84	0.8929	0.6522
0.4921	14.0	91	0.8338	0.7391
0.4921	14.92	97	0.7686	0.7826
0.4318	16.0	104	0.8430	0.7609
0.386	16.92	110	0.8315	0.7826
0.386	18.0	117	0.7669	0.8261
0.3483	18.92	123	0.8347	0.7174
0.3023	20.0	130	1.1037	0.6304
0.3023	20.92	136	0.9024	0.7174
0.2973	22.0	143	0.7760	0.7826
0.2973	22.92	149	0.7400	0.7826
0.2529	24.0	156	1.0058	0.7174
0.2086	24.92	162	0.9260	0.7609
0.2086	26.0	169	0.8370	0.7174
0.2265	26.92	175	0.8060	0.7391
0.1942	28.0	182	0.9812	0.6957
0.1942	28.92	188	0.8996	0.7391
0.1708	30.0	195	0.9630	0.6957
0.1747	30.92	201	0.9691	0.7174
0.1747	32.0	208	1.0017	0.7391
0.1461	32.92	214	0.9965	0.6957
0.1457	34.0	221	0.9506	0.7391
0.1457	34.92	227	0.9834	0.7391
0.1814	36.0	234	1.0191	0.7609
0.1383	36.92	240	0.8807	0.7609
0.1383	38.0	247	0.8724	0.7609
0.1718	38.92	253	0.8090	0.7391
0.1289	40.0	260	0.8709	0.7609
0.1289	40.92	266	0.9704	0.7391
0.1193	42.0	273	1.0518	0.7391
0.1193	42.92	279	1.0157	0.7174
0.1224	44.0	286	1.0794	0.7391
0.1104	44.92	292	1.0402	0.7391
0.1104	46.0	299	0.9837	0.7609
0.1055	46.92	305	1.0323	0.7174
0.1242	48.0	312	0.9907	0.7391
0.1242	48.92	318	1.0436	0.7609
0.1283	50.0	325	0.9829	0.7391
0.1035	50.92	331	1.0400	0.7174
0.1035	52.0	338	1.0414	0.7174
0.1066	52.92	344	1.0958	0.6957
0.0863	54.0	351	1.0914	0.7174
0.0863	54.92	357	1.0816	0.7174
0.1062	56.0	364	1.0087	0.7174
0.1214	56.92	370	1.0454	0.7391
0.1214	58.0	377	1.0324	0.7391
0.0984	58.92	383	1.0591	0.6739
0.0966	60.0	390	1.0037	0.6957
0.0966	60.92	396	0.9887	0.6957
0.0626	62.0	403	1.0294	0.6739
0.0626	62.92	409	0.9939	0.7174
0.085	64.0	416	0.9886	0.6957
0.068	64.62	420	1.0773	0.6739

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0