SF-DMAE-DA

This model is a fine-tuned version of MBZUAI/swiftformer-xs on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.8371
• Accuracy: 0.8261

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.15
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.3856	0.96	11	1.3864	0.1304
1.376	2.0	23	1.3770	0.1522
1.3201	2.96	34	1.3578	0.2609
1.2272	4.0	46	1.3456	0.2174
1.0812	4.96	57	1.3423	0.4348
1.0301	6.0	69	1.2316	0.4130
0.8375	6.96	80	1.1414	0.5
0.806	8.0	92	1.1792	0.4348
0.6895	8.96	103	1.0475	0.5
0.6142	10.0	115	0.9662	0.6522
0.5602	10.96	126	1.0034	0.6739
0.6118	12.0	138	0.9099	0.6087
0.56	12.96	149	0.9165	0.7609
0.4118	14.0	161	0.8947	0.7826
0.4115	14.96	172	0.9526	0.7391
0.4596	16.0	184	1.0376	0.6957
0.4066	16.96	195	0.8371	0.8261
0.3437	18.0	207	0.9002	0.8043
0.3279	18.96	218	0.8110	0.8043
0.3039	20.0	230	0.6589	0.8043
0.2315	20.96	241	0.8245	0.8261
0.2588	22.0	253	0.8610	0.7826
0.1889	22.96	264	0.7608	0.8043
0.2326	24.0	276	0.8180	0.8261
0.2107	24.96	287	0.8907	0.8261
0.1917	26.0	299	0.8404	0.7826
0.2	26.96	310	0.8136	0.7826
0.194	28.0	322	0.8591	0.7609
0.21	28.96	333	0.8823	0.8261
0.1917	30.0	345	0.7817	0.7609
0.1342	30.96	356	0.8810	0.8043
0.1185	32.0	368	0.9961	0.8043
0.1805	32.96	379	0.9183	0.8043
0.1548	34.0	391	0.9017	0.8261
0.1276	34.96	402	0.9056	0.8043
0.186	36.0	414	0.9376	0.8043
0.135	36.96	425	0.9160	0.8043
0.1313	38.0	437	0.8822	0.8043
0.1628	38.26	440	0.8337	0.8261

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0