swinv2-tiny-patch4-window8-256-DMAE-da2-4e-5

This model is a fine-tuned version of microsoft/swinv2-tiny-patch4-window8-256 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.8252
• Accuracy: 0.6522

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
6.6265	0.96	11	7.9417	0.1087
6.1679	2.0	23	7.9339	0.1087
6.8561	2.96	34	7.9149	0.1087
6.399	4.0	46	7.8629	0.1087
6.7961	4.96	57	7.7408	0.1087
6.5019	6.0	69	7.4856	0.1087
6.085	6.96	80	7.1388	0.1087
6.014	8.0	92	6.7124	0.1087
5.6147	8.96	103	6.2444	0.1087
5.2206	10.0	115	5.6953	0.1087
4.9027	10.96	126	5.1300	0.1087
4.4278	12.0	138	4.4629	0.1087
4.0841	12.96	149	3.8089	0.1087
3.0004	14.0	161	3.0893	0.1087
2.5654	14.96	172	2.4630	0.1087
2.094	16.0	184	1.8911	0.1087
1.8348	16.96	195	1.5454	0.1087
1.5383	18.0	207	1.4124	0.1087
1.4068	18.96	218	1.4348	0.1087
1.379	20.0	230	1.4207	0.1522
1.3552	20.96	241	1.3786	0.3478
1.262	22.0	253	1.3580	0.4783
1.2235	22.96	264	1.4413	0.3696
1.2065	24.0	276	1.3013	0.1957
1.2196	24.96	287	1.3489	0.2609
1.1815	26.0	299	1.3158	0.3043
1.0469	26.96	310	1.2959	0.4565
0.9854	28.0	322	1.4712	0.5217
0.9443	28.96	333	1.2061	0.3261
0.9168	30.0	345	1.0655	0.4348
0.8138	30.96	356	1.0981	0.5435
0.8066	32.0	368	1.0023	0.6304
0.8035	32.96	379	0.9529	0.5652
0.7693	34.0	391	1.0104	0.5
0.6773	34.96	402	1.0684	0.5
0.6726	36.0	414	0.8252	0.6522
0.5859	36.96	425	0.8158	0.6304
0.6204	38.0	437	0.8518	0.6087
0.5921	38.26	440	0.8757	0.5870

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0