swinv2-tiny-patch4-window8-256-DMAE-5e-1

This model is a fine-tuned version of microsoft/swinv2-tiny-patch4-window8-256 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 54.3349
• Accuracy: 0.4565

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.5
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.86	3	343.8003	0.3261
No log	2.0	7	1260.1489	0.1087
517.6714	2.86	10	457.5103	0.3261
517.6714	4.0	14	180.1205	0.1087
517.6714	4.86	17	190.9627	0.1087
201.7487	6.0	21	54.3349	0.4565
201.7487	6.86	24	70.2849	0.3261
201.7487	8.0	28	57.7033	0.3261
64.7194	8.86	31	115.5257	0.1087
64.7194	10.0	35	72.7990	0.3261
64.7194	10.86	38	41.8670	0.4565
58.6249	12.0	42	26.2765	0.4565
58.6249	12.86	45	41.7245	0.3261
58.6249	14.0	49	23.6962	0.4565
49.7372	14.86	52	13.4265	0.3261
49.7372	16.0	56	7.0405	0.4565
49.7372	16.86	59	5.0777	0.4565
11.7669	18.0	63	13.5690	0.4565
11.7669	18.86	66	5.5425	0.1087
13.3323	20.0	70	6.4491	0.4565
13.3323	20.86	73	7.3066	0.3261
13.3323	22.0	77	10.8431	0.4565
9.2763	22.86	80	12.1588	0.3261
9.2763	24.0	84	5.4926	0.4565
9.2763	24.86	87	4.4689	0.3261
6.8526	26.0	91	3.7880	0.4565
6.8526	26.86	94	2.3297	0.4565
6.8526	28.0	98	2.8532	0.4565
3.0687	28.86	101	2.6943	0.1087
3.0687	30.0	105	2.0957	0.3261
3.0687	30.86	108	1.4001	0.4565
2.1059	32.0	112	1.3081	0.3261
2.1059	32.86	115	1.2392	0.3261
2.1059	34.0	119	1.2510	0.4565
1.3417	34.29	120	1.2338	0.4565

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0