vit-base-patch16-224-dmae-va-da2-40

This model is a fine-tuned version of google/vit-base-patch16-224 on Augusto777/dmae-ve-da2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2660
• Accuracy: 0.9655

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.94	4	1.2931	0.4310
No log	1.88	8	1.2024	0.5517
1.2651	2.82	12	0.9896	0.6552
1.2651	4.0	17	0.7972	0.7241
1.2651	4.94	21	0.7336	0.6552
0.7523	5.88	25	0.5781	0.8103
0.7523	6.82	29	0.4912	0.8793
0.7523	8.0	34	0.4112	0.9138
0.4209	8.94	38	0.3383	0.9138
0.4209	9.88	42	0.3129	0.9483
0.4209	10.82	46	0.2660	0.9655
0.2647	12.0	51	0.3184	0.9310
0.2647	12.94	55	0.2871	0.9310
0.2647	13.88	59	0.2766	0.9138
0.1743	14.82	63	0.2727	0.8966
0.1743	16.0	68	0.2282	0.9310
0.1511	16.94	72	0.2892	0.8966
0.1511	17.88	76	0.2482	0.8966
0.1511	18.82	80	0.2363	0.9310
0.1253	20.0	85	0.1622	0.9483
0.1253	20.94	89	0.1753	0.9483
0.1253	21.88	93	0.1593	0.9655
0.087	22.82	97	0.1334	0.9483
0.087	24.0	102	0.1088	0.9483
0.087	24.94	106	0.1130	0.9483
0.0856	25.88	110	0.1459	0.9138
0.0856	26.82	114	0.1445	0.9655
0.0856	28.0	119	0.1234	0.9655
0.081	28.94	123	0.1224	0.9483
0.081	29.88	127	0.1303	0.9483
0.081	30.82	131	0.1372	0.9483
0.0554	32.0	136	0.1421	0.9483
0.0554	32.94	140	0.1307	0.9483
0.0783	33.88	144	0.1244	0.9483
0.0783	34.82	148	0.1195	0.9483
0.0783	36.0	153	0.1171	0.9483
0.0646	36.94	157	0.1165	0.9483
0.0646	37.65	160	0.1163	0.9483

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1