beit-base-patch16-224-OT

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3612
• Accuracy: 0.9516

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.91	5	1.3762	0.4677
1.3741	2.0	11	1.3227	0.4516
1.3741	2.91	16	1.2451	0.4516
1.2883	4.0	22	1.1303	0.5484
1.2883	4.91	27	1.0044	0.7419
1.1053	6.0	33	0.8687	0.7581
1.1053	6.91	38	0.7694	0.8387
0.917	8.0	44	0.6563	0.8065
0.917	8.91	49	0.5870	0.8710
0.7172	10.0	55	0.5842	0.7903
0.5924	10.91	60	0.4820	0.8710
0.5924	12.0	66	0.5346	0.8065
0.5272	12.91	71	0.3612	0.9516
0.5272	14.0	77	0.3838	0.9194
0.4901	14.91	82	0.4009	0.9032
0.4901	16.0	88	0.3721	0.8548
0.47	16.91	93	0.4358	0.8710
0.47	18.0	99	0.3734	0.8710
0.4714	18.91	104	0.4338	0.8548
0.3805	20.0	110	0.4152	0.8548
0.3805	20.91	115	0.3676	0.9194
0.388	22.0	121	0.3727	0.8871
0.388	22.91	126	0.3751	0.8871
0.3868	24.0	132	0.4173	0.8548
0.3868	24.91	137	0.3992	0.8710
0.3399	26.0	143	0.3749	0.8871
0.3399	26.91	148	0.4060	0.8548
0.3271	28.0	154	0.3926	0.9032
0.3271	28.91	159	0.3731	0.8710
0.3299	30.0	165	0.3836	0.8710
0.3114	30.91	170	0.4074	0.8871
0.3114	32.0	176	0.4274	0.8548
0.2738	32.91	181	0.3812	0.8710
0.2738	34.0	187	0.3795	0.8710
0.2906	34.91	192	0.3813	0.8710
0.2906	36.0	198	0.3886	0.8710
0.2623	36.36	200	0.3893	0.8710

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0