git-base-pokemon

This model is a fine-tuned version of microsoft/git-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0627
• Wer Score: 8.5567

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Score
2.2863	2.1277	50	0.3771	0.4680
0.1088	4.2553	100	0.0445	0.4631
0.0219	6.3830	150	0.0438	0.4483
0.0152	8.5106	200	0.0437	0.4532
0.0124	10.6383	250	0.0474	0.4877
0.0101	12.7660	300	0.0499	2.7241
0.008	14.8936	350	0.0512	4.0493
0.0064	17.0213	400	0.0535	5.2857
0.0039	19.1489	450	0.0574	7.3103
0.0025	21.2766	500	0.0587	7.6847
0.0015	23.4043	550	0.0620	8.0443
0.0011	25.5319	600	0.0617	9.0788
0.0009	27.6596	650	0.0627	8.5567

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.19.1