git-base-naruto2

This model is a fine-tuned version of microsoft/git-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0286
• Wer Score: 0.3515

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 3
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Score
10.2502	0.0909	5	9.1485	62.2485
8.8428	0.1818	10	8.3401	80.6545
8.1981	0.2727	15	7.8041	80.8121
7.7018	0.3636	20	7.3280	80.3636
7.2331	0.4545	25	6.8603	40.0485
6.7819	0.5455	30	6.3932	20.9152
6.3079	0.6364	35	5.9241	22.0424
5.8546	0.7273	40	5.4566	6.7515
5.3926	0.8182	45	4.9938	6.9636
4.9258	0.9091	50	4.5336	6.8909
4.4733	1.0	55	4.0808	7.4061
4.0241	1.0909	60	3.6383	6.9273
3.5895	1.1818	65	3.2081	7.2121
3.1657	1.2727	70	2.7929	7.2121
2.7513	1.3636	75	2.3954	7.8303
2.3667	1.4545	80	2.0232	8.1576
1.9959	1.5455	85	1.6770	8.8727
1.6518	1.6364	90	1.3655	9.2242
1.349	1.7273	95	1.0882	9.3758
1.081	1.8182	100	0.8536	9.1939
0.8455	1.9091	105	0.6599	8.8667
0.6619	2.0	110	0.5056	9.2727
0.5017	2.0909	115	0.3868	9.3636
0.385	2.1818	120	0.2934	10.9758
0.2916	2.2727	125	0.2254	10.6848
0.2292	2.3636	130	0.1742	9.3758
0.1773	2.4545	135	0.1368	8.8909
0.1397	2.5455	140	0.1088	8.4364
0.1214	2.6364	145	0.0907	0.4121
0.0964	2.7273	150	0.0764	0.3939
0.0812	2.8182	155	0.0649	0.3818
0.07	2.9091	160	0.0597	0.3818
0.0613	3.0	165	0.0516	0.4121
0.0454	3.0909	170	0.0472	0.3879
0.0492	3.1818	175	0.0422	0.4
0.0411	3.2727	180	0.0411	0.4364
0.035	3.3636	185	0.0394	0.4303
0.0378	3.4545	190	0.0370	0.3879
0.0389	3.5455	195	0.0348	0.3939
0.0341	3.6364	200	0.0335	0.3636
0.0391	3.7273	205	0.0327	0.3697
0.0266	3.8182	210	0.0314	0.5212
0.0282	3.9091	215	0.0308	2.6364
0.0306	4.0	220	0.0300	0.4848
0.0263	4.0909	225	0.0306	0.3758
0.0237	4.1818	230	0.0300	0.3697
0.0255	4.2727	235	0.0292	0.3515
0.0232	4.3636	240	0.0290	0.3576
0.024	4.4545	245	0.0291	0.3515
0.0243	4.5455	250	0.0294	0.3636
0.0245	4.6364	255	0.0296	0.3697
0.022	4.7273	260	0.0294	0.3576
0.0228	4.8182	265	0.0291	0.3576
0.0255	4.9091	270	0.0287	0.3515
0.025	5.0	275	0.0286	0.3515

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1