fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-large-p2-with-ITTL-with-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2003
• Exact Match: 60.2113
• F1: 73.9948

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 64
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
6.2316	0.5	19	3.5321	11.9718	21.8197
6.2316	0.99	38	2.6566	19.1901	31.9985
3.5132	1.5	57	2.1442	27.2887	40.7031
3.5132	1.99	76	1.6755	41.5493	53.9850
3.5132	2.5	95	1.4228	48.2394	61.2829
1.845	2.99	114	1.2882	52.8169	66.2197
1.845	3.5	133	1.2352	54.7535	68.3725
1.2542	3.99	152	1.2033	56.6901	70.5019
1.2542	4.5	171	1.2117	57.9225	72.0740
1.2542	4.99	190	1.1748	58.4507	71.9264
0.9877	5.5	209	1.1763	58.8028	72.2772
0.9877	5.99	228	1.1827	59.5070	73.5652
0.9877	6.5	247	1.1789	59.8592	73.2748
0.8293	6.99	266	1.1835	60.0352	73.4695
0.8293	7.5	285	1.1669	59.8592	73.7145
0.7663	7.99	304	1.1912	60.3873	74.3001
0.7663	8.5	323	1.1828	60.2113	74.1533
0.7663	8.99	342	1.2046	60.3873	74.0424
0.7068	9.5	361	1.2003	60.2113	73.9948

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2