indosum-base-2

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6175
• Rouge1: 72.8333
• Rouge2: 66.3463
• Rougel: 70.0278
• Rougelsum: 72.0501
• Gen Len: 97.6051

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.2107	1.0	894	0.7716	66.2818	58.8499	63.3826	65.4963	106.3440
0.6904	2.0	1788	0.6373	72.0398	65.3647	69.1761	71.211	100.4793
0.4894	3.0	2682	0.6036	71.7057	65.1188	68.8683	70.8645	98.3481
0.3369	4.0	3576	0.6175	72.2133	65.6425	69.3517	71.4102	97.7162
0.1982	5.0	4470	0.7000	72.1304	65.3551	69.0923	71.291	98.3614

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1