indosum-lora-1

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5017
• Rouge1: 73.2585
• Rouge2: 66.378
• Rougel: 70.2761
• Rougelsum: 72.4613
• Gen Len: 102.6021

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.7824	1.0	892	0.5557	70.0617	62.6298	66.9506	69.2215	103.8571
0.6003	2.0	1784	0.5394	70.7684	63.445	67.6025	69.9195	102.4539
0.5559	3.0	2676	0.5173	72.718	65.7162	69.7084	71.9054	102.0601
0.5274	4.0	3568	0.5044	72.4621	65.4284	69.4763	71.685	103.5300
0.5052	5.0	4460	0.5017	72.8123	65.8699	69.8629	72.0214	102.3445

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1