T5_base_title_v2

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0995
• Rouge1: 0.3574
• Rouge2: 0.1666
• Rougel: 0.3037
• Rougelsum: 0.303
• Gen Len: 16.495

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	100	2.1695	0.3249	0.1495	0.2795	0.2798	17.315
No log	2.0	200	2.0994	0.3595	0.1696	0.3078	0.3085	16.825
No log	3.0	300	2.0724	0.3679	0.1836	0.312	0.3131	16.525
No log	4.0	400	2.0745	0.3669	0.1767	0.3137	0.3141	16.505
2.0908	5.0	500	2.0567	0.3725	0.181	0.3205	0.3211	16.545
2.0908	6.0	600	2.0575	0.3654	0.174	0.3101	0.3097	16.62
2.0908	7.0	700	2.0640	0.3475	0.1649	0.2959	0.2956	16.485
2.0908	8.0	800	2.0588	0.3678	0.1827	0.312	0.3113	16.54
2.0908	9.0	900	2.0615	0.3654	0.1774	0.3106	0.3098	16.565
1.696	10.0	1000	2.0689	0.3654	0.1767	0.3077	0.3069	16.78
1.696	11.0	1100	2.0767	0.3633	0.1736	0.309	0.3078	16.57
1.696	12.0	1200	2.0749	0.366	0.1802	0.3147	0.3145	16.755
1.696	13.0	1300	2.0782	0.3632	0.1714	0.3117	0.3111	16.95
1.696	14.0	1400	2.0841	0.3637	0.1718	0.3118	0.3111	16.855
1.5311	15.0	1500	2.0873	0.3618	0.1713	0.3073	0.307	16.57
1.5311	16.0	1600	2.0940	0.3655	0.1714	0.3115	0.3111	16.625
1.5311	17.0	1700	2.0943	0.3619	0.1683	0.3089	0.3082	16.525
1.5311	18.0	1800	2.0981	0.3609	0.1697	0.3074	0.3065	16.44
1.5311	19.0	1900	2.0990	0.3567	0.1665	0.3047	0.3036	16.47
1.447	20.0	2000	2.0995	0.3574	0.1666	0.3037	0.303	16.495

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.2
• Datasets 2.1.0
• Tokenizers 0.15.1