antonkurylo/t5-base-cleaned_news_headlines

t5-base-news_headlines

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9947
• Rouge1: 53.8834
• Rouge2: 35.147
• Rougel: 50.8217
• Rougelsum: 50.9105

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.0129	1.0	1531	1.5160	44.1037	21.8536	40.2425	40.3433
1.6207	2.0	3062	1.2865	46.6327	25.2538	43.0594	43.1583
1.4243	3.0	4593	1.2410	48.3304	27.729	45.0085	45.0977
1.2828	4.0	6124	1.1008	50.7514	30.7978	47.5413	47.6432
1.1796	5.0	7655	1.0646	52.4672	33.0679	49.2593	49.3381
1.1059	6.0	9186	1.0082	53.4044	34.4035	50.3925	50.4943
1.0596	7.0	10717	0.9947	53.8834	35.147	50.8217	50.9105

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3