antonkurylo/t5-base-news_headlines_12tokens

t5-base-news_headlines

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0458
• Rouge1: 54.4139
• Rouge2: 37.646
• Rougel: 52.7585
• Rougelsum: 52.7718

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.0018	1.0	1531	1.5329	43.1551	22.8869	40.7905	40.8246
1.574	2.0	3062	1.3419	46.5375	26.8791	44.41	44.4587
1.3702	3.0	4593	1.2201	48.6514	29.6228	46.6431	46.7098
1.2289	4.0	6124	1.1366	51.7488	33.7562	50.0123	50.0537
1.126	5.0	7655	1.0810	52.9846	35.6371	51.3321	51.3433
1.0569	6.0	9186	1.0585	53.8125	36.646	52.1451	52.1865
1.0105	7.0	10717	1.0458	54.4139	37.646	52.7585	52.7718

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3