t5-small-trainings

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2580
• Rouge1: 41.5251
• Rouge2: 19.8842
• Rougel: 36.4895
• Rougelsum: 37.2565

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.1338	1.0	51	2.5825	35.4169	15.379	30.8859	31.524
2.5905	2.0	102	2.3975	38.4266	17.2571	33.5912	34.312
2.3881	3.0	153	2.3329	39.8082	19.1925	34.8269	35.5295
2.3167	4.0	204	2.2938	41.3488	20.1513	35.6879	36.5864
2.2357	5.0	255	2.2727	41.2457	19.5358	36.0033	36.8405
2.232	6.0	306	2.2645	41.2746	20.0345	35.9226	36.7001
2.1986	7.0	357	2.2595	41.7542	19.9428	36.6819	37.4718
2.1457	8.0	408	2.2580	41.5251	19.8842	36.4895	37.2565

Framework versions

• Transformers 4.18.0
• Pytorch 1.11.0+cu113
• Datasets 2.1.0
• Tokenizers 0.12.1