t5-small-finetuned-2048

This model is a fine-tuned version of google-t5/t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 13.3433
• Rouge1: 0.029
• Rouge2: 0.0023
• Rougel: 0.0267
• Rougelsum: 0.0284

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
No log	0.67	1	25.1883	0.0242	0.0023	0.0218	0.0241
No log	2.0	3	23.4392	0.0242	0.0023	0.0218	0.0241
No log	2.67	4	22.5166	0.0252	0.0023	0.0229	0.0251
No log	4.0	6	20.6643	0.0252	0.0023	0.0229	0.0251
No log	4.67	7	19.7334	0.0252	0.0023	0.0229	0.0251
No log	6.0	9	17.8137	0.0252	0.0023	0.0229	0.0251
No log	6.67	10	17.1117	0.0252	0.0023	0.0229	0.0251
No log	8.0	12	16.4384	0.0329	0.005	0.0269	0.0324
No log	8.67	13	16.2401	0.0329	0.005	0.0269	0.0324
No log	10.0	15	15.9056	0.0329	0.005	0.0269	0.0324
No log	10.67	16	15.7547	0.0329	0.005	0.0269	0.0324
No log	12.0	18	15.4599	0.0329	0.005	0.0269	0.0324
No log	12.67	19	15.3192	0.0329	0.005	0.0269	0.0324
17.3983	14.0	21	15.0513	0.0329	0.005	0.0269	0.0324
17.3983	14.67	22	14.9270	0.0367	0.005	0.0307	0.0357
17.3983	16.0	24	14.7037	0.0367	0.005	0.0307	0.0357
17.3983	16.67	25	14.5987	0.0367	0.005	0.0307	0.0357
17.3983	18.0	27	14.4010	0.0367	0.005	0.0307	0.0357
17.3983	18.67	28	14.3084	0.0367	0.005	0.0307	0.0357
17.3983	20.0	30	14.1348	0.0367	0.005	0.0307	0.0357
17.3983	20.67	31	14.0554	0.0367	0.005	0.0307	0.0357
17.3983	22.0	33	13.9103	0.0367	0.005	0.0307	0.0357
17.3983	22.67	34	13.8446	0.029	0.0023	0.0267	0.0284
17.3983	24.0	36	13.7251	0.029	0.0023	0.0267	0.0284
17.3983	24.67	37	13.6713	0.029	0.0023	0.0267	0.0284
17.3983	26.0	39	13.5781	0.029	0.0023	0.0267	0.0284
13.2153	26.67	40	13.5376	0.029	0.0023	0.0267	0.0284
13.2153	28.0	42	13.4689	0.029	0.0023	0.0267	0.0284
13.2153	28.67	43	13.4408	0.029	0.0023	0.0267	0.0284
13.2153	30.0	45	13.3953	0.029	0.0023	0.0267	0.0284
13.2153	30.67	46	13.3780	0.029	0.0023	0.0267	0.0284
13.2153	32.0	48	13.3538	0.029	0.0023	0.0267	0.0284
13.2153	32.67	49	13.3468	0.029	0.0023	0.0267	0.0284
13.2153	33.33	50	13.3433	0.029	0.0023	0.0267	0.0284

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.2.0
• Datasets 2.16.1
• Tokenizers 0.15.1