t5-fine-tuning

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.4274
• Bleu: 0.1699
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
3.9901	0.44	25	3.3580	0.5345	18.9005
3.4338	0.88	50	3.1131	0.227	19.0
3.2615	1.32	75	2.9891	0.1444	19.0
3.1286	1.75	100	2.8978	0.1767	19.0
3.0565	2.19	125	2.8249	0.1813	19.0
2.963	2.63	150	2.7604	0.1566	19.0
2.9233	3.07	175	2.7123	0.1815	19.0
2.8678	3.51	200	2.6672	0.1564	19.0
2.8132	3.95	225	2.6269	0.1746	19.0
2.7812	4.39	250	2.5950	0.1573	19.0
2.7436	4.82	275	2.5690	0.1841	19.0
2.7031	5.26	300	2.5425	0.2019	19.0
2.6841	5.7	325	2.5212	0.1956	19.0
2.6859	6.14	350	2.5053	0.1478	19.0
2.6585	6.58	375	2.4855	0.1754	19.0
2.6153	7.02	400	2.4720	0.1697	19.0
2.594	7.46	425	2.4600	0.1696	19.0
2.6114	7.89	450	2.4476	0.1685	19.0
2.5904	8.33	475	2.4413	0.1664	19.0
2.5829	8.77	500	2.4337	0.1675	19.0
2.5683	9.21	525	2.4304	0.1634	19.0
2.5731	9.65	550	2.4274	0.1699	19.0

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0