t5-small-finetuned-chinese-to-hausa

This model is a fine-tuned version of google-t5/t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6584
• Bleu: 12.1339
• Gen Len: 17.7108

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0008
• train_batch_size: 32
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 5000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
3.7025	1.0	846	2.6105	10.8813	18.9939
2.5979	2.0	1692	2.1871	11.8977	18.9845
2.2642	3.0	2538	1.9964	10.0898	18.9515
2.0674	4.0	3384	1.8804	10.6493	17.1889
1.943	5.0	4230	1.8018	10.3424	18.9035
1.8501	6.0	5076	1.7529	12.523	18.8649
1.7736	7.0	5922	1.7176	11.9247	18.8803
1.711	8.0	6768	1.6823	11.623	18.3937
1.6537	9.0	7614	1.6550	12.3403	18.8995
1.6083	10.0	8460	1.6398	11.774	18.6873
1.5689	11.0	9306	1.6300	12.8081	18.8652
1.5314	12.0	10152	1.6279	12.2929	18.854
1.4962	13.0	10998	1.6197	12.1522	17.8497
1.4655	14.0	11844	1.6050	12.0764	17.9718
1.435	15.0	12690	1.6076	12.2447	17.9524
1.4047	16.0	13536	1.6048	11.4017	18.6209
1.38	17.0	14382	1.6134	11.5516	18.7405
1.3513	18.0	15228	1.6105	8.6492	14.2507
1.3282	19.0	16074	1.6174	12.094	17.7717
1.3052	20.0	16920	1.6239	11.5085	18.7053
1.2835	21.0	17766	1.6238	12.1588	17.7876
1.2653	22.0	18612	1.6339	12.0899	17.7112
1.2503	23.0	19458	1.6399	12.1466	17.7452
1.2357	24.0	20304	1.6461	9.0097	14.3578
1.2236	25.0	21150	1.6501	12.2617	17.7143
1.2139	26.0	21996	1.6528	12.1118	17.7233
1.2066	27.0	22842	1.6576	12.1435	17.7094
1.2037	28.0	23688	1.6568	12.1328	17.7133
1.2	29.0	24534	1.6581	12.1289	17.7109
1.1967	30.0	25380	1.6584	12.1339	17.7108

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1