zhko_mbartLarge_19p_run2

This model is a fine-tuned version of facebook/mbart-large-50-many-to-many-mmt on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4800
• Bleu: 16.4426
• Gen Len: 14.8195

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• total_eval_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1500
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.6564	0.9	2500	1.5828	14.0512	15.1829
1.1837	1.79	5000	1.4800	16.4426	14.826
0.8715	2.69	7500	1.4947	16.7298	14.6211
0.6403	3.59	10000	1.5886	16.9459	14.6253
0.4763	4.49	12500	1.6680	17.285	14.6948
0.3526	5.38	15000	1.7372	17.4396	14.8157

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu121
• Datasets 2.14.6
• Tokenizers 0.14.1