mBART-TextSimp-LT-BatchSize4-lr1e-4

This model is a fine-tuned version of facebook/mbart-large-50 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0810
• Rouge1: 0.8026
• Rouge2: 0.6711
• Rougel: 0.796
• Sacrebleu: 55.7975
• Gen Len: 33.3938

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Sacrebleu	Gen Len
0.0978	1.0	209	0.0948	0.6734	0.5117	0.664	43.1877	33.3938
0.0617	2.0	418	0.0598	0.7702	0.6222	0.7609	49.5794	33.3938
1.1021	3.0	627	0.8158	0.0161	0.0	0.0162	0.0009	34.3938
0.0471	4.0	836	0.0822	0.6874	0.5335	0.6779	44.9573	33.3938
0.0276	5.0	1045	0.0664	0.7767	0.6339	0.7686	52.2135	33.3938
0.0162	6.0	1254	0.0756	0.7856	0.6452	0.7796	50.4352	33.3938
0.0069	7.0	1463	0.0796	0.7939	0.6586	0.7877	52.9489	33.3938
0.0051	8.0	1672	0.0810	0.8026	0.6711	0.796	55.7975	33.3938

Framework versions

• Transformers 4.33.0
• Pytorch 2.1.2+cu121
• Datasets 2.14.4
• Tokenizers 0.13.3