PRIMERA-multinews-lora-finetuned

This model is a fine-tuned version of allenai/PRIMERA-multinews on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.6767
  • Rouge1: 13.1661
  • Rouge2: 6.075
  • Rougel: 11.1948
  • Rougelsum: 12.1382
  • Gen Len: 20.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 16
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
2.2385 1.0 4725 1.9183 15.8308 7.6039 12.6624 14.1866 20.0
2.1419 2.0 9450 1.8933 14.8545 6.8893 12.144 13.4623 20.0
2.1286 3.0 14175 1.8619 16.2585 8.1431 13.4226 14.8653 20.0
2.0669 4.0 18900 1.8129 15.9624 7.5293 13.3809 14.7765 20.0
2.0448 5.0 23625 1.7636 16.2801 8.045 13.6178 14.996 20.0
1.9831 6.0 28350 1.7037 13.8735 5.9956 10.8251 12.2545 20.0
1.9926 7.0 33075 1.7623 13.9591 5.8861 11.112 12.4349 20.0
1.99 8.0 37800 1.7247 13.1441 5.2565 10.7117 11.851 20.0
1.9495 9.0 42525 1.7065 12.4863 4.6444 10.0155 11.3874 20.0
1.9782 10.0 47250 1.6919 11.8394 4.0068 9.4554 10.6421 20.0
1.9087 11.0 51975 1.6910 13.011 5.5644 10.7255 11.8532 20.0
1.9693 12.0 56700 1.6872 13.2678 5.7966 11.0537 12.1103 20.0
1.9445 13.0 61425 1.7084 13.2757 5.9337 11.084 12.2354 20.0
1.9467 14.0 66150 1.6729 12.9202 5.424 10.5315 11.6661 20.0
1.9582 15.0 70875 1.6786 13.2851 6.0806 11.291 12.2518 20.0
1.9186 16.0 75600 1.6767 13.1661 6.075 11.1948 12.1382 20.0

Framework versions

  • PEFT 0.12.0
  • Transformers 4.43.2
  • Pytorch 2.2.1+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
0
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for toanduc/PRIMERA-multinews-lora-finetuned

Adapter
(1)
this model