PRIMERA-lora-finetuned

This model is a fine-tuned version of allenai/PRIMERA on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4326
  • Rouge1: 13.8331
  • Rouge2: 6.379
  • Rougel: 11.3582
  • Rougelsum: 13.0901
  • Gen Len: 19.0217

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 16
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
1.9399 1.0 4725 1.6257 13.1779 5.0715 10.6386 11.9435 19.3478
1.8124 2.0 9450 1.5723 12.8712 5.1519 10.5072 11.9192 19.3478
1.7698 3.0 14175 1.5261 13.6339 5.5927 10.9273 12.3707 19.3478
1.7171 4.0 18900 1.4990 13.0984 6.164 10.9698 12.1863 19.6739
1.7096 5.0 23625 1.4832 12.7164 5.4214 10.3325 11.8134 19.3478
1.6889 6.0 28350 1.4743 13.0677 4.9162 10.3945 11.9299 19.3478
1.6667 7.0 33075 1.4652 13.8986 6.1753 10.9896 12.8072 19.3478
1.6836 8.0 37800 1.4573 13.0179 5.5771 10.4498 12.2198 19.0217
1.6611 9.0 42525 1.4523 12.8773 5.3502 10.294 11.9712 19.0217
1.6842 10.0 47250 1.4503 13.1982 5.0089 10.4547 12.2554 19.0217
1.6373 11.0 51975 1.4432 13.0444 5.533 10.3895 12.3086 19.0217
1.6882 12.0 56700 1.4408 13.7092 5.9873 11.1084 12.8159 19.3478
1.6707 13.0 61425 1.4395 13.6103 6.2231 10.7643 12.7343 19.0217
1.677 14.0 66150 1.4366 13.5232 6.3114 10.9432 12.7234 19.0217
1.6825 15.0 70875 1.4347 13.7907 6.1456 11.0682 12.803 19.0217
1.6516 16.0 75600 1.4326 13.8331 6.379 11.3582 13.0901 19.0217

Framework versions

  • PEFT 0.12.0
  • Transformers 4.43.3
  • Pytorch 2.2.1+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
3
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for toanduc/PRIMERA-lora-finetuned

Base model

allenai/PRIMERA
Adapter
(1)
this model