|
--- |
|
library_name: transformers |
|
base_model: google/pegasus-cnn_dailymail |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- samsum |
|
model-index: |
|
- name: pegasus-samsum |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# pegasus-samsum |
|
|
|
This model is a fine-tuned version of [google/pegasus-cnn_dailymail](https://huggingface.co/google/pegasus-cnn_dailymail) on the samsum dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 6.0105 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 1 |
|
- eval_batch_size: 1 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 16 |
|
- total_train_batch_size: 16 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 500 |
|
- num_epochs: 1 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:------:|:----:|:---------------:| |
|
| 10.6079 | 0.0109 | 10 | 10.7159 | |
|
| 10.5191 | 0.0217 | 20 | 10.6397 | |
|
| 10.5539 | 0.0326 | 30 | 10.5210 | |
|
| 10.5117 | 0.0434 | 40 | 10.3669 | |
|
| 10.3111 | 0.0543 | 50 | 10.1941 | |
|
| 10.2 | 0.0652 | 60 | 10.0230 | |
|
| 10.1121 | 0.0760 | 70 | 9.8584 | |
|
| 10.0677 | 0.0869 | 80 | 9.7129 | |
|
| 9.7897 | 0.0977 | 90 | 9.5781 | |
|
| 9.744 | 0.1086 | 100 | 9.4538 | |
|
| 9.533 | 0.1195 | 110 | 9.3431 | |
|
| 9.5248 | 0.1303 | 120 | 9.2552 | |
|
| 9.3331 | 0.1412 | 130 | 9.1575 | |
|
| 9.2551 | 0.1520 | 140 | 9.0751 | |
|
| 9.2382 | 0.1629 | 150 | 8.9993 | |
|
| 9.1323 | 0.1738 | 160 | 8.9287 | |
|
| 9.0574 | 0.1846 | 170 | 8.8628 | |
|
| 9.0137 | 0.1955 | 180 | 8.7964 | |
|
| 8.9097 | 0.2064 | 190 | 8.7340 | |
|
| 8.8268 | 0.2172 | 200 | 8.6765 | |
|
| 8.7116 | 0.2281 | 210 | 8.6173 | |
|
| 8.7483 | 0.2389 | 220 | 8.5521 | |
|
| 8.6252 | 0.2498 | 230 | 8.4884 | |
|
| 8.5844 | 0.2607 | 240 | 8.4275 | |
|
| 8.4614 | 0.2715 | 250 | 8.3626 | |
|
| 8.4375 | 0.2824 | 260 | 8.2901 | |
|
| 8.445 | 0.2932 | 270 | 8.2102 | |
|
| 8.2966 | 0.3041 | 280 | 8.1135 | |
|
| 8.0934 | 0.3150 | 290 | 8.0113 | |
|
| 8.1551 | 0.3258 | 300 | 7.8961 | |
|
| 8.0542 | 0.3367 | 310 | 7.7751 | |
|
| 8.0183 | 0.3475 | 320 | 7.6504 | |
|
| 7.9352 | 0.3584 | 330 | 7.5371 | |
|
| 7.7615 | 0.3693 | 340 | 7.4084 | |
|
| 7.6484 | 0.3801 | 350 | 7.2715 | |
|
| 7.5845 | 0.3910 | 360 | 7.1340 | |
|
| 7.4799 | 0.4018 | 370 | 7.0563 | |
|
| 7.3388 | 0.4127 | 380 | 6.9495 | |
|
| 7.1078 | 0.4236 | 390 | 6.8582 | |
|
| 7.0819 | 0.4344 | 400 | 6.7707 | |
|
| 7.0465 | 0.4453 | 410 | 6.6897 | |
|
| 6.9038 | 0.4561 | 420 | 6.6184 | |
|
| 6.9359 | 0.4670 | 430 | 6.5533 | |
|
| 6.8038 | 0.4779 | 440 | 6.4962 | |
|
| 6.8648 | 0.4887 | 450 | 6.4452 | |
|
| 6.7589 | 0.4996 | 460 | 6.3966 | |
|
| 6.6804 | 0.5105 | 470 | 6.3588 | |
|
| 6.5603 | 0.5213 | 480 | 6.3247 | |
|
| 6.6002 | 0.5322 | 490 | 6.2937 | |
|
| 6.59 | 0.5430 | 500 | 6.2665 | |
|
| 6.5103 | 0.5539 | 510 | 6.2434 | |
|
| 6.4911 | 0.5648 | 520 | 6.2210 | |
|
| 6.4606 | 0.5756 | 530 | 6.2054 | |
|
| 6.5193 | 0.5865 | 540 | 6.1870 | |
|
| 6.4794 | 0.5973 | 550 | 6.1724 | |
|
| 6.4579 | 0.6082 | 560 | 6.1598 | |
|
| 6.3855 | 0.6191 | 570 | 6.1482 | |
|
| 6.3071 | 0.6299 | 580 | 6.1367 | |
|
| 6.4043 | 0.6408 | 590 | 6.1279 | |
|
| 6.354 | 0.6516 | 600 | 6.1188 | |
|
| 6.4038 | 0.6625 | 610 | 6.1114 | |
|
| 6.3475 | 0.6734 | 620 | 6.1008 | |
|
| 6.257 | 0.6842 | 630 | 6.0958 | |
|
| 6.4359 | 0.6951 | 640 | 6.0872 | |
|
| 6.2238 | 0.7059 | 650 | 6.0820 | |
|
| 6.3904 | 0.7168 | 660 | 6.0754 | |
|
| 6.2488 | 0.7277 | 670 | 6.0706 | |
|
| 6.2648 | 0.7385 | 680 | 6.0644 | |
|
| 6.303 | 0.7494 | 690 | 6.0601 | |
|
| 6.3133 | 0.7602 | 700 | 6.0553 | |
|
| 6.3229 | 0.7711 | 710 | 6.0516 | |
|
| 6.3165 | 0.7820 | 720 | 6.0469 | |
|
| 6.3353 | 0.7928 | 730 | 6.0438 | |
|
| 6.2581 | 0.8037 | 740 | 6.0391 | |
|
| 6.2688 | 0.8146 | 750 | 6.0361 | |
|
| 6.2193 | 0.8254 | 760 | 6.0342 | |
|
| 6.2247 | 0.8363 | 770 | 6.0305 | |
|
| 6.1711 | 0.8471 | 780 | 6.0284 | |
|
| 6.3126 | 0.8580 | 790 | 6.0259 | |
|
| 6.3182 | 0.8689 | 800 | 6.0239 | |
|
| 6.2298 | 0.8797 | 810 | 6.0214 | |
|
| 6.287 | 0.8906 | 820 | 6.0198 | |
|
| 6.2472 | 0.9014 | 830 | 6.0181 | |
|
| 6.205 | 0.9123 | 840 | 6.0165 | |
|
| 6.2359 | 0.9232 | 850 | 6.0147 | |
|
| 6.3013 | 0.9340 | 860 | 6.0135 | |
|
| 6.2035 | 0.9449 | 870 | 6.0129 | |
|
| 6.2529 | 0.9557 | 880 | 6.0122 | |
|
| 6.2043 | 0.9666 | 890 | 6.0114 | |
|
| 6.2785 | 0.9775 | 900 | 6.0110 | |
|
| 6.3018 | 0.9883 | 910 | 6.0106 | |
|
| 6.1616 | 0.9992 | 920 | 6.0105 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.44.2 |
|
- Pytorch 2.5.0+cu121 |
|
- Datasets 3.1.0 |
|
- Tokenizers 0.19.1 |
|
|
