Hugging Face's logo

tanatapanun
/
fine-tuned-BioBART-20-epochs-wang-lab

Text2Text Generation
Transformers
TensorBoard
Safetensors
bart
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
fine-tuned-BioBART-20-epochs-wang-lab / README.md
tanatapanun's picture
tanatapanun
Model save
5929589
preview code
|
raw
history blame contribute delete
3.54 kB
metadata
license: apache-2.0
base_model: GanjinZero/biobart-v2-base
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: fine-tuned-BioBART-20-epochs-wang-lab
    results: []

fine-tuned-BioBART-20-epochs-wang-lab

This model is a fine-tuned version of GanjinZero/biobart-v2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0360
  • Rouge1: 0.308
  • Rouge2: 0.1254
  • Rougel: 0.2791
  • Rougelsum: 0.2797
  • Gen Len: 15.85

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 301 0.7404 0.2534 0.1025 0.2218 0.221 15.29
0.851 2.0 602 0.7099 0.2907 0.1069 0.255 0.2557 15.04
0.851 3.0 903 0.6956 0.2652 0.1049 0.2313 0.23 16.04
0.5436 4.0 1204 0.7288 0.3168 0.129 0.286 0.2872 15.11
0.3356 5.0 1505 0.7856 0.2869 0.1041 0.2569 0.2601 15.4
0.3356 6.0 1806 0.8174 0.2863 0.1214 0.2447 0.2444 15.95
0.1807 7.0 2107 0.8477 0.3048 0.1169 0.2787 0.2797 15.82
0.1807 8.0 2408 0.8816 0.3118 0.1227 0.2726 0.2733 15.66
0.1072 9.0 2709 0.9081 0.2988 0.1169 0.2665 0.2668 14.73
0.0649 10.0 3010 0.9342 0.2869 0.1175 0.2531 0.2535 15.63
0.0649 11.0 3311 0.9588 0.3094 0.1212 0.2722 0.2747 15.77
0.0376 12.0 3612 0.9761 0.3147 0.1197 0.2867 0.2882 15.62
0.0376 13.0 3913 0.9870 0.3144 0.1172 0.2843 0.285 16.09
0.0244 14.0 4214 0.9918 0.3217 0.1267 0.2931 0.2942 15.94
0.0145 15.0 4515 1.0044 0.3102 0.1196 0.2801 0.2815 15.91
0.0145 16.0 4816 1.0152 0.3094 0.1316 0.2796 0.2804 16.01
0.0094 17.0 5117 1.0290 0.317 0.1133 0.2838 0.2857 15.77
0.0094 18.0 5418 1.0337 0.3006 0.1216 0.2712 0.272 15.9
0.0066 19.0 5719 1.0346 0.307 0.1254 0.2785 0.2797 15.85
0.0047 20.0 6020 1.0360 0.308 0.1254 0.2791 0.2797 15.85

Framework versions

  • Transformers 4.36.2
  • Pytorch 1.12.1+cu113
  • Datasets 2.15.0
  • Tokenizers 0.15.0