Hugging Face's logo

brandonRivas
/
roberta-large-ner-qlorafinetune-runs-colab

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
roberta-large-ner-qlorafinetune-runs-colab / README.md
brandonRivas's picture
brandonRivas
End of training
a6bb91d verified
preview code
|
raw
history blame
10.2 kB
metadata
library_name: peft
license: mit
base_model: FacebookAI/xlm-roberta-large
tags:
  - generated_from_trainer
datasets:
  - biobert_json
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: roberta-large-ner-qlorafinetune-runs-colab
    results: []

roberta-large-ner-qlorafinetune-runs-colab

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on the biobert_json dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0747
  • Precision: 0.9341
  • Recall: 0.9552
  • F1: 0.9446
  • Accuracy: 0.9805

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0004
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • training_steps: 1820
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
2.3591 0.0654 20 1.1618 0.0 0.0 0.0 0.7196
1.016 0.1307 40 0.5998 0.5713 0.5601 0.5657 0.8528
0.5032 0.1961 60 0.2553 0.8199 0.8139 0.8169 0.9357
0.3019 0.2614 80 0.1630 0.8376 0.8817 0.8591 0.9522
0.2149 0.3268 100 0.1543 0.8341 0.9190 0.8745 0.9557
0.2131 0.3922 120 0.1331 0.8587 0.9157 0.8863 0.9592
0.1722 0.4575 140 0.1217 0.9150 0.8981 0.9065 0.9660
0.1646 0.5229 160 0.1060 0.8982 0.9465 0.9217 0.9711
0.1663 0.5882 180 0.1212 0.8900 0.9454 0.9169 0.9684
0.1316 0.6536 200 0.0979 0.9071 0.9409 0.9237 0.9713
0.1342 0.7190 220 0.0846 0.9123 0.9382 0.9251 0.9743
0.1334 0.7843 240 0.0919 0.9141 0.9491 0.9313 0.9743
0.1124 0.8497 260 0.0927 0.9143 0.9405 0.9272 0.9735
0.124 0.9150 280 0.0989 0.9047 0.9457 0.9247 0.9710
0.1295 0.9804 300 0.0898 0.9121 0.9302 0.9211 0.9730
0.1102 1.0458 320 0.0894 0.9060 0.9417 0.9235 0.9730
0.0943 1.1111 340 0.1187 0.9132 0.9434 0.9281 0.9722
0.1055 1.1765 360 0.1075 0.8977 0.9473 0.9218 0.9690
0.1003 1.2418 380 0.0876 0.9193 0.9471 0.9330 0.9754
0.1073 1.3072 400 0.0774 0.9259 0.9526 0.9391 0.9784
0.1176 1.3725 420 0.0986 0.8830 0.9363 0.9089 0.9692
0.1028 1.4379 440 0.0778 0.9318 0.9504 0.9410 0.9785
0.0953 1.5033 460 0.0747 0.9294 0.9527 0.9409 0.9783
0.1023 1.5686 480 0.0757 0.9263 0.9532 0.9395 0.9779
0.0865 1.6340 500 0.0884 0.9057 0.9469 0.9258 0.9738
0.0744 1.6993 520 0.0889 0.9045 0.9654 0.9339 0.9752
0.0779 1.7647 540 0.0792 0.9176 0.9402 0.9288 0.9761
0.0946 1.8301 560 0.0821 0.9139 0.9548 0.9339 0.9761
0.0966 1.8954 580 0.1007 0.9029 0.9552 0.9283 0.9715
0.098 1.9608 600 0.0960 0.9137 0.9460 0.9296 0.9737
0.0829 2.0261 620 0.0859 0.9232 0.9514 0.9371 0.9766
0.077 2.0915 640 0.0821 0.9196 0.9575 0.9382 0.9768
0.0598 2.1569 660 0.0756 0.9226 0.9490 0.9356 0.9771
0.0864 2.2222 680 0.0768 0.9240 0.9560 0.9397 0.9777
0.0836 2.2876 700 0.0846 0.9175 0.9577 0.9372 0.9762
0.0629 2.3529 720 0.0706 0.9355 0.9513 0.9433 0.9796
0.0667 2.4183 740 0.0781 0.9273 0.9527 0.9398 0.9779
0.09 2.4837 760 0.0728 0.9297 0.9615 0.9453 0.9802
0.0718 2.5490 780 0.0772 0.9338 0.9609 0.9471 0.9808
0.0765 2.6144 800 0.0712 0.9403 0.9558 0.9480 0.9810
0.0632 2.6797 820 0.0759 0.9377 0.9512 0.9444 0.9795
0.0667 2.7451 840 0.0787 0.9168 0.9484 0.9323 0.9774
0.0652 2.8105 860 0.0754 0.9320 0.9587 0.9452 0.9804
0.0704 2.8758 880 0.0715 0.9335 0.9576 0.9454 0.9802
0.0583 2.9412 900 0.0744 0.9227 0.9587 0.9404 0.9789
0.0589 3.0065 920 0.0710 0.9334 0.9610 0.9470 0.9810
0.0489 3.0719 940 0.0720 0.9342 0.9583 0.9461 0.9809
0.0545 3.1373 960 0.0886 0.9170 0.9551 0.9357 0.9765
0.0491 3.2026 980 0.0738 0.9353 0.9551 0.9451 0.9802
0.0563 3.2680 1000 0.0741 0.9308 0.9544 0.9424 0.9794
0.0527 3.3333 1020 0.0755 0.9257 0.9525 0.9389 0.9785
0.0525 3.3987 1040 0.0800 0.9218 0.9655 0.9432 0.9790
0.0599 3.4641 1060 0.0838 0.9190 0.9543 0.9363 0.9777
0.0474 3.5294 1080 0.0770 0.9267 0.9497 0.9381 0.9790
0.0508 3.5948 1100 0.0821 0.9293 0.9577 0.9433 0.9780
0.048 3.6601 1120 0.0739 0.9360 0.9527 0.9443 0.9802
0.0592 3.7255 1140 0.0733 0.9374 0.9611 0.9491 0.9810
0.0546 3.7908 1160 0.0721 0.9389 0.9575 0.9481 0.9814
0.0668 3.8562 1180 0.0773 0.9278 0.9581 0.9427 0.9792
0.0626 3.9216 1200 0.0816 0.9280 0.9567 0.9421 0.9783
0.0664 3.9869 1220 0.0720 0.9332 0.9577 0.9453 0.9806
0.0465 4.0523 1240 0.0743 0.9350 0.9579 0.9463 0.9804
0.0389 4.1176 1260 0.0742 0.9279 0.9583 0.9429 0.9802
0.0412 4.1830 1280 0.0720 0.9382 0.9572 0.9476 0.9816
0.0451 4.2484 1300 0.0749 0.9327 0.9514 0.9419 0.9799
0.0455 4.3137 1320 0.0732 0.9399 0.9494 0.9446 0.9807
0.0474 4.3791 1340 0.0731 0.9349 0.9569 0.9458 0.9805
0.0414 4.4444 1360 0.0749 0.9312 0.9558 0.9434 0.9800
0.0338 4.5098 1380 0.0774 0.9305 0.9576 0.9438 0.9799
0.0495 4.5752 1400 0.0774 0.9286 0.9501 0.9392 0.9794
0.0442 4.6405 1420 0.0751 0.9370 0.9611 0.9489 0.9813
0.0376 4.7059 1440 0.0819 0.9255 0.9532 0.9392 0.9778
0.0466 4.7712 1460 0.0768 0.9373 0.9566 0.9468 0.9805
0.0407 4.8366 1480 0.0763 0.9359 0.9574 0.9465 0.9803
0.0419 4.9020 1500 0.0769 0.9310 0.9570 0.9438 0.9796
0.0521 4.9673 1520 0.0743 0.9336 0.9522 0.9428 0.9796
0.0356 5.0327 1540 0.0793 0.9271 0.9544 0.9406 0.9788
0.0359 5.0980 1560 0.0744 0.9386 0.9574 0.9479 0.9807
0.0411 5.1634 1580 0.0749 0.9354 0.9592 0.9472 0.9806
0.036 5.2288 1600 0.0764 0.9322 0.9518 0.9419 0.9799
0.0316 5.2941 1620 0.0777 0.9287 0.9531 0.9408 0.9788
0.0314 5.3595 1640 0.0756 0.9330 0.9568 0.9447 0.9802
0.0321 5.4248 1660 0.0727 0.9405 0.9569 0.9486 0.9815
0.0427 5.4902 1680 0.0728 0.9381 0.9570 0.9474 0.9810
0.031 5.5556 1700 0.0718 0.9414 0.9573 0.9493 0.9817
0.033 5.6209 1720 0.0752 0.9324 0.9562 0.9442 0.9800
0.0342 5.6863 1740 0.0742 0.9367 0.9545 0.9455 0.9804
0.0348 5.7516 1760 0.0748 0.9328 0.9554 0.9439 0.9800
0.0295 5.8170 1780 0.0750 0.9326 0.9561 0.9442 0.9802
0.035 5.8824 1800 0.0752 0.9328 0.9560 0.9443 0.9803
0.0285 5.9477 1820 0.0747 0.9341 0.9552 0.9446 0.9805

Framework versions

  • PEFT 0.13.2
  • Transformers 4.46.3
  • Pytorch 2.5.1+cu121
  • Datasets 3.2.0
  • Tokenizers 0.20.3