metadata
license: apache-2.0
base_model: nferruz/ProtGPT2
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: output_hemo_aug_4
results: []
output_hemo_aug_4
This model is a fine-tuned version of nferruz/ProtGPT2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 3.9627
- Accuracy: 0.3978
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 200.0
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 9.4301 | 1.0 | 6 | 8.6845 | 0.0244 |
| 8.4172 | 2.0 | 12 | 7.8149 | 0.0811 |
| 7.6869 | 3.0 | 18 | 7.2012 | 0.1554 |
| 7.1731 | 4.0 | 24 | 6.9139 | 0.1828 |
| 6.8807 | 5.0 | 30 | 6.6238 | 0.1955 |
| 6.6009 | 6.0 | 36 | 6.3847 | 0.1935 |
| 6.4347 | 7.0 | 42 | 6.2341 | 0.2063 |
| 6.2831 | 8.0 | 48 | 6.0964 | 0.2141 |
| 6.1728 | 9.0 | 54 | 5.9864 | 0.2209 |
| 6.0805 | 10.0 | 60 | 5.8936 | 0.2317 |
| 5.9959 | 11.0 | 66 | 5.8161 | 0.2405 |
| 5.925 | 12.0 | 72 | 5.7456 | 0.2385 |
| 5.8787 | 13.0 | 78 | 5.6646 | 0.2483 |
| 5.7996 | 14.0 | 84 | 5.5901 | 0.2493 |
| 5.7312 | 15.0 | 90 | 5.5216 | 0.2532 |
| 5.6751 | 16.0 | 96 | 5.4695 | 0.2590 |
| 5.6076 | 17.0 | 102 | 5.4216 | 0.2620 |
| 5.569 | 18.0 | 108 | 5.3735 | 0.2620 |
| 5.5037 | 19.0 | 114 | 5.3272 | 0.2630 |
| 5.4681 | 20.0 | 120 | 5.2856 | 0.2698 |
| 5.4225 | 21.0 | 126 | 5.2559 | 0.2717 |
| 5.3805 | 22.0 | 132 | 5.2126 | 0.2766 |
| 5.3527 | 23.0 | 138 | 5.1884 | 0.2757 |
| 5.3033 | 24.0 | 144 | 5.1539 | 0.2796 |
| 5.2635 | 25.0 | 150 | 5.1110 | 0.2854 |
| 5.2411 | 26.0 | 156 | 5.0882 | 0.2854 |
| 5.1972 | 27.0 | 162 | 5.0575 | 0.2903 |
| 5.163 | 28.0 | 168 | 5.0293 | 0.2913 |
| 5.1273 | 29.0 | 174 | 5.0047 | 0.2903 |
| 5.1032 | 30.0 | 180 | 4.9817 | 0.2952 |
| 5.0726 | 31.0 | 186 | 4.9583 | 0.2952 |
| 5.0405 | 32.0 | 192 | 4.9355 | 0.2952 |
| 5.007 | 33.0 | 198 | 4.9184 | 0.2952 |
| 4.9897 | 34.0 | 204 | 4.8911 | 0.2972 |
| 4.9416 | 35.0 | 210 | 4.8628 | 0.2972 |
| 4.9245 | 36.0 | 216 | 4.8499 | 0.2981 |
| 4.901 | 37.0 | 222 | 4.8263 | 0.3030 |
| 4.8713 | 38.0 | 228 | 4.8035 | 0.3030 |
| 4.845 | 39.0 | 234 | 4.7874 | 0.3060 |
| 4.8052 | 40.0 | 240 | 4.7535 | 0.3040 |
| 4.7786 | 41.0 | 246 | 4.7313 | 0.3060 |
| 4.7501 | 42.0 | 252 | 4.7175 | 0.3089 |
| 4.7221 | 43.0 | 258 | 4.6978 | 0.3118 |
| 4.7038 | 44.0 | 264 | 4.6785 | 0.3109 |
| 4.681 | 45.0 | 270 | 4.6661 | 0.3128 |
| 4.6566 | 46.0 | 276 | 4.6532 | 0.3157 |
| 4.632 | 47.0 | 282 | 4.6361 | 0.3157 |
| 4.618 | 48.0 | 288 | 4.6162 | 0.3196 |
| 4.5928 | 49.0 | 294 | 4.5987 | 0.3245 |
| 4.5716 | 50.0 | 300 | 4.5848 | 0.3216 |
| 4.5485 | 51.0 | 306 | 4.5721 | 0.3245 |
| 4.5324 | 52.0 | 312 | 4.5579 | 0.3196 |
| 4.5038 | 53.0 | 318 | 4.5423 | 0.3196 |
| 4.4831 | 54.0 | 324 | 4.5240 | 0.3265 |
| 4.4347 | 55.0 | 330 | 4.5087 | 0.3255 |
| 4.4218 | 56.0 | 336 | 4.4850 | 0.3255 |
| 4.3939 | 57.0 | 342 | 4.4791 | 0.3275 |
| 4.3766 | 58.0 | 348 | 4.4640 | 0.3265 |
| 4.3472 | 59.0 | 354 | 4.4471 | 0.3275 |
| 4.3241 | 60.0 | 360 | 4.4334 | 0.3275 |
| 4.2919 | 61.0 | 366 | 4.4296 | 0.3304 |
| 4.2678 | 62.0 | 372 | 4.4281 | 0.3343 |
| 4.2515 | 63.0 | 378 | 4.4120 | 0.3372 |
| 4.2244 | 64.0 | 384 | 4.4038 | 0.3343 |
| 4.2129 | 65.0 | 390 | 4.3826 | 0.3392 |
| 4.1882 | 66.0 | 396 | 4.3834 | 0.3372 |
| 4.1503 | 67.0 | 402 | 4.3738 | 0.3372 |
| 4.1398 | 68.0 | 408 | 4.3596 | 0.3372 |
| 4.115 | 69.0 | 414 | 4.3376 | 0.3412 |
| 4.1052 | 70.0 | 420 | 4.3330 | 0.3412 |
| 4.0932 | 71.0 | 426 | 4.3295 | 0.3412 |
| 4.0573 | 72.0 | 432 | 4.3111 | 0.3412 |
| 4.0449 | 73.0 | 438 | 4.3048 | 0.3441 |
| 4.0165 | 74.0 | 444 | 4.2936 | 0.3460 |
| 3.9936 | 75.0 | 450 | 4.2815 | 0.3509 |
| 3.967 | 76.0 | 456 | 4.2686 | 0.3539 |
| 3.9524 | 77.0 | 462 | 4.2697 | 0.3509 |
| 3.9287 | 78.0 | 468 | 4.2546 | 0.3529 |
| 3.9092 | 79.0 | 474 | 4.2484 | 0.3539 |
| 3.8907 | 80.0 | 480 | 4.2420 | 0.3539 |
| 3.8704 | 81.0 | 486 | 4.2418 | 0.3529 |
| 3.8499 | 82.0 | 492 | 4.2265 | 0.3548 |
| 3.8325 | 83.0 | 498 | 4.2089 | 0.3548 |
| 3.8024 | 84.0 | 504 | 4.2058 | 0.3568 |
| 3.8058 | 85.0 | 510 | 4.2039 | 0.3558 |
| 3.7888 | 86.0 | 516 | 4.1906 | 0.3578 |
| 3.7622 | 87.0 | 522 | 4.1792 | 0.3617 |
| 3.746 | 88.0 | 528 | 4.1819 | 0.3578 |
| 3.7196 | 89.0 | 534 | 4.1789 | 0.3597 |
| 3.7046 | 90.0 | 540 | 4.1610 | 0.3607 |
| 3.7078 | 91.0 | 546 | 4.1515 | 0.3607 |
| 3.6687 | 92.0 | 552 | 4.1752 | 0.3607 |
| 3.6559 | 93.0 | 558 | 4.1287 | 0.3636 |
| 3.6401 | 94.0 | 564 | 4.1569 | 0.3646 |
| 3.6281 | 95.0 | 570 | 4.1234 | 0.3627 |
| 3.5978 | 96.0 | 576 | 4.1270 | 0.3695 |
| 3.5951 | 97.0 | 582 | 4.1188 | 0.3646 |
| 3.5679 | 98.0 | 588 | 4.1282 | 0.3685 |
| 3.5618 | 99.0 | 594 | 4.1089 | 0.3646 |
| 3.5404 | 100.0 | 600 | 4.1090 | 0.3695 |
| 3.5255 | 101.0 | 606 | 4.1039 | 0.3646 |
| 3.5111 | 102.0 | 612 | 4.1010 | 0.3695 |
| 3.5015 | 103.0 | 618 | 4.0889 | 0.3705 |
| 3.493 | 104.0 | 624 | 4.0826 | 0.3705 |
| 3.5643 | 105.0 | 630 | 4.0915 | 0.3754 |
| 3.4543 | 106.0 | 636 | 4.0912 | 0.3724 |
| 3.4517 | 107.0 | 642 | 4.0844 | 0.3754 |
| 3.4387 | 108.0 | 648 | 4.0664 | 0.3754 |
| 3.4274 | 109.0 | 654 | 4.0885 | 0.3763 |
| 3.4241 | 110.0 | 660 | 4.0583 | 0.3793 |
| 3.4016 | 111.0 | 666 | 4.0627 | 0.3803 |
| 3.383 | 112.0 | 672 | 4.0626 | 0.3812 |
| 3.3709 | 113.0 | 678 | 4.0414 | 0.3871 |
| 3.3646 | 114.0 | 684 | 4.0562 | 0.3822 |
| 3.3456 | 115.0 | 690 | 4.0361 | 0.3861 |
| 3.3369 | 116.0 | 696 | 4.0524 | 0.3851 |
| 3.3136 | 117.0 | 702 | 4.0424 | 0.3842 |
| 3.307 | 118.0 | 708 | 4.0477 | 0.3861 |
| 3.2954 | 119.0 | 714 | 4.0287 | 0.3851 |
| 3.2887 | 120.0 | 720 | 4.0392 | 0.3900 |
| 3.2776 | 121.0 | 726 | 4.0191 | 0.3910 |
| 3.2527 | 122.0 | 732 | 4.0339 | 0.3910 |
| 3.259 | 123.0 | 738 | 4.0064 | 0.3930 |
| 3.2559 | 124.0 | 744 | 4.0285 | 0.3881 |
| 3.2335 | 125.0 | 750 | 4.0151 | 0.3930 |
| 3.2318 | 126.0 | 756 | 4.0277 | 0.3900 |
| 3.2266 | 127.0 | 762 | 3.9929 | 0.3978 |
| 3.2051 | 128.0 | 768 | 3.9945 | 0.3978 |
| 3.2009 | 129.0 | 774 | 4.0291 | 0.3930 |
| 3.1791 | 130.0 | 780 | 3.9956 | 0.3930 |
| 3.1759 | 131.0 | 786 | 4.0012 | 0.3969 |
| 3.1622 | 132.0 | 792 | 4.0107 | 0.3949 |
| 3.1559 | 133.0 | 798 | 4.0090 | 0.3939 |
| 3.1521 | 134.0 | 804 | 4.0028 | 0.3910 |
| 3.1353 | 135.0 | 810 | 4.0033 | 0.3939 |
| 3.1427 | 136.0 | 816 | 3.9995 | 0.3939 |
| 3.1276 | 137.0 | 822 | 3.9963 | 0.3920 |
| 3.1228 | 138.0 | 828 | 3.9996 | 0.3978 |
| 3.1039 | 139.0 | 834 | 3.9928 | 0.3988 |
| 3.097 | 140.0 | 840 | 3.9969 | 0.3969 |
| 3.083 | 141.0 | 846 | 3.9918 | 0.3949 |
| 3.0844 | 142.0 | 852 | 3.9900 | 0.3969 |
| 3.077 | 143.0 | 858 | 3.9812 | 0.3959 |
| 3.0601 | 144.0 | 864 | 3.9948 | 0.3959 |
| 3.0669 | 145.0 | 870 | 3.9938 | 0.3959 |
| 3.0515 | 146.0 | 876 | 3.9895 | 0.3978 |
| 3.0405 | 147.0 | 882 | 3.9803 | 0.3988 |
| 3.029 | 148.0 | 888 | 3.9856 | 0.3969 |
| 3.0342 | 149.0 | 894 | 3.9828 | 0.3969 |
| 3.0137 | 150.0 | 900 | 3.9977 | 0.3978 |
| 3.0277 | 151.0 | 906 | 3.9793 | 0.3998 |
| 3.0005 | 152.0 | 912 | 3.9779 | 0.3998 |
| 3.0027 | 153.0 | 918 | 3.9891 | 0.3988 |
| 3.0034 | 154.0 | 924 | 3.9687 | 0.4008 |
| 2.9853 | 155.0 | 930 | 3.9887 | 0.3978 |
| 2.9947 | 156.0 | 936 | 3.9860 | 0.4027 |
| 2.9768 | 157.0 | 942 | 3.9900 | 0.4027 |
| 2.9752 | 158.0 | 948 | 3.9993 | 0.3988 |
| 2.9773 | 159.0 | 954 | 3.9694 | 0.4018 |
| 2.9662 | 160.0 | 960 | 3.9924 | 0.3998 |
| 2.9661 | 161.0 | 966 | 4.0089 | 0.3988 |
| 2.9488 | 162.0 | 972 | 3.9749 | 0.3988 |
| 2.9487 | 163.0 | 978 | 3.9932 | 0.3978 |
| 2.9482 | 164.0 | 984 | 3.9987 | 0.3988 |
| 2.9624 | 165.0 | 990 | 3.9627 | 0.3978 |
| 2.9524 | 166.0 | 996 | 3.9791 | 0.4008 |
| 2.9357 | 167.0 | 1002 | 3.9969 | 0.3998 |
| 2.9323 | 168.0 | 1008 | 3.9854 | 0.4008 |
| 2.9334 | 169.0 | 1014 | 3.9778 | 0.4008 |
| 2.9228 | 170.0 | 1020 | 3.9859 | 0.4027 |
| 2.9305 | 171.0 | 1026 | 3.9821 | 0.4037 |
| 2.9239 | 172.0 | 1032 | 3.9876 | 0.4066 |
| 2.9181 | 173.0 | 1038 | 3.9792 | 0.4057 |
| 2.9162 | 174.0 | 1044 | 3.9731 | 0.4037 |
| 2.9171 | 175.0 | 1050 | 3.9796 | 0.4066 |
| 2.9132 | 176.0 | 1056 | 3.9914 | 0.4047 |
| 2.9168 | 177.0 | 1062 | 3.9826 | 0.4047 |
| 2.8974 | 178.0 | 1068 | 3.9753 | 0.4057 |
| 2.8954 | 179.0 | 1074 | 3.9766 | 0.4057 |
| 2.9003 | 180.0 | 1080 | 3.9865 | 0.4027 |
| 2.9012 | 181.0 | 1086 | 3.9835 | 0.4047 |
| 2.8994 | 182.0 | 1092 | 3.9802 | 0.4047 |
| 2.8918 | 183.0 | 1098 | 3.9811 | 0.4066 |
| 2.8893 | 184.0 | 1104 | 3.9810 | 0.4057 |
| 2.8865 | 185.0 | 1110 | 3.9852 | 0.4076 |
| 2.8784 | 186.0 | 1116 | 3.9805 | 0.4057 |
| 2.8875 | 187.0 | 1122 | 3.9781 | 0.4066 |
| 2.8948 | 188.0 | 1128 | 3.9831 | 0.4057 |
| 2.8927 | 189.0 | 1134 | 3.9837 | 0.4066 |
| 2.8739 | 190.0 | 1140 | 3.9822 | 0.4057 |
| 2.8919 | 191.0 | 1146 | 3.9792 | 0.4066 |
| 2.8713 | 192.0 | 1152 | 3.9800 | 0.4057 |
| 2.8798 | 193.0 | 1158 | 3.9854 | 0.4047 |
| 2.8835 | 194.0 | 1164 | 3.9845 | 0.4057 |
| 2.878 | 195.0 | 1170 | 3.9820 | 0.4057 |
| 2.8931 | 196.0 | 1176 | 3.9816 | 0.4057 |
| 2.8662 | 197.0 | 1182 | 3.9830 | 0.4057 |
| 2.8734 | 198.0 | 1188 | 3.9841 | 0.4057 |
| 2.8825 | 199.0 | 1194 | 3.9830 | 0.4057 |
| 2.8825 | 200.0 | 1200 | 3.9827 | 0.4057 |
Framework versions
- Transformers 4.38.0.dev0
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1