File size: 4,738 Bytes
4e254e3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
---
library_name: peft
license: bsd-3-clause
base_model: hugohrban/progen2-base
tags:
- generated_from_trainer
model-index:
- name: Progen2_Kinase_PhosphositeGen
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# Progen2_Kinase_PhosphositeGen
This model is a fine-tuned version of [hugohrban/progen2-base](https://huggingface.co/hugohrban/progen2-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0025
- Perplexity: 7.4078
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- training_steps: 5000
### Training results
| Training Loss | Epoch | Step | Validation Loss | Perplexity |
|:-------------:|:------:|:----:|:---------------:|:----------:|
| 4.9303 | 0.1415 | 100 | 2.1811 | 8.8564 |
| 4.2874 | 0.2831 | 200 | 2.1198 | 8.3296 |
| 4.1941 | 0.4246 | 300 | 2.0694 | 7.9202 |
| 4.0803 | 0.5662 | 400 | 2.0362 | 7.6616 |
| 4.0613 | 0.7077 | 500 | 2.0053 | 7.4284 |
| 3.9511 | 0.8493 | 600 | 1.9922 | 7.3315 |
| 3.9216 | 0.9908 | 700 | 1.9477 | 7.0124 |
| 3.5053 | 1.1316 | 800 | 1.9200 | 6.8208 |
| 3.4311 | 1.2732 | 900 | 1.9035 | 6.7094 |
| 3.4238 | 1.4147 | 1000 | 1.8714 | 6.4974 |
| 3.392 | 1.5563 | 1100 | 1.8527 | 6.3772 |
| 3.3621 | 1.6978 | 1200 | 1.8317 | 6.2444 |
| 3.3577 | 1.8393 | 1300 | 1.8237 | 6.1945 |
| 3.3419 | 1.9809 | 1400 | 1.7889 | 5.9826 |
| 2.8256 | 2.1217 | 1500 | 1.7977 | 6.0356 |
| 2.8061 | 2.2633 | 1600 | 1.7860 | 5.9653 |
| 2.7837 | 2.4048 | 1700 | 1.7666 | 5.8507 |
| 2.7504 | 2.5464 | 1800 | 1.7428 | 5.7133 |
| 2.829 | 2.6879 | 1900 | 1.7288 | 5.6337 |
| 2.7567 | 2.8294 | 2000 | 1.7088 | 5.5225 |
| 2.7443 | 2.9710 | 2100 | 1.6986 | 5.4664 |
| 2.3409 | 3.1118 | 2200 | 1.7382 | 5.6869 |
| 2.2568 | 3.2534 | 2300 | 1.7487 | 5.7471 |
| 2.2481 | 3.3949 | 2400 | 1.7181 | 5.5740 |
| 2.2323 | 3.5364 | 2500 | 1.7058 | 5.5059 |
| 2.2654 | 3.6780 | 2600 | 1.7031 | 5.4912 |
| 2.2611 | 3.8195 | 2700 | 1.6707 | 5.3157 |
| 2.256 | 3.9611 | 2800 | 1.6719 | 5.3222 |
| 1.8849 | 4.1019 | 2900 | 1.7899 | 5.9886 |
| 1.771 | 4.2435 | 3000 | 1.7697 | 5.8694 |
| 1.7992 | 4.3850 | 3100 | 1.7880 | 5.9775 |
| 1.838 | 4.5265 | 3200 | 1.7871 | 5.9722 |
| 1.8285 | 4.6681 | 3300 | 1.7342 | 5.6644 |
| 1.8127 | 4.8096 | 3400 | 1.7196 | 5.5825 |
| 1.8353 | 4.9512 | 3500 | 1.7471 | 5.7377 |
| 1.5511 | 5.0920 | 3600 | 1.8285 | 6.2248 |
| 1.4449 | 5.2335 | 3700 | 1.8683 | 6.4770 |
| 1.4631 | 5.3751 | 3800 | 1.8880 | 6.6063 |
| 1.4525 | 5.5166 | 3900 | 1.8807 | 6.5581 |
| 1.4516 | 5.6582 | 4000 | 1.8723 | 6.5031 |
| 1.4423 | 5.7997 | 4100 | 1.8828 | 6.5716 |
| 1.4626 | 5.9413 | 4200 | 1.8535 | 6.3824 |
| 1.3065 | 6.0821 | 4300 | 1.9369 | 6.9369 |
| 1.1889 | 6.2236 | 4400 | 1.9767 | 7.2191 |
| 1.1865 | 6.3652 | 4500 | 1.9845 | 7.2752 |
| 1.1927 | 6.5067 | 4600 | 2.0029 | 7.4103 |
| 1.1937 | 6.6483 | 4700 | 1.9931 | 7.3380 |
| 1.1893 | 6.7898 | 4800 | 1.9814 | 7.2532 |
| 1.1654 | 6.9314 | 4900 | 1.9931 | 7.3383 |
| 1.1036 | 7.0722 | 5000 | 2.0025 | 7.4078 |
### Framework versions
- PEFT 0.13.2
- Transformers 4.47.1
- Pytorch 2.1.0.post301
- Datasets 3.0.2
- Tokenizers 0.21.0 |