metadata
library_name: peft
license: bsd-3-clause
base_model: hugohrban/progen2-base
tags:
- generated_from_trainer
model-index:
- name: Progen2_Kinase_PhosphositeGen_dkz_trainwithunlabeled
results: []
Progen2_Kinase_PhosphositeGen_dkz_trainwithunlabeled
This model is a fine-tuned version of hugohrban/progen2-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.2967
- Perplexity: 9.9412
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- training_steps: 5000
Training results
| Training Loss | Epoch | Step | Validation Loss | Perplexity |
|---|---|---|---|---|
| 4.763 | 0.0905 | 100 | 2.1876 | 8.9135 |
| 4.6193 | 0.1811 | 200 | 2.9409 | 18.9328 |
| 5.017 | 0.2716 | 300 | 2.4134 | 11.1717 |
| 4.7964 | 0.3622 | 400 | 2.3927 | 10.9434 |
| 4.7938 | 0.4527 | 500 | 2.3926 | 10.9421 |
| 4.7905 | 0.5432 | 600 | 2.3904 | 10.9178 |
| 4.7763 | 0.6338 | 700 | 2.3880 | 10.8919 |
| 4.7542 | 0.7243 | 800 | 2.3813 | 10.8193 |
| 4.763 | 0.8148 | 900 | 2.3772 | 10.7748 |
| 4.7519 | 0.9054 | 1000 | 2.3704 | 10.7016 |
| 4.7497 | 0.9959 | 1100 | 2.3618 | 10.6099 |
| 4.7085 | 1.0860 | 1200 | 2.3547 | 10.5355 |
| 4.7148 | 1.1766 | 1300 | 2.3564 | 10.5527 |
| 4.6749 | 1.2671 | 1400 | 2.3437 | 10.4194 |
| 4.6825 | 1.3576 | 1500 | 2.3395 | 10.3756 |
| 4.668 | 1.4482 | 1600 | 2.3389 | 10.3699 |
| 4.6826 | 1.5387 | 1700 | 2.3344 | 10.3232 |
| 4.6505 | 1.6292 | 1800 | 2.3321 | 10.3000 |
| 4.6549 | 1.7198 | 1900 | 2.3267 | 10.2445 |
| 4.6448 | 1.8103 | 2000 | 2.3268 | 10.2450 |
| 4.6368 | 1.9009 | 2100 | 2.3274 | 10.2516 |
| 4.6255 | 1.9914 | 2200 | 2.3308 | 10.2862 |
| 4.6042 | 2.0815 | 2300 | 2.3210 | 10.1855 |
| 4.6325 | 2.1720 | 2400 | 2.3211 | 10.1865 |
| 4.6238 | 2.2626 | 2500 | 2.3210 | 10.1856 |
| 4.6272 | 2.3531 | 2600 | 2.3153 | 10.1279 |
| 4.6027 | 2.4436 | 2700 | 2.3154 | 10.1287 |
| 4.6121 | 2.5342 | 2800 | 2.3122 | 10.0962 |
| 4.6061 | 2.6247 | 2900 | 2.3110 | 10.0841 |
| 4.6195 | 2.7153 | 3000 | 2.3131 | 10.1058 |
| 4.6046 | 2.8058 | 3100 | 2.3089 | 10.0634 |
| 4.6049 | 2.8963 | 3200 | 2.3133 | 10.1074 |
| 4.6221 | 2.9869 | 3300 | 2.3119 | 10.0932 |
| 4.5677 | 3.0770 | 3400 | 2.3087 | 10.0615 |
| 4.5952 | 3.1675 | 3500 | 2.3085 | 10.0590 |
| 4.5809 | 3.2580 | 3600 | 2.3084 | 10.0582 |
| 4.5803 | 3.3486 | 3700 | 2.3076 | 10.0507 |
| 4.5857 | 3.4391 | 3800 | 2.3077 | 10.0512 |
| 4.585 | 3.5297 | 3900 | 2.3073 | 10.0476 |
| 4.5868 | 3.6202 | 4000 | 2.3032 | 10.0060 |
| 4.5978 | 3.7107 | 4100 | 2.3047 | 10.0213 |
| 4.5732 | 3.8013 | 4200 | 2.3017 | 9.9909 |
| 4.5759 | 3.8918 | 4300 | 2.3021 | 9.9951 |
| 4.5808 | 3.9823 | 4400 | 2.3005 | 9.9793 |
| 4.5465 | 4.0724 | 4500 | 2.3014 | 9.9880 |
| 4.5563 | 4.1630 | 4600 | 2.3004 | 9.9784 |
| 4.5592 | 4.2535 | 4700 | 2.2992 | 9.9659 |
| 4.5596 | 4.3440 | 4800 | 2.2979 | 9.9532 |
| 4.5683 | 4.4346 | 4900 | 2.2969 | 9.9432 |
| 4.5703 | 4.5251 | 5000 | 2.2967 | 9.9412 |
Framework versions
- PEFT 0.13.2
- Transformers 4.47.1
- Pytorch 2.1.0.post301
- Datasets 3.0.2
- Tokenizers 0.21.0