|
--- |
|
base_model: google/paligemma-3b-pt-224 |
|
library_name: peft |
|
license: gemma |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: paligemmamultidataset |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# paligemmamultidataset |
|
|
|
This model is a fine-tuned version of [google/paligemma-3b-pt-224](https://huggingface.co/google/paligemma-3b-pt-224) on the None dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 2.0804 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 10 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 4 |
|
- total_train_batch_size: 40 |
|
- optimizer: Use OptimizerNames.ADAMW_HF with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 20 |
|
- num_epochs: 12 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-------:|:----:|:---------------:| |
|
| 4.7372 | 0.1233 | 100 | 3.6204 | |
|
| 3.213 | 0.2465 | 200 | 3.0079 | |
|
| 2.6333 | 0.3698 | 300 | 2.6781 | |
|
| 2.4789 | 0.4931 | 400 | 2.5150 | |
|
| 2.2423 | 0.6163 | 500 | 2.4068 | |
|
| 2.1555 | 0.7396 | 600 | 2.2984 | |
|
| 2.1202 | 0.8629 | 700 | 2.2410 | |
|
| 2.1345 | 0.9861 | 800 | 2.1763 | |
|
| 2.0111 | 1.1094 | 900 | 2.1304 | |
|
| 1.9591 | 1.2327 | 1000 | 2.1017 | |
|
| 1.8412 | 1.3559 | 1100 | 2.0653 | |
|
| 1.8451 | 1.4792 | 1200 | 2.0440 | |
|
| 1.8383 | 1.6025 | 1300 | 2.0194 | |
|
| 1.8782 | 1.7257 | 1400 | 1.9879 | |
|
| 1.7502 | 1.8490 | 1500 | 1.9676 | |
|
| 1.7612 | 1.9723 | 1600 | 1.9579 | |
|
| 1.7285 | 2.0955 | 1700 | 1.9335 | |
|
| 1.6545 | 2.2188 | 1800 | 1.9220 | |
|
| 1.6289 | 2.3421 | 1900 | 1.9146 | |
|
| 1.7027 | 2.4653 | 2000 | 1.8895 | |
|
| 1.5917 | 2.5886 | 2100 | 1.8812 | |
|
| 1.5515 | 2.7119 | 2200 | 1.8754 | |
|
| 1.5598 | 2.8351 | 2300 | 1.8583 | |
|
| 1.625 | 2.9584 | 2400 | 1.8443 | |
|
| 1.4844 | 3.0817 | 2500 | 1.8452 | |
|
| 1.4847 | 3.2049 | 2600 | 1.8313 | |
|
| 1.4573 | 3.3282 | 2700 | 1.8216 | |
|
| 1.446 | 3.4515 | 2800 | 1.8087 | |
|
| 1.446 | 3.5747 | 2900 | 1.8062 | |
|
| 1.4052 | 3.6980 | 3000 | 1.8127 | |
|
| 1.4376 | 3.8213 | 3100 | 1.7946 | |
|
| 1.4436 | 3.9445 | 3200 | 1.7834 | |
|
| 1.3534 | 4.0678 | 3300 | 1.8001 | |
|
| 1.3562 | 4.1911 | 3400 | 1.7946 | |
|
| 1.3416 | 4.3143 | 3500 | 1.7894 | |
|
| 1.269 | 4.4376 | 3600 | 1.7802 | |
|
| 1.3105 | 4.5609 | 3700 | 1.7751 | |
|
| 1.3331 | 4.6841 | 3800 | 1.7627 | |
|
| 1.2788 | 4.8074 | 3900 | 1.7766 | |
|
| 1.256 | 4.9307 | 4000 | 1.7723 | |
|
| 1.2342 | 5.0539 | 4100 | 1.7943 | |
|
| 1.1391 | 5.1772 | 4200 | 1.7807 | |
|
| 1.165 | 5.3005 | 4300 | 1.8016 | |
|
| 1.2122 | 5.4237 | 4400 | 1.7840 | |
|
| 1.1536 | 5.5470 | 4500 | 1.7805 | |
|
| 1.217 | 5.6703 | 4600 | 1.7775 | |
|
| 1.1769 | 5.7935 | 4700 | 1.7817 | |
|
| 1.225 | 5.9168 | 4800 | 1.7758 | |
|
| 1.1306 | 6.0401 | 4900 | 1.8010 | |
|
| 1.0248 | 6.1633 | 5000 | 1.8035 | |
|
| 1.036 | 6.2866 | 5100 | 1.8228 | |
|
| 1.1205 | 6.4099 | 5200 | 1.8145 | |
|
| 1.0873 | 6.5331 | 5300 | 1.7970 | |
|
| 1.0785 | 6.6564 | 5400 | 1.8077 | |
|
| 1.0628 | 6.7797 | 5500 | 1.8102 | |
|
| 1.0423 | 6.9029 | 5600 | 1.8027 | |
|
| 1.0392 | 7.0262 | 5700 | 1.8268 | |
|
| 0.9586 | 7.1495 | 5800 | 1.8684 | |
|
| 0.8986 | 7.2727 | 5900 | 1.8406 | |
|
| 0.9616 | 7.3960 | 6000 | 1.8408 | |
|
| 1.026 | 7.5193 | 6100 | 1.8500 | |
|
| 0.9494 | 7.6425 | 6200 | 1.8326 | |
|
| 0.9769 | 7.7658 | 6300 | 1.8455 | |
|
| 0.9322 | 7.8891 | 6400 | 1.8445 | |
|
| 0.9305 | 8.0123 | 6500 | 1.8578 | |
|
| 0.8791 | 8.1356 | 6600 | 1.8941 | |
|
| 0.8852 | 8.2589 | 6700 | 1.9108 | |
|
| 0.8771 | 8.3821 | 6800 | 1.8951 | |
|
| 0.8697 | 8.5054 | 6900 | 1.9177 | |
|
| 0.8676 | 8.6287 | 7000 | 1.9179 | |
|
| 0.8527 | 8.7519 | 7100 | 1.8762 | |
|
| 0.8281 | 8.8752 | 7200 | 1.9050 | |
|
| 0.88 | 8.9985 | 7300 | 1.9411 | |
|
| 0.7569 | 9.1217 | 7400 | 1.9684 | |
|
| 0.7265 | 9.2450 | 7500 | 1.9705 | |
|
| 0.7659 | 9.3683 | 7600 | 1.9602 | |
|
| 0.7858 | 9.4915 | 7700 | 1.9635 | |
|
| 0.7349 | 9.6148 | 7800 | 1.9725 | |
|
| 0.7882 | 9.7381 | 7900 | 1.9662 | |
|
| 0.8001 | 9.8613 | 8000 | 1.9443 | |
|
| 0.8258 | 9.9846 | 8100 | 1.9441 | |
|
| 0.7034 | 10.1079 | 8200 | 2.0047 | |
|
| 0.6926 | 10.2311 | 8300 | 2.0246 | |
|
| 0.6978 | 10.3544 | 8400 | 2.0163 | |
|
| 0.7311 | 10.4777 | 8500 | 2.0388 | |
|
| 0.6835 | 10.6009 | 8600 | 2.0432 | |
|
| 0.7273 | 10.7242 | 8700 | 2.0073 | |
|
| 0.6887 | 10.8475 | 8800 | 2.0164 | |
|
| 0.7032 | 10.9707 | 8900 | 2.0060 | |
|
| 0.6222 | 11.0940 | 9000 | 2.0942 | |
|
| 0.6344 | 11.2173 | 9100 | 2.0941 | |
|
| 0.5905 | 11.3405 | 9200 | 2.1227 | |
|
| 0.6398 | 11.4638 | 9300 | 2.1075 | |
|
| 0.6342 | 11.5871 | 9400 | 2.1051 | |
|
| 0.6245 | 11.7103 | 9500 | 2.0911 | |
|
| 0.6529 | 11.8336 | 9600 | 2.0880 | |
|
| 0.6416 | 11.9569 | 9700 | 2.0804 | |
|
|
|
|
|
### Framework versions |
|
|
|
- PEFT 0.13.0 |
|
- Transformers 4.46.0.dev0 |
|
- Pytorch 2.4.1+cu121 |
|
- Datasets 3.0.1 |
|
- Tokenizers 0.20.0 |
