--- base_model: google/paligemma-3b-pt-224 library_name: peft license: gemma tags: - generated_from_trainer model-index: - name: paligemmamultidataset results: [] --- # paligemmamultidataset This model is a fine-tuned version of [google/paligemma-3b-pt-224](https://huggingface.co/google/paligemma-3b-pt-224) on the None dataset. It achieves the following results on the evaluation set: - Loss: 2.0804 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 10 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 40 - optimizer: Use OptimizerNames.ADAMW_HF with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 20 - num_epochs: 12 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-------:|:----:|:---------------:| | 4.7372 | 0.1233 | 100 | 3.6204 | | 3.213 | 0.2465 | 200 | 3.0079 | | 2.6333 | 0.3698 | 300 | 2.6781 | | 2.4789 | 0.4931 | 400 | 2.5150 | | 2.2423 | 0.6163 | 500 | 2.4068 | | 2.1555 | 0.7396 | 600 | 2.2984 | | 2.1202 | 0.8629 | 700 | 2.2410 | | 2.1345 | 0.9861 | 800 | 2.1763 | | 2.0111 | 1.1094 | 900 | 2.1304 | | 1.9591 | 1.2327 | 1000 | 2.1017 | | 1.8412 | 1.3559 | 1100 | 2.0653 | | 1.8451 | 1.4792 | 1200 | 2.0440 | | 1.8383 | 1.6025 | 1300 | 2.0194 | | 1.8782 | 1.7257 | 1400 | 1.9879 | | 1.7502 | 1.8490 | 1500 | 1.9676 | | 1.7612 | 1.9723 | 1600 | 1.9579 | | 1.7285 | 2.0955 | 1700 | 1.9335 | | 1.6545 | 2.2188 | 1800 | 1.9220 | | 1.6289 | 2.3421 | 1900 | 1.9146 | | 1.7027 | 2.4653 | 2000 | 1.8895 | | 1.5917 | 2.5886 | 2100 | 1.8812 | | 1.5515 | 2.7119 | 2200 | 1.8754 | | 1.5598 | 2.8351 | 2300 | 1.8583 | | 1.625 | 2.9584 | 2400 | 1.8443 | | 1.4844 | 3.0817 | 2500 | 1.8452 | | 1.4847 | 3.2049 | 2600 | 1.8313 | | 1.4573 | 3.3282 | 2700 | 1.8216 | | 1.446 | 3.4515 | 2800 | 1.8087 | | 1.446 | 3.5747 | 2900 | 1.8062 | | 1.4052 | 3.6980 | 3000 | 1.8127 | | 1.4376 | 3.8213 | 3100 | 1.7946 | | 1.4436 | 3.9445 | 3200 | 1.7834 | | 1.3534 | 4.0678 | 3300 | 1.8001 | | 1.3562 | 4.1911 | 3400 | 1.7946 | | 1.3416 | 4.3143 | 3500 | 1.7894 | | 1.269 | 4.4376 | 3600 | 1.7802 | | 1.3105 | 4.5609 | 3700 | 1.7751 | | 1.3331 | 4.6841 | 3800 | 1.7627 | | 1.2788 | 4.8074 | 3900 | 1.7766 | | 1.256 | 4.9307 | 4000 | 1.7723 | | 1.2342 | 5.0539 | 4100 | 1.7943 | | 1.1391 | 5.1772 | 4200 | 1.7807 | | 1.165 | 5.3005 | 4300 | 1.8016 | | 1.2122 | 5.4237 | 4400 | 1.7840 | | 1.1536 | 5.5470 | 4500 | 1.7805 | | 1.217 | 5.6703 | 4600 | 1.7775 | | 1.1769 | 5.7935 | 4700 | 1.7817 | | 1.225 | 5.9168 | 4800 | 1.7758 | | 1.1306 | 6.0401 | 4900 | 1.8010 | | 1.0248 | 6.1633 | 5000 | 1.8035 | | 1.036 | 6.2866 | 5100 | 1.8228 | | 1.1205 | 6.4099 | 5200 | 1.8145 | | 1.0873 | 6.5331 | 5300 | 1.7970 | | 1.0785 | 6.6564 | 5400 | 1.8077 | | 1.0628 | 6.7797 | 5500 | 1.8102 | | 1.0423 | 6.9029 | 5600 | 1.8027 | | 1.0392 | 7.0262 | 5700 | 1.8268 | | 0.9586 | 7.1495 | 5800 | 1.8684 | | 0.8986 | 7.2727 | 5900 | 1.8406 | | 0.9616 | 7.3960 | 6000 | 1.8408 | | 1.026 | 7.5193 | 6100 | 1.8500 | | 0.9494 | 7.6425 | 6200 | 1.8326 | | 0.9769 | 7.7658 | 6300 | 1.8455 | | 0.9322 | 7.8891 | 6400 | 1.8445 | | 0.9305 | 8.0123 | 6500 | 1.8578 | | 0.8791 | 8.1356 | 6600 | 1.8941 | | 0.8852 | 8.2589 | 6700 | 1.9108 | | 0.8771 | 8.3821 | 6800 | 1.8951 | | 0.8697 | 8.5054 | 6900 | 1.9177 | | 0.8676 | 8.6287 | 7000 | 1.9179 | | 0.8527 | 8.7519 | 7100 | 1.8762 | | 0.8281 | 8.8752 | 7200 | 1.9050 | | 0.88 | 8.9985 | 7300 | 1.9411 | | 0.7569 | 9.1217 | 7400 | 1.9684 | | 0.7265 | 9.2450 | 7500 | 1.9705 | | 0.7659 | 9.3683 | 7600 | 1.9602 | | 0.7858 | 9.4915 | 7700 | 1.9635 | | 0.7349 | 9.6148 | 7800 | 1.9725 | | 0.7882 | 9.7381 | 7900 | 1.9662 | | 0.8001 | 9.8613 | 8000 | 1.9443 | | 0.8258 | 9.9846 | 8100 | 1.9441 | | 0.7034 | 10.1079 | 8200 | 2.0047 | | 0.6926 | 10.2311 | 8300 | 2.0246 | | 0.6978 | 10.3544 | 8400 | 2.0163 | | 0.7311 | 10.4777 | 8500 | 2.0388 | | 0.6835 | 10.6009 | 8600 | 2.0432 | | 0.7273 | 10.7242 | 8700 | 2.0073 | | 0.6887 | 10.8475 | 8800 | 2.0164 | | 0.7032 | 10.9707 | 8900 | 2.0060 | | 0.6222 | 11.0940 | 9000 | 2.0942 | | 0.6344 | 11.2173 | 9100 | 2.0941 | | 0.5905 | 11.3405 | 9200 | 2.1227 | | 0.6398 | 11.4638 | 9300 | 2.1075 | | 0.6342 | 11.5871 | 9400 | 2.1051 | | 0.6245 | 11.7103 | 9500 | 2.0911 | | 0.6529 | 11.8336 | 9600 | 2.0880 | | 0.6416 | 11.9569 | 9700 | 2.0804 | ### Framework versions - PEFT 0.13.0 - Transformers 4.46.0.dev0 - Pytorch 2.4.1+cu121 - Datasets 3.0.1 - Tokenizers 0.20.0