RSI-LLAVA-64LORA-3EPOCHS

This model is a fine-tuned version of llava-hf/llava-1.5-7b-hf on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8138

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00025
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 5
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
1.269	0.1546	25	1.1089
0.9311	0.3091	50	0.9328
0.9195	0.4637	75	0.8907
0.8603	0.6182	100	0.8647
0.8347	0.7728	125	0.8590
0.8272	0.9274	150	0.8398
0.8098	1.0819	175	0.8437
0.7734	1.2365	200	0.8274
0.7265	1.3910	225	0.8215
0.7579	1.5456	250	0.8178
0.7563	1.7002	275	0.8125
0.7343	1.8547	300	0.8032
0.725	2.0093	325	0.8070
0.653	2.1638	350	0.8126
0.6451	2.3184	375	0.8152
0.6623	2.4730	400	0.8156
0.6592	2.6275	425	0.8157
0.6711	2.7821	450	0.8148
0.6376	2.9366	475	0.8138

Framework versions

• PEFT 0.12.0
• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1