qwen2-7b-instruct-trl-sft-mrg

This model is a fine-tuned version of Qwen/Qwen2-VL-7B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0746

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 15

Training results

Training Loss Epoch Step Validation Loss
2.0164 0.4947 32 1.8794
1.7004 0.9894 64 1.5669
1.4913 1.4841 96 1.4239
1.4568 1.9787 128 1.3451
1.3309 2.4734 160 1.2841
1.2276 2.9681 192 1.2427
1.2531 3.4628 224 1.2152
1.1983 3.9575 256 1.1806
1.1846 4.4522 288 1.1549
1.121 4.9469 320 1.1357
1.096 5.4415 352 1.1263
1.0818 5.9362 384 1.1133
1.0113 6.4309 416 1.1039
1.0566 6.9256 448 1.0896
1.0195 7.4203 480 1.0926
0.9407 7.9150 512 1.0759
1.0077 8.4097 544 1.0790
1.0554 8.9043 576 1.0694
0.9664 9.3990 608 1.0774
0.9157 9.8937 640 1.0643
0.8869 10.3884 672 1.0705
0.9715 10.8831 704 1.0660
0.8926 11.3778 736 1.0752
0.8906 11.8725 768 1.0718
0.8649 12.3671 800 1.0686
0.9162 12.8618 832 1.0685
0.8708 13.3565 864 1.0798
0.9127 13.8512 896 1.0702
0.8355 14.3459 928 1.0732
0.806 14.8406 960 1.0746

Framework versions

  • PEFT 0.11.1
  • Transformers 4.45.2
  • Pytorch 2.3.1+cu121
  • Datasets 3.0.1
  • Tokenizers 0.20.0
Downloads last month
45
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.

Model tree for DLingo/qwen2-7b-instruct-trl-sft-mrg

Base model

Qwen/Qwen2-VL-7B
Adapter
(111)
this model