doplhin-2.1-mistral-7b-orpo-ultrafeedback-binarized-preferences

This model is a fine-tuned version of cognitivecomputations/dolphin-2.1-mistral-7b on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8506
  • Rewards/chosen: -0.0852
  • Rewards/rejected: -0.1166
  • Rewards/accuracies: 0.6457
  • Rewards/margins: 0.0314
  • Logps/rejected: -1.1665
  • Logps/chosen: -0.8525
  • Logits/rejected: -2.6517
  • Logits/chosen: -2.7250
  • Nll Loss: 0.7896
  • Log Odds Ratio: -0.6110
  • Log Odds Chosen: 0.4581

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen Nll Loss Log Odds Ratio Log Odds Chosen
0.9101 0.25 700 0.8845 -0.0869 -0.1106 0.6428 0.0237 -1.1059 -0.8694 -2.6631 -2.7431 0.8224 -0.6225 0.3631
0.8554 0.51 1400 0.8609 -0.0877 -0.1233 0.6555 0.0357 -1.2332 -0.8766 -2.6169 -2.6996 0.8007 -0.6040 0.5048
0.9011 0.76 2100 0.8506 -0.0852 -0.1166 0.6457 0.0314 -1.1665 -0.8525 -2.6517 -2.7250 0.7896 -0.6110 0.4581

Framework versions

  • PEFT 0.10.1.dev0
  • Transformers 4.40.0.dev0
  • Pytorch 2.1.2+cu121
  • Datasets 2.18.1.dev0
  • Tokenizers 0.15.2
Downloads last month
3
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for DrishtiSharma/doplhin-2.1-mistral-7b-orpo-ultrafeedback-binarized-preferences

Adapter
(11)
this model