Hugging Face's logo

mtzig
/
v4_mistral_lora

PEFT
Safetensors
Generated from Trainer
Model card Files Community
v4_mistral_lora / README.md
mtzig's picture
mtzig
Model save
325ff82 verified
preview code
|
raw
history blame contribute delete
5.47 kB
metadata
library_name: peft
base_model: peiyi9979/math-shepherd-mistral-7b-prm
tags:
  - generated_from_trainer
metrics:
  - accuracy
  - precision
  - recall
  - f1
model-index:
  - name: v4_mistral_lora
    results: []

v4_mistral_lora

This model is a fine-tuned version of peiyi9979/math-shepherd-mistral-7b-prm on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2841
  • Accuracy: 0.8687
  • Precision: 0.8392
  • Recall: 0.6575
  • F1: 0.7373

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 6
  • eval_batch_size: 8
  • seed: 89234
  • distributed_type: multi-GPU
  • num_devices: 4
  • gradient_accumulation_steps: 3
  • total_train_batch_size: 72
  • total_eval_batch_size: 32
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Accuracy Precision Recall F1
No log 0 0 0.5995 0.7340 0.6 0.1535 0.2445
0.6057 0.0254 20 0.5918 0.7373 0.625 0.1575 0.2516
0.5356 0.0507 40 0.5521 0.7517 0.6790 0.2165 0.3284
0.5141 0.0761 60 0.5021 0.7627 0.5890 0.5079 0.5455
0.3594 0.1015 80 0.4427 0.7980 0.7448 0.4252 0.5414
0.3988 0.1268 100 0.4067 0.8245 0.7684 0.5354 0.6311
0.3205 0.1522 120 0.3738 0.8201 0.8138 0.4646 0.5915
0.3026 0.1776 140 0.3680 0.8289 0.8235 0.4961 0.6192
0.2886 0.2030 160 0.3467 0.8433 0.8590 0.5276 0.6537
0.2345 0.2283 180 0.3289 0.8587 0.7972 0.6654 0.7253
0.2964 0.2537 200 0.3322 0.8377 0.8497 0.5118 0.6388
0.2655 0.2791 220 0.3495 0.8278 0.8657 0.4567 0.5979
0.3252 0.3044 240 0.3189 0.8455 0.8314 0.5630 0.6714
0.2561 0.3298 260 0.3228 0.8532 0.8201 0.6102 0.6998
0.1661 0.3552 280 0.3141 0.8499 0.8598 0.5551 0.6746
0.1812 0.3805 300 0.3330 0.8300 0.8378 0.4882 0.6169
0.3265 0.4059 320 0.2961 0.8543 0.8280 0.6063 0.7
0.2217 0.4313 340 0.2970 0.8664 0.8065 0.6890 0.7431
0.2058 0.4567 360 0.3054 0.8521 0.8333 0.5906 0.6912
0.225 0.4820 380 0.3018 0.8576 0.8531 0.5945 0.7007
0.2045 0.5074 400 0.3174 0.8510 0.8742 0.5472 0.6731
0.2368 0.5328 420 0.3156 0.8477 0.8537 0.5512 0.6699
0.2162 0.5581 440 0.2928 0.8609 0.8441 0.6181 0.7136
0.1664 0.5835 460 0.2978 0.8598 0.8325 0.6260 0.7146
0.2282 0.6089 480 0.3031 0.8587 0.8539 0.5984 0.7037
0.1983 0.6342 500 0.2958 0.8543 0.8177 0.6181 0.7040
0.1843 0.6596 520 0.3055 0.8609 0.8556 0.6063 0.7097
0.1915 0.6850 540 0.2818 0.8675 0.8160 0.6811 0.7425
0.1582 0.7104 560 0.2887 0.8675 0.8641 0.6260 0.7260
0.2003 0.7357 580 0.2872 0.8653 0.8511 0.6299 0.7240
0.2345 0.7611 600 0.2827 0.8687 0.8293 0.6693 0.7407
0.2107 0.7865 620 0.2954 0.8642 0.8701 0.6063 0.7146
0.2562 0.8118 640 0.2938 0.8642 0.8503 0.6260 0.7211
0.1054 0.8372 660 0.2917 0.8642 0.8503 0.6260 0.7211
0.2837 0.8626 680 0.2842 0.8664 0.8376 0.6496 0.7317
0.1779 0.8879 700 0.2841 0.8709 0.8477 0.6575 0.7406
0.2277 0.9133 720 0.2847 0.8675 0.8384 0.6535 0.7345
0.2099 0.9387 740 0.2828 0.8720 0.8485 0.6614 0.7434
0.2167 0.9641 760 0.2835 0.8709 0.8477 0.6575 0.7406
0.1901 0.9894 780 0.2841 0.8687 0.8392 0.6575 0.7373

Framework versions

  • PEFT 0.13.2
  • Transformers 4.46.0
  • Pytorch 2.5.1+cu124
  • Datasets 3.1.0
  • Tokenizers 0.20.3