---
license: apache-2.0
library_name: peft
tags:
- generated_from_trainer
base_model: mistralai/Mistral-7B-Instruct-v0.2
model-index:
- name: Mistral-7B-Instruct-v0.2-finetune-SWE_90_10
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Mistral-7B-Instruct-v0.2-finetune-SWE_90_10

This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 3.7372

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 3
- eval_batch_size: 3
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: reduce_lr_on_plateau
- num_epochs: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step  | Validation Loss |
|:-------------:|:-------:|:-----:|:---------------:|
| 1.4991        | 0.9992  | 1231  | 1.7493          |
| 1.5973        | 1.9984  | 2462  | 1.5530          |
| 0.7761        | 2.9976  | 3693  | 1.6627          |
| 0.4026        | 3.9968  | 4924  | 1.9381          |
| 0.3181        | 4.9959  | 6155  | 2.1410          |
| 0.2572        | 5.9951  | 7386  | 2.3047          |
| 0.2783        | 6.9943  | 8617  | 2.4170          |
| 0.1911        | 7.9935  | 9848  | 2.5913          |
| 0.2101        | 8.9927  | 11079 | 2.5669          |
| 0.1934        | 9.9919  | 12310 | 2.5707          |
| 0.1641        | 10.9911 | 13541 | 2.5205          |
| 0.1534        | 11.9903 | 14772 | 2.6706          |
| 0.1887        | 12.9894 | 16003 | 2.7875          |
| 0.1146        | 13.9886 | 17234 | 2.9092          |
| 0.0891        | 14.9878 | 18465 | 3.2176          |
| 0.0845        | 15.9870 | 19696 | 3.3288          |
| 0.0901        | 16.9862 | 20927 | 3.4202          |
| 0.0805        | 17.9854 | 22158 | 3.4854          |
| 0.0768        | 18.9846 | 23389 | 3.4997          |
| 0.0788        | 19.9838 | 24620 | 3.5510          |
| 0.0829        | 20.9830 | 25851 | 3.5782          |
| 0.0729        | 21.9821 | 27082 | 3.5944          |
| 0.0747        | 22.9813 | 28313 | 3.6143          |
| 0.0767        | 23.9805 | 29544 | 3.6171          |
| 0.0655        | 24.9797 | 30775 | 3.6633          |
| 0.0695        | 25.9789 | 32006 | 3.6780          |
| 0.0632        | 26.9781 | 33237 | 3.6896          |
| 0.0628        | 27.9773 | 34468 | 3.6971          |
| 0.0626        | 28.9765 | 35699 | 3.7027          |
| 0.0601        | 29.9756 | 36930 | 3.7070          |
| 0.0576        | 30.9748 | 38161 | 3.7114          |
| 0.1134        | 31.9740 | 39392 | 3.7157          |
| 0.1046        | 32.9732 | 40623 | 3.7186          |
| 0.1019        | 33.9724 | 41854 | 3.7199          |
| 0.0935        | 34.9716 | 43085 | 3.7234          |
| 0.0911        | 35.9708 | 44316 | 3.7252          |
| 0.0899        | 36.9700 | 45547 | 3.7271          |
| 0.0919        | 37.9692 | 46778 | 3.7285          |
| 0.0823        | 38.9683 | 48009 | 3.7299          |
| 0.0871        | 39.9675 | 49240 | 3.7312          |
| 0.0824        | 40.9667 | 50471 | 3.7322          |
| 0.0812        | 41.9659 | 51702 | 3.7332          |
| 0.0813        | 42.9651 | 52933 | 3.7342          |
| 0.0802        | 43.9643 | 54164 | 3.7350          |
| 0.0809        | 44.9635 | 55395 | 3.7359          |
| 0.0782        | 45.9627 | 56626 | 3.7368          |
| 0.0765        | 46.9619 | 57857 | 3.7369          |
| 0.0787        | 47.9610 | 59088 | 3.7370          |
| 0.076         | 48.9602 | 60319 | 3.7370          |
| 0.0756        | 49.9594 | 61550 | 3.7372          |


### Framework versions

- PEFT 0.10.0
- Transformers 4.40.2
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1