metadata
license: apache-2.0
library_name: peft
tags:
- generated_from_trainer
base_model: mistralai/Mistral-7B-Instruct-v0.2
model-index:
- name: Mistral-7B-Instruct-v0.2-finetune-SWE_90_10
results: []
Mistral-7B-Instruct-v0.2-finetune-SWE_90_10
This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 3.7372
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 3
- eval_batch_size: 3
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: reduce_lr_on_plateau
- num_epochs: 50
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.4991 | 0.9992 | 1231 | 1.7493 |
| 1.5973 | 1.9984 | 2462 | 1.5530 |
| 0.7761 | 2.9976 | 3693 | 1.6627 |
| 0.4026 | 3.9968 | 4924 | 1.9381 |
| 0.3181 | 4.9959 | 6155 | 2.1410 |
| 0.2572 | 5.9951 | 7386 | 2.3047 |
| 0.2783 | 6.9943 | 8617 | 2.4170 |
| 0.1911 | 7.9935 | 9848 | 2.5913 |
| 0.2101 | 8.9927 | 11079 | 2.5669 |
| 0.1934 | 9.9919 | 12310 | 2.5707 |
| 0.1641 | 10.9911 | 13541 | 2.5205 |
| 0.1534 | 11.9903 | 14772 | 2.6706 |
| 0.1887 | 12.9894 | 16003 | 2.7875 |
| 0.1146 | 13.9886 | 17234 | 2.9092 |
| 0.0891 | 14.9878 | 18465 | 3.2176 |
| 0.0845 | 15.9870 | 19696 | 3.3288 |
| 0.0901 | 16.9862 | 20927 | 3.4202 |
| 0.0805 | 17.9854 | 22158 | 3.4854 |
| 0.0768 | 18.9846 | 23389 | 3.4997 |
| 0.0788 | 19.9838 | 24620 | 3.5510 |
| 0.0829 | 20.9830 | 25851 | 3.5782 |
| 0.0729 | 21.9821 | 27082 | 3.5944 |
| 0.0747 | 22.9813 | 28313 | 3.6143 |
| 0.0767 | 23.9805 | 29544 | 3.6171 |
| 0.0655 | 24.9797 | 30775 | 3.6633 |
| 0.0695 | 25.9789 | 32006 | 3.6780 |
| 0.0632 | 26.9781 | 33237 | 3.6896 |
| 0.0628 | 27.9773 | 34468 | 3.6971 |
| 0.0626 | 28.9765 | 35699 | 3.7027 |
| 0.0601 | 29.9756 | 36930 | 3.7070 |
| 0.0576 | 30.9748 | 38161 | 3.7114 |
| 0.1134 | 31.9740 | 39392 | 3.7157 |
| 0.1046 | 32.9732 | 40623 | 3.7186 |
| 0.1019 | 33.9724 | 41854 | 3.7199 |
| 0.0935 | 34.9716 | 43085 | 3.7234 |
| 0.0911 | 35.9708 | 44316 | 3.7252 |
| 0.0899 | 36.9700 | 45547 | 3.7271 |
| 0.0919 | 37.9692 | 46778 | 3.7285 |
| 0.0823 | 38.9683 | 48009 | 3.7299 |
| 0.0871 | 39.9675 | 49240 | 3.7312 |
| 0.0824 | 40.9667 | 50471 | 3.7322 |
| 0.0812 | 41.9659 | 51702 | 3.7332 |
| 0.0813 | 42.9651 | 52933 | 3.7342 |
| 0.0802 | 43.9643 | 54164 | 3.7350 |
| 0.0809 | 44.9635 | 55395 | 3.7359 |
| 0.0782 | 45.9627 | 56626 | 3.7368 |
| 0.0765 | 46.9619 | 57857 | 3.7369 |
| 0.0787 | 47.9610 | 59088 | 3.7370 |
| 0.076 | 48.9602 | 60319 | 3.7370 |
| 0.0756 | 49.9594 | 61550 | 3.7372 |
Framework versions
- PEFT 0.10.0
- Transformers 4.40.2
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1