Mistral-7B-Instruct-v0.2-finetune-SWE_90_10

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.7372

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 3
• eval_batch_size: 3
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: reduce_lr_on_plateau
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.4991	0.9992	1231	1.7493
1.5973	1.9984	2462	1.5530
0.7761	2.9976	3693	1.6627
0.4026	3.9968	4924	1.9381
0.3181	4.9959	6155	2.1410
0.2572	5.9951	7386	2.3047
0.2783	6.9943	8617	2.4170
0.1911	7.9935	9848	2.5913
0.2101	8.9927	11079	2.5669
0.1934	9.9919	12310	2.5707
0.1641	10.9911	13541	2.5205
0.1534	11.9903	14772	2.6706
0.1887	12.9894	16003	2.7875
0.1146	13.9886	17234	2.9092
0.0891	14.9878	18465	3.2176
0.0845	15.9870	19696	3.3288
0.0901	16.9862	20927	3.4202
0.0805	17.9854	22158	3.4854
0.0768	18.9846	23389	3.4997
0.0788	19.9838	24620	3.5510
0.0829	20.9830	25851	3.5782
0.0729	21.9821	27082	3.5944
0.0747	22.9813	28313	3.6143
0.0767	23.9805	29544	3.6171
0.0655	24.9797	30775	3.6633
0.0695	25.9789	32006	3.6780
0.0632	26.9781	33237	3.6896
0.0628	27.9773	34468	3.6971
0.0626	28.9765	35699	3.7027
0.0601	29.9756	36930	3.7070
0.0576	30.9748	38161	3.7114
0.1134	31.9740	39392	3.7157
0.1046	32.9732	40623	3.7186
0.1019	33.9724	41854	3.7199
0.0935	34.9716	43085	3.7234
0.0911	35.9708	44316	3.7252
0.0899	36.9700	45547	3.7271
0.0919	37.9692	46778	3.7285
0.0823	38.9683	48009	3.7299
0.0871	39.9675	49240	3.7312
0.0824	40.9667	50471	3.7322
0.0812	41.9659	51702	3.7332
0.0813	42.9651	52933	3.7342
0.0802	43.9643	54164	3.7350
0.0809	44.9635	55395	3.7359
0.0782	45.9627	56626	3.7368
0.0765	46.9619	57857	3.7369
0.0787	47.9610	59088	3.7370
0.076	48.9602	60319	3.7370
0.0756	49.9594	61550	3.7372

Framework versions

• PEFT 0.10.0
• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1