mistralinstruct-7b-sft-lora-belgianlaw

README.md

@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.4251
+- Loss: 1.2937
 
 ## Model description
 
@@ -53,12 +53,12 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| 1.4326        | 0.8807 | 6    | 1.4251          |
+| 1.3021        | 0.9778 | 10   | 1.2937          |
 
 
 ### Framework versions
 
-- PEFT 0.11.1
+- PEFT 0.12.0
 - Transformers 4.42.4
 - Pytorch 2.3.1+cu121
 - Datasets 2.20.0

adapter_config.json

@@ -20,9 +20,9 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "o_proj",
     "k_proj",
     "v_proj",
-    "o_proj",
     "q_proj"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3f973ab09f6874d183dbd626377c9be9569d0aa0d5ac0969ad591b1c89c51b78
-size 109086672
+oid sha256:89a17c1202ccb53c886297794f5e497a7c88434bf8e0aa8ead671620a41866e7
+size 218138576

runs/Jul29_18-30-54_ac73eaa7c770/events.out.tfevents.1722278019.ac73eaa7c770.198.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:643038ae036813932f541ca7fe87db1d67765c4e8825d41ea07c9a3e7425e66b
+size 6382

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:66207095291eda917acd277d2b7cefabb7a166a1e1564da047c470a1ba15d159
+oid sha256:08159b9ce9c9af2db3e8eed7674e89d51a1c64d8943c97f367fa076c1835cc87
 size 5560