djuhas/finetune

README.md

@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [NousResearch/Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.7500
+- Loss: 1.8992
 
 ## Model description
 
@@ -50,12 +50,12 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| 1.5408        | 1.25  | 25   | 1.2418          |
-| 0.5166        | 2.5   | 50   | 1.4973          |
-| 0.2425        | 3.75  | 75   | 1.7257          |
-| 0.1325        | 5.0   | 100  | 1.7500          |
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 1.4662        | 1.1364 | 25   | 1.3819          |
+| 0.6972        | 2.2727 | 50   | 1.5203          |
+| 0.3476        | 3.4091 | 75   | 1.8026          |
+| 0.0995        | 4.5455 | 100  | 1.8992          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,11 +20,11 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "gate_proj",
+    "k_proj",
     "o_proj",
     "up_proj",
     "v_proj",
-    "gate_proj",
-    "k_proj",
     "q_proj",
     "down_proj"
   ],

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:aecae43b1cddaa1071bbd4b6c955785243343b959ee886ba04ea1fcaa06807cf
+oid sha256:df27498e3c57f0f06d3cbd219f511a1fc74f776af15b16b71c8570f480a1035d
 size 167832240

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3fe1024bc73434c4418468a18e47778dc168b5eb3be1c82b13583c23ee1f87f4
+oid sha256:1090f4233bd0f3a0f855edc46401c5245817f701c7e0a0a49ebae2761cee7ecf
 size 5432