ai-maker-space/llama38binstruct-summary-jun14-b4-merge

README.md

@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [NousResearch/Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.9957
+- Loss: 1.4391
 
 ## Model description
 
@@ -50,18 +50,18 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| 1.4216        | 1.25  | 25   | 1.4108          |
-| 0.5039        | 2.5   | 50   | 1.6955          |
-| 0.183         | 3.75  | 75   | 1.8566          |
-| 0.1127        | 5.0   | 100  | 1.9957          |
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 1.4009        | 1.3889 | 25   | 1.0738          |
+| 0.4554        | 2.7778 | 50   | 1.2629          |
+| 0.2384        | 4.1667 | 75   | 1.3364          |
+| 0.0555        | 5.5556 | 100  | 1.4391          |
 
 
 ### Framework versions
 
 - PEFT 0.11.1
 - Transformers 4.41.2
-- Pytorch 2.3.0+cu121
+- Pytorch 2.3.1+cu121
 - Datasets 2.20.0
 - Tokenizers 0.19.1

adapter_config.json

@@ -20,12 +20,12 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "o_proj",
-    "v_proj",
-    "q_proj",
+    "k_proj",
     "down_proj",
+    "q_proj",
     "gate_proj",
-    "k_proj",
+    "o_proj",
+    "v_proj",
     "up_proj"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
-size 48
+oid sha256:327a960fa71119ffe1ff78f6b2017652be2348c0921ced6558f431f27f099cd3
+size 167832240

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ddeed7fc25a7b2d97345ae61c62c8ef1296bab1c3c6a1c366bd84db2d6c05323
-size 5368
+oid sha256:5a10f393c1617151db0aa89533642c35b66d61da5abea9a055150180aa528bf1
+size 5432