Padu98/ampazephyr-2-prompt-2-versuch-2

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.7187
+- Loss: 0.3778
 
 ## Model description
 
@@ -49,15 +49,15 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| No log        | 1.0   | 143  | 0.8529          |
-| No log        | 2.0   | 286  | 0.7502          |
-| No log        | 3.0   | 429  | 0.7187          |
+| No log        | 1.0   | 136  | 0.4528          |
+| No log        | 2.0   | 272  | 0.3838          |
+| No log        | 3.0   | 408  | 0.3778          |
 
 
 ### Framework versions
 
 - PEFT 0.9.0
 - Transformers 4.39.0.dev0
-- Pytorch 2.0.0
+- Pytorch 2.1.2
 - Datasets 2.16.0
 - Tokenizers 0.15.2

adapter_config.json

@@ -19,8 +19,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "up_proj",
+    "down_proj",
+    "v_proj",
+    "o_proj",
     "q_proj",
-    "v_proj"
+    "k_proj",
+    "gate_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cae5160df8f0391d6488d4fc476d53fe5e7957b5ee68fc680ff6904775e535b7
-size 54543184
+oid sha256:0260ed0775dd5385b8bf9dbfefb99357c632fa8c01c36564e134015ff5a65a78
+size 335604696

runs/Mar05_18-15-01_b84c459f25d9/events.out.tfevents.1709662503.b84c459f25d9.174.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a20f2c8e2432d9cc8a995b4fa9d464bf473531cfd6973749ca39c4334557ed95
+size 5360

runs/Mar05_23-53-34_fd8102db6ce4/events.out.tfevents.1709682816.fd8102db6ce4.229.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:107ce9fa3b4e2de4f8c61452443ffc959a97bc94018ae66154f907dd5d4486b3
+size 5089

runs/Mar06_00-12-58_fd8102db6ce4/events.out.tfevents.1709683980.fd8102db6ce4.229.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6803aab8789c3aaf5d835b3782f079249c4b97768eccf60aab1b2e3b8f68df84
+size 6256

runs/Mar06_00-12-58_fd8102db6ce4/events.out.tfevents.1709701064.fd8102db6ce4.229.2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4983b37de9193dbe9e1635b3bd4c7477d01bcdd29c2b3754cb42b07c5a1d944a
+size 359

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:679d7825894ae33e64f8cd8a5037030aacb1c76a0ca0539bbb176d1e0bb3e3f1
-size 4475
+oid sha256:eda54c08c2be8b54edd5c6d9ec5d8435a7476629a1f39c458d52f66064591088
+size 4920