Model save

README.md

@@ -6,8 +6,6 @@ tags:
 - sft
 - generated_from_trainer
 base_model: google/gemma-2b-it
-datasets:
-- generator
 model-index:
 - name: gemma-2b-lahacks
   results: []
@@ -18,9 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # gemma-2b-lahacks
 
-This model is a fine-tuned version of [google/gemma-2b-it](https://huggingface.co/google/gemma-2b-it) on the generator dataset.
-It achieves the following results on the evaluation set:
-- Loss: 2.2077
+This model is a fine-tuned version of [google/gemma-2b-it](https://huggingface.co/google/gemma-2b-it) on an unknown dataset.
 
 ## Model description
 
@@ -48,27 +44,7 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.05
-- training_steps: 1480
-
-### Training results
-
-| Training Loss | Epoch   | Step | Validation Loss |
-|:-------------:|:-------:|:----:|:---------------:|
-| 3.6438        | 1.6393  | 100  | 2.7298          |
-| 2.5556        | 3.2787  | 200  | 2.3638          |
-| 2.3244        | 4.9180  | 300  | 2.2709          |
-| 2.2537        | 6.5574  | 400  | 2.2349          |
-| 2.2077        | 8.1967  | 500  | 2.2143          |
-| 2.1685        | 9.8361  | 600  | 2.2032          |
-| 2.1429        | 11.4754 | 700  | 2.1962          |
-| 2.1116        | 13.1148 | 800  | 2.1939          |
-| 2.0834        | 14.7541 | 900  | 2.1944          |
-| 2.0626        | 16.3934 | 1000 | 2.1977          |
-| 2.0412        | 18.0328 | 1100 | 2.2000          |
-| 2.0286        | 19.6721 | 1200 | 2.2040          |
-| 2.0217        | 21.3115 | 1300 | 2.2062          |
-| 2.0134        | 22.9508 | 1400 | 2.2077          |
-
+- training_steps: 10
 
 ### Framework versions
 

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "v_proj",
-    "k_proj",
-    "down_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
     "gate_proj",
-    "o_proj"
+    "k_proj",
+    "down_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c841af3cdb925e4ad731629ebd351b026ce8ebe08df3ba8147e7ad5896d685c
+oid sha256:c130bc7bba51fcf7ba71cf49236239a0e7fc180ba8073c7a8f5d29162b5def1d
 size 156926880

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f4cd025f08cab4a0cce837f20f3f35c5e126f3d886262e96ddaa1efd58c3ebcb
+oid sha256:a37fd9e26ec03cda8763231bf0e23468667784d92a977515b44587389b710b87
 size 4539