End of training

README.md

@@ -61,7 +61,7 @@ output_dir: ./outputs/lora-alpaca-pythia-160m-storytelling
 gradient_accumulation_steps: 16
 micro_batch_size: 1
 num_epochs: 3
-learning_rate: 0.001
+learning_rate: 0.004
 lr_scheduler: cosine_with_restarts
 #cosine_min_lr_ratio: 0.1
 train_on_inputs: false
@@ -98,7 +98,7 @@ tokens:
 
 This model is a fine-tuned version of [EleutherAI/pythia-160m-deduped](https://huggingface.co/EleutherAI/pythia-160m-deduped) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 7.3539
+- Loss: 5.0097
 
 ## Model description
 
@@ -117,7 +117,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.001
+- learning_rate: 0.004
 - train_batch_size: 1
 - eval_batch_size: 1
 - seed: 42
@@ -139,6 +139,9 @@ The following hyperparameters were used during training:
 | 8.1159        | 0.9391 | 800  | 8.4966          |
 | 6.7656        | 1.1739 | 1000 | 7.1575          |
 | 7.0548        | 1.4087 | 1200 | 7.3539          |
+| 5.9982        | 1.6445 | 1400 | 5.9954          |
+| 5.7662        | 1.8792 | 1600 | 6.0222          |
+| 4.8094        | 2.1140 | 1800 | 5.0097          |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:beed07264889070614923e60987914dfc89e9442eeeac7fc7e983c8338de1f46
-size 159266376
+oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
+size 48

config.json

@@ -22,7 +22,7 @@
   "rotary_emb_base": 10000,
   "rotary_pct": 0.25,
   "tie_word_embeddings": false,
-  "torch_dtype": "float16",
+  "torch_dtype": "bfloat16",
   "transformers_version": "4.41.2",
   "use_cache": false,
   "use_parallel_residual": true,

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9dec837ce83e8a9edef8bb8c740c7c5826b1c3b830849b2c1c30a0e610b54bc6
+oid sha256:0144df65701734c02c9a9a78f6dcaf1b3b5f4ec06aaa3866efa3dc31c90aafc6
 size 324696090