End of training

README.md

@@ -61,7 +61,7 @@ output_dir: ./outputs/lora-alpaca-pythia-160m-storytelling
 gradient_accumulation_steps: 16
 micro_batch_size: 1
 num_epochs: 3
-learning_rate: 0.0006
+learning_rate: 0.001
 lr_scheduler: cosine_with_restarts
 #cosine_min_lr_ratio: 0.1
 train_on_inputs: false
@@ -75,7 +75,7 @@ xformers_attention: true
 optimizer: paged_adamw_8bit
 gpu_memory_limit: 8GiB
 hub_model_id: jtatman/pythia-160m-storytelling 
-early_stopping_patience: 2
+early_stopping_patience: 3
 #resume_from_checkpoint: outputs/lora-alpaca-pythia-125m/checkpoint-51040
 auto_resume_from_checkpoints: true
 local_rank:
@@ -98,7 +98,7 @@ tokens:
 
 This model is a fine-tuned version of [EleutherAI/pythia-160m-deduped](https://huggingface.co/EleutherAI/pythia-160m-deduped) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 5.0363
+- Loss: 7.3539
 
 ## Model description
 
@@ -117,7 +117,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0006
+- learning_rate: 0.001
 - train_batch_size: 1
 - eval_batch_size: 1
 - seed: 42
@@ -136,6 +136,9 @@ The following hyperparameters were used during training:
 | 4.2012        | 0.2348 | 200  | 4.1556          |
 | 4.4185        | 0.4696 | 400  | 4.8159          |
 | 5.0973        | 0.7043 | 600  | 5.0363          |
+| 8.1159        | 0.9391 | 800  | 8.4966          |
+| 6.7656        | 1.1739 | 1000 | 7.1575          |
+| 7.0548        | 1.4087 | 1200 | 7.3539          |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2e82a4ad5a2cd99d4cd86f4e5bd79962b304eb0c1833326168080d4dbdf35a29
-size 159266376
+oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
+size 48

config.json

@@ -22,7 +22,7 @@
   "rotary_emb_base": 10000,
   "rotary_pct": 0.25,
   "tie_word_embeddings": false,
-  "torch_dtype": "float16",
+  "torch_dtype": "bfloat16",
   "transformers_version": "4.41.2",
   "use_cache": false,
   "use_parallel_residual": true,

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:168d7d3a0afe4d5734fa7304faf1fe11e36d3bc39c475402248255244c303708
+oid sha256:9dec837ce83e8a9edef8bb8c740c7c5826b1c3b830849b2c1c30a0e610b54bc6
 size 324696090