tcarwash/tinyllama-instruct

README.md

@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [tinyllama/tinyllama-1.1b-intermediate-step-1431k-3t](https://huggingface.co/tinyllama/tinyllama-1.1b-intermediate-step-1431k-3t) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.3012
+- Loss: 1.3383
 
 ## Model description
 
@@ -46,16 +46,13 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
 - lr_scheduler_warmup_steps: 0.03
-- num_epochs: 4
+- training_steps: 300
 
 ### Training results
 
-| Training Loss | Epoch | Step  | Validation Loss |
-|:-------------:|:-----:|:-----:|:---------------:|
-| 1.4385        | 1.0   | 4263  | 1.2766          |
-| 1.4772        | 2.0   | 8526  | 1.2743          |
-| 1.0998        | 3.0   | 12789 | 1.2837          |
-| 1.3263        | 4.0   | 17052 | 1.3012          |
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 1.4051        | 0.0704 | 300  | 1.3383          |
 
 
 ### Framework versions

adapter_config.json

@@ -1,7 +1,7 @@
 {
   "alpha_pattern": {},
   "auto_mapping": null,
-  "base_model_name_or_path": null,
+  "base_model_name_or_path": "tinyllama/tinyllama-1.1b-intermediate-step-1431k-3t",
   "bias": "none",
   "fan_in_fan_out": false,
   "inference_mode": true,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dcb0045517d11e5dc96444a144c9f7e188f081679dbbc1a52764bdf7ffcf551b
-size 36058104
+oid sha256:cf3322ebabf5fdd0f394ea20ade983db6aa6ae49d99316d807711cbca8333b5c
+size 36056608

runs/May04_05-29-08_fc985ddde5a8/events.out.tfevents.1714800551.fc985ddde5a8.702.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:593a294c4880410da5f68b8ab81e95306d8544da5e1a70d70de7b03dd2f8d965
+size 11654

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:db0cad0f34b276b350f7482750bf5f1c4fe718452bce3cf3888dce72e6f5b0e0
+oid sha256:9c424d2d2c93624e7bd109cd8c85073defd3d9f6fc8ff9e60ba5a4db87e81d44
 size 4984