Model save

README.md

@@ -1,11 +1,8 @@
 ---
 base_model: barc0/cot-400k-barc-llama3.1-8b-ins-fft-transduction_lr1e-5_epoch3
-datasets:
-- barc0/cot_transduction_train_dataset_960_ms10
 library_name: peft
 license: llama3.1
 tags:
-- alignment-handbook
 - trl
 - sft
 - generated_from_trainer
@@ -19,9 +16,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # cot-trainset-ft-transduction-v2-lora-train
 
-This model is a fine-tuned version of [barc0/cot-400k-barc-llama3.1-8b-ins-fft-transduction_lr1e-5_epoch3](https://huggingface.co/barc0/cot-400k-barc-llama3.1-8b-ins-fft-transduction_lr1e-5_epoch3) on the barc0/cot_transduction_train_dataset_960_ms10 dataset.
+This model is a fine-tuned version of [barc0/cot-400k-barc-llama3.1-8b-ins-fft-transduction_lr1e-5_epoch3](https://huggingface.co/barc0/cot-400k-barc-llama3.1-8b-ins-fft-transduction_lr1e-5_epoch3) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.2107
+- Loss: 0.1333
 
 ## Model description
 
@@ -52,15 +49,14 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 3
+- num_epochs: 2
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| 0.1316        | 1.0   | 155  | 0.1494          |
-| 0.073         | 2.0   | 310  | 0.1669          |
-| 0.0305        | 3.0   | 465  | 0.2107          |
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 0.1018        | 0.9982 | 277  | 0.1238          |
+| 0.0822        | 1.9964 | 554  | 0.1333          |
 
 
 ### Framework versions

all_results.json

@@ -1,14 +1,9 @@
 {
-    "epoch": 3.0,
-    "eval_loss": 0.2107405662536621,
-    "eval_runtime": 25.4497,
-    "eval_samples": 131,
-    "eval_samples_per_second": 5.147,
-    "eval_steps_per_second": 0.668,
-    "total_flos": 261374226563072.0,
-    "train_loss": 0.09038178783751304,
-    "train_runtime": 3542.1658,
-    "train_samples": 2476,
-    "train_samples_per_second": 2.097,
-    "train_steps_per_second": 0.131
+    "epoch": 1.9963963963963964,
+    "total_flos": 450799970877440.0,
+    "train_loss": 0.0942714535009237,
+    "train_runtime": 4963.7264,
+    "train_samples": 4434,
+    "train_samples_per_second": 1.787,
+    "train_steps_per_second": 0.112
 }

train_results.json

@@ -1,9 +1,9 @@
 {
-    "epoch": 3.0,
-    "total_flos": 261374226563072.0,
-    "train_loss": 0.09038178783751304,
-    "train_runtime": 3542.1658,
-    "train_samples": 2476,
-    "train_samples_per_second": 2.097,
-    "train_steps_per_second": 0.131
+    "epoch": 1.9963963963963964,
+    "total_flos": 450799970877440.0,
+    "train_loss": 0.0942714535009237,
+    "train_runtime": 4963.7264,
+    "train_samples": 4434,
+    "train_samples_per_second": 1.787,
+    "train_steps_per_second": 0.112
 }