End of training

README.md

@@ -14,18 +14,18 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/pauld/huggingface/runs/em482maw)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/pauld/huggingface/runs/y7di7l44)
 # kto-aligned-model-lora
 
 This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.5001
-- Eval/rewards/chosen: 0.1739
-- Eval/logps/chosen: -0.4029
-- Eval/rewards/rejected: 0.1787
-- Eval/logps/rejected: -0.0087
-- Eval/rewards/margins: -0.0048
-- Eval/kl: 1.7305
+- Loss: 0.4990
+- Eval/rewards/chosen: 0.1561
+- Eval/logps/chosen: -0.6624
+- Eval/rewards/rejected: 0.1281
+- Eval/logps/rejected: -1.9415
+- Eval/rewards/margins: 0.0281
+- Eval/kl: 1.5643
 
 ## Model description
 
@@ -58,13 +58,13 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |        |
-|:-------------:|:-----:|:----:|:---------------:|:------:|
-| 0.5           | 1.0   | 9    | 0.5000          | 1.2364 |
-| 0.4994        | 2.0   | 18   | 0.5002          | 1.7169 |
-| 0.4985        | 3.0   | 27   | 0.5003          | 1.7311 |
-| 0.4981        | 4.0   | 36   | 0.5002          | 1.7306 |
-| 0.4976        | 5.0   | 45   | 0.5001          | 1.7305 |
+| Training Loss | Epoch  | Step | Validation Loss |        |
+|:-------------:|:------:|:----:|:---------------:|:------:|
+| 0.4994        | 0.9057 | 8    | 0.4997          | 0.8856 |
+| 0.5           | 1.9245 | 17   | 0.4994          | 1.5546 |
+| 0.501         | 2.9434 | 26   | 0.4992          | 1.5634 |
+| 0.5004        | 3.9623 | 35   | 0.4991          | 1.5675 |
+| 0.4999        | 4.5283 | 40   | 0.4990          | 1.5643 |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:51cf6a46415880ad28aa06798a0e1c911e1c554721e670aaa19708322019f84d
+oid sha256:e9edc78ac5314459cade5002a9b7fbea45d0f906f16037086f352c5e657d6730
 size 8397184

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e9122972af136c3835ff9b69597ab9835cc8f727c57b9a37afdb1bff7816ddd4
+oid sha256:b09854525ea44d0ece38e709aa75664028ad8e7e57216b0f5a61dfec8fa1bb4c
 size 5496