LBusser/mistral_dpo0.0

README.md

@@ -15,15 +15,15 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [TheBloke/OpenHermes-2-Mistral-7B-GPTQ](https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GPTQ) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.6931
-- Rewards/chosen: 0.0
-- Rewards/rejected: 0.0
-- Rewards/accuracies: 0.0
-- Rewards/margins: 0.0
-- Logps/rejected: -710.1063
-- Logps/chosen: -753.4531
-- Logits/rejected: -1.6178
-- Logits/chosen: -1.7566
+- Loss: 1.3007
+- Rewards/chosen: 3.3485
+- Rewards/rejected: -0.5915
+- Rewards/accuracies: 0.5625
+- Rewards/margins: 3.9400
+- Logps/rejected: -240.6290
+- Logps/chosen: -290.0968
+- Logits/rejected: -2.6366
+- Logits/chosen: -2.8894
 
 ## Model description
 
@@ -56,11 +56,11 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
 |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.6931        | 0.01  | 10   | 0.6931          | 0.0            | 0.0              | 0.0                | 0.0             | -710.1063      | -753.4531    | -1.6178         | -1.7566       |
-| 0.6931        | 0.01  | 20   | 0.6931          | 0.0            | 0.0              | 0.0                | 0.0             | -710.1063      | -753.4531    | -1.6178         | -1.7566       |
-| 0.6931        | 0.01  | 30   | 0.6931          | 0.0            | 0.0              | 0.0                | 0.0             | -710.1063      | -753.4531    | -1.6178         | -1.7566       |
-| 0.6931        | 0.02  | 40   | 0.6931          | 0.0            | 0.0              | 0.0                | 0.0             | -710.1063      | -753.4531    | -1.6178         | -1.7566       |
-| 0.6931        | 0.03  | 50   | 0.6931          | 0.0            | 0.0              | 0.0                | 0.0             | -710.1063      | -753.4531    | -1.6178         | -1.7566       |
+| 0.7115        | 0.01  | 10   | 0.8102          | 0.4909         | 0.3420           | 0.6875             | 0.1489          | -240.0067      | -292.0018    | -2.5876         | -2.8529       |
+| 0.8377        | 0.01  | 20   | 0.7520          | 1.3853         | 0.4623           | 0.5625             | 0.9230          | -239.9265      | -291.4055    | -2.5997         | -2.8644       |
+| 1.7703        | 0.01  | 30   | 1.2673          | 2.0943         | 0.4701           | 0.5                | 1.6242          | -239.9213      | -290.9329    | -2.6152         | -2.8803       |
+| 4.3734        | 0.02  | 40   | 1.1677          | 3.0584         | 0.0188           | 0.5625             | 3.0396          | -240.2222      | -290.2901    | -2.6272         | -2.8860       |
+| 6.8893        | 0.03  | 50   | 1.3007          | 3.3485         | -0.5915          | 0.5625             | 3.9400          | -240.6290      | -290.0968    | -2.6366         | -2.8894       |
 
 
 ### Framework versions

adapter_config.json

@@ -19,8 +19,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "v_proj",
-    "q_proj"
+    "q_proj",
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM"
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c9a52614c9912d66ee94a410085b0f48dd16cd58becb67a6e9a1bcc9e7322987
+oid sha256:c18a3ef8cb66c0118d5e653070ac3e524057b58b21e039b86a488b08ebd18de6
 size 13648432

runs/Dec11_10-06-18_70b772f1ab48/events.out.tfevents.1702289192.70b772f1ab48.14693.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ddcb19028173bd5dd32d6eb1c0c1fd0d23d839c7806787ca86e0360b6e5484f
+size 12594

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2b97882146d62af5b65ef1c79b4483b39ab44d2f592e5386a256ac2f677f950d
+oid sha256:b7a1f3747b87802c2c225abe0faa47a5a588d700bc321029fd1d1c839f9e9d8c
 size 4155