Model save

README.md

@@ -17,15 +17,15 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [TheBloke/OpenHermes-2-Mistral-7B-GPTQ](https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GPTQ) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0196
-- Rewards/chosen: -0.2633
-- Rewards/rejected: -12.1300
-- Rewards/accuracies: 0.9950
-- Rewards/margins: 11.8667
-- Logps/rejected: -169.5972
-- Logps/chosen: -52.3002
-- Logits/rejected: -1.9113
-- Logits/chosen: -1.8049
+- Loss: 0.0522
+- Rewards/chosen: 0.2306
+- Rewards/rejected: -9.1473
+- Rewards/accuracies: 0.9940
+- Rewards/margins: 9.3779
+- Logps/rejected: -139.6582
+- Logps/chosen: -54.3255
+- Logits/rejected: -1.8763
+- Logits/chosen: -2.0675
 
 ## Model description
 
@@ -51,18 +51,16 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 2
-- training_steps: 50
+- training_steps: 30
 - mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
 |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.4709        | 0.01  | 10   | 0.0573          | 0.1661         | -3.3338          | 0.9900             | 3.4999          | -81.6352       | -48.0065     | -2.2004         | -2.0890       |
-| 0.4898        | 0.01  | 20   | 0.0222          | 0.0046         | -9.7065          | 0.9950             | 9.7111          | -145.3615      | -49.6211     | -2.0249         | -1.9125       |
-| 0.0           | 0.02  | 30   | 0.0191          | -0.2257        | -11.5488         | 0.9950             | 11.3231         | -163.7852      | -51.9239     | -1.9370         | -1.8274       |
-| 0.0           | 0.02  | 40   | 0.0189          | -0.3198        | -11.9637         | 0.9950             | 11.6439         | -167.9338      | -52.8651     | -1.9080         | -1.7998       |
-| 0.0007        | 0.03  | 50   | 0.0196          | -0.2633        | -12.1300         | 0.9950             | 11.8667         | -169.5972      | -52.3002     | -1.9113         | -1.8049       |
+| 0.4801        | 0.0   | 10   | 0.0657          | 0.3409         | -2.6602          | 0.9980             | 3.0011          | -74.7870       | -53.2224     | -2.0627         | -2.2254       |
+| 0.0563        | 0.0   | 20   | 0.0414          | 0.3013         | -7.6547          | 0.9940             | 7.9559          | -124.7320      | -53.6186     | -1.9099         | -2.1024       |
+| 0.0001        | 0.01  | 30   | 0.0522          | 0.2306         | -9.1473          | 0.9940             | 9.3779          | -139.6582      | -54.3255     | -1.8763         | -2.0675       |
 
 
 ### Framework versions
@@ -70,4 +68,4 @@ The following hyperparameters were used during training:
 - Transformers 4.35.2
 - Pytorch 2.0.1+cu117
 - Datasets 2.17.0
-- Tokenizers 0.15.1
+- Tokenizers 0.15.2

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:985a5ad5bf8c5b33189bf0bc1d7340d0a686015ca8f85a5df1f269c3d8bafb2e
+oid sha256:7c49e9afbd990f95bcb3ae34f3404ef0124a4fdbf7de7269d1e872d5060fb316
 size 13648432

runs/Feb17_22-12-20_1827e93c0501/events.out.tfevents.1708208020.1827e93c0501.461.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:572da1edc3460cf79d4856079ec28d3e8490a3ecf4a22d924982b279c3e33580
+size 9894

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2d83330c7ae406c30cf86ea19224c0ebbbace3825af747e9b1b657148115b82d
+oid sha256:8aa78da14ff45fc0bd0fc9fbbd08d77552ef3dabb1ac544e30460218605d0477
 size 4155