End of training

README.md

@@ -51,7 +51,7 @@ sequence_len: 896
 sample_packing: false
 pad_to_sequence_len: true
 
-lora_r: 16
+lora_r: 32
 lora_alpha: 16
 lora_dropout: 0.05
 lora_target_linear: true
@@ -82,7 +82,7 @@ eval_batch_size: 16
 num_epochs: 6
 optimizer: adamw_bnb_8bit
 lr_scheduler: cosine
-learning_rate: 0.0005
+learning_rate: 0.0002
 max_grad_norm: 1.0
 adam_beta2: 0.95
 adam_epsilon: 0.00001
@@ -129,7 +129,7 @@ save_safetensors: true
 
 This model is a fine-tuned version of [deepseek-ai/deepseek-coder-1.3b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.3839
+- Loss: 0.4051
 
 ## Model description
 
@@ -148,7 +148,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0005
+- learning_rate: 0.0002
 - train_batch_size: 16
 - eval_batch_size: 16
 - seed: 49
@@ -162,27 +162,27 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
 | 1.8723        | 0.1429 | 1    | 1.6354          |
-| 1.9222        | 0.2857 | 2    | 1.6215          |
-| 1.6971        | 0.5714 | 4    | 1.4205          |
-| 1.2458        | 0.8571 | 6    | 0.9204          |
-| 0.6179        | 1.1429 | 8    | 0.6923          |
-| 0.366         | 1.4286 | 10   | 0.5647          |
-| 0.2752        | 1.7143 | 12   | 0.5225          |
-| 0.2931        | 2.0    | 14   | 0.5167          |
-| 0.1812        | 2.2857 | 16   | 0.4564          |
-| 0.1258        | 2.5714 | 18   | 0.4038          |
-| 0.0885        | 2.8571 | 20   | 0.3689          |
-| 0.0886        | 3.1429 | 22   | 0.3647          |
-| 0.1281        | 3.4286 | 24   | 0.3503          |
-| 0.0606        | 3.7143 | 26   | 0.3458          |
-| 0.0603        | 4.0    | 28   | 0.3635          |
-| 0.0479        | 4.2857 | 30   | 0.3724          |
-| 0.0963        | 4.5714 | 32   | 0.3827          |
-| 0.0725        | 4.8571 | 34   | 0.3868          |
-| 0.049         | 5.1429 | 36   | 0.3873          |
-| 0.0572        | 5.4286 | 38   | 0.3860          |
-| 0.061         | 5.7143 | 40   | 0.3890          |
-| 0.0702        | 6.0    | 42   | 0.3839          |
+| 1.9222        | 0.2857 | 2    | 1.6278          |
+| 1.7642        | 0.5714 | 4    | 1.5956          |
+| 1.8259        | 0.8571 | 6    | 1.4414          |
+| 1.334         | 1.1429 | 8    | 1.0972          |
+| 0.9019        | 1.4286 | 10   | 0.8305          |
+| 0.5977        | 1.7143 | 12   | 0.6896          |
+| 0.621         | 2.0    | 14   | 0.6125          |
+| 0.3513        | 2.2857 | 16   | 0.5361          |
+| 0.2399        | 2.5714 | 18   | 0.4976          |
+| 0.1689        | 2.8571 | 20   | 0.4783          |
+| 0.192         | 3.1429 | 22   | 0.4579          |
+| 0.1873        | 3.4286 | 24   | 0.4330          |
+| 0.1426        | 3.7143 | 26   | 0.4143          |
+| 0.0909        | 4.0    | 28   | 0.4106          |
+| 0.1129        | 4.2857 | 30   | 0.4111          |
+| 0.1584        | 4.5714 | 32   | 0.4084          |
+| 0.1479        | 4.8571 | 34   | 0.4041          |
+| 0.122         | 5.1429 | 36   | 0.4086          |
+| 0.1212        | 5.4286 | 38   | 0.4064          |
+| 0.1464        | 5.7143 | 40   | 0.4097          |
+| 0.0915        | 6.0    | 42   | 0.4051          |
 
 
 ### Framework versions