Hugging Face's logo

yalhessi
/
lemexp-task1-min_symbols_template_small-deepseek-coder-1.3b-base-ddp

PEFT
Safetensors
Generated from Trainer
Model card Files Community
yalhessi commited on
Commit
247aed8
·
verified ·
1 Parent(s): f6d8ce8

End of training

Browse files
Files changed (5) hide show
  1. README.md +64 -34
  2. adapter_config.json +2 -2
  3. adapter_model.safetensors +1 -1
  4. loss_plot.png +0 -0
  5. training_args.bin +1 -1
README.md CHANGED
@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [deepseek-ai/deepseek-coder-1.3b-base](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 0.1887
20
 
21
  ## Model description
22
 
@@ -35,7 +35,7 @@ More information needed
35
  ### Training hyperparameters
36
 
37
  The following hyperparameters were used during training:
38
- - learning_rate: 0.0001
39
  - train_batch_size: 2
40
  - eval_batch_size: 2
41
  - seed: 42
@@ -45,42 +45,72 @@ The following hyperparameters were used during training:
45
  - total_eval_batch_size: 16
46
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
47
  - lr_scheduler_type: linear
48
- - num_epochs: 6
49
  - mixed_precision_training: Native AMP
50
 
51
  ### Training results
52
 
53
- | Training Loss | Epoch | Step | Validation Loss |
54
- |:-------------:|:------:|:-----:|:---------------:|
55
- | 0.4557 | 0.2001 | 629 | 0.3633 |
56
- | 0.3618 | 0.4001 | 1258 | 0.3296 |
57
- | 0.3388 | 0.6002 | 1887 | 0.3132 |
58
- | 0.3131 | 0.8003 | 2516 | 0.2975 |
59
- | 0.3022 | 1.0003 | 3145 | 0.2878 |
60
- | 0.2884 | 1.2004 | 3774 | 0.2849 |
61
- | 0.2806 | 1.4004 | 4403 | 0.2791 |
62
- | 0.2695 | 1.6005 | 5032 | 0.2651 |
63
- | 0.2684 | 1.8006 | 5661 | 0.2560 |
64
- | 0.261 | 2.0006 | 6290 | 0.2564 |
65
- | 0.2544 | 2.2007 | 6919 | 0.2513 |
66
- | 0.2437 | 2.4008 | 7548 | 0.2441 |
67
- | 0.2393 | 2.6008 | 8177 | 0.2406 |
68
- | 0.2375 | 2.8009 | 8806 | 0.2338 |
69
- | 0.2326 | 3.0010 | 9435 | 0.2257 |
70
- | 0.2124 | 3.2010 | 10064 | 0.2227 |
71
- | 0.2137 | 3.4011 | 10693 | 0.2215 |
72
- | 0.2102 | 3.6011 | 11322 | 0.2127 |
73
- | 0.2079 | 3.8012 | 11951 | 0.2103 |
74
- | 0.2034 | 4.0013 | 12580 | 0.2070 |
75
- | 0.1862 | 4.2013 | 13209 | 0.2049 |
76
- | 0.1831 | 4.4014 | 13838 | 0.2029 |
77
- | 0.185 | 4.6015 | 14467 | 0.1987 |
78
- | 0.1754 | 4.8015 | 15096 | 0.1975 |
79
- | 0.1753 | 5.0016 | 15725 | 0.1937 |
80
- | 0.1622 | 5.2017 | 16354 | 0.1959 |
81
- | 0.155 | 5.4017 | 16983 | 0.1912 |
82
- | 0.1501 | 5.6018 | 17612 | 0.1897 |
83
- | 0.1481 | 5.8018 | 18241 | 0.1887 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84
 
85
 
86
  ### Framework versions
 
16
 
17
  This model is a fine-tuned version of [deepseek-ai/deepseek-coder-1.3b-base](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 0.2115
20
 
21
  ## Model description
22
 
 
35
  ### Training hyperparameters
36
 
37
  The following hyperparameters were used during training:
38
+ - learning_rate: 0.0002
39
  - train_batch_size: 2
40
  - eval_batch_size: 2
41
  - seed: 42
 
45
  - total_eval_batch_size: 16
46
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
47
  - lr_scheduler_type: linear
48
+ - num_epochs: 12
49
  - mixed_precision_training: Native AMP
50
 
51
  ### Training results
52
 
53
+ | Training Loss | Epoch | Step | Validation Loss |
54
+ |:-------------:|:-------:|:-----:|:---------------:|
55
+ | 0.5172 | 0.2001 | 629 | 0.3780 |
56
+ | 0.3747 | 0.4001 | 1258 | 0.3336 |
57
+ | 0.3399 | 0.6002 | 1887 | 0.3039 |
58
+ | 0.3014 | 0.8003 | 2516 | 0.2870 |
59
+ | 0.2891 | 1.0003 | 3145 | 0.2798 |
60
+ | 0.2721 | 1.2004 | 3774 | 0.2671 |
61
+ | 0.2643 | 1.4004 | 4403 | 0.2577 |
62
+ | 0.2522 | 1.6005 | 5032 | 0.2514 |
63
+ | 0.2496 | 1.8006 | 5661 | 0.2455 |
64
+ | 0.2426 | 2.0006 | 6290 | 0.2414 |
65
+ | 0.2365 | 2.2007 | 6919 | 0.2360 |
66
+ | 0.2267 | 2.4008 | 7548 | 0.2346 |
67
+ | 0.2223 | 2.6008 | 8177 | 0.2306 |
68
+ | 0.2225 | 2.8009 | 8806 | 0.2264 |
69
+ | 0.218 | 3.0010 | 9435 | 0.2238 |
70
+ | 0.2003 | 3.2010 | 10064 | 0.2245 |
71
+ | 0.2015 | 3.4011 | 10693 | 0.2217 |
72
+ | 0.2019 | 3.6011 | 11322 | 0.2167 |
73
+ | 0.2 | 3.8012 | 11951 | 0.2136 |
74
+ | 0.1991 | 4.0013 | 12580 | 0.2127 |
75
+ | 0.1848 | 4.2013 | 13209 | 0.2149 |
76
+ | 0.1831 | 4.4014 | 13838 | 0.2123 |
77
+ | 0.1867 | 4.6015 | 14467 | 0.2114 |
78
+ | 0.1819 | 4.8015 | 15096 | 0.2091 |
79
+ | 0.1833 | 5.0016 | 15725 | 0.2065 |
80
+ | 0.1741 | 5.2017 | 16354 | 0.2082 |
81
+ | 0.1699 | 5.4017 | 16983 | 0.2067 |
82
+ | 0.1689 | 5.6018 | 17612 | 0.2049 |
83
+ | 0.1692 | 5.8018 | 18241 | 0.2029 |
84
+ | 0.1703 | 6.0019 | 18870 | 0.2057 |
85
+ | 0.1649 | 6.2020 | 19499 | 0.2055 |
86
+ | 0.1563 | 6.4020 | 20128 | 0.2076 |
87
+ | 0.1568 | 6.6021 | 20757 | 0.2033 |
88
+ | 0.1559 | 6.8022 | 21386 | 0.2007 |
89
+ | 0.159 | 7.0022 | 22015 | 0.2013 |
90
+ | 0.1446 | 7.2023 | 22644 | 0.2051 |
91
+ | 0.1455 | 7.4024 | 23273 | 0.2031 |
92
+ | 0.147 | 7.6024 | 23902 | 0.2053 |
93
+ | 0.1478 | 7.8025 | 24531 | 0.1988 |
94
+ | 0.1462 | 8.0025 | 25160 | 0.2023 |
95
+ | 0.1371 | 8.2026 | 25789 | 0.2073 |
96
+ | 0.1374 | 8.4027 | 26418 | 0.2018 |
97
+ | 0.1382 | 8.6027 | 27047 | 0.2013 |
98
+ | 0.136 | 8.8028 | 27676 | 0.2036 |
99
+ | 0.1396 | 9.0029 | 28305 | 0.2004 |
100
+ | 0.1335 | 9.2029 | 28934 | 0.2069 |
101
+ | 0.1278 | 9.4030 | 29563 | 0.2079 |
102
+ | 0.1295 | 9.6031 | 30192 | 0.2053 |
103
+ | 0.1303 | 9.8031 | 30821 | 0.2066 |
104
+ | 0.129 | 10.0032 | 31450 | 0.2040 |
105
+ | 0.1208 | 10.2032 | 32079 | 0.2099 |
106
+ | 0.1219 | 10.4033 | 32708 | 0.2096 |
107
+ | 0.1219 | 10.6034 | 33337 | 0.2105 |
108
+ | 0.1217 | 10.8034 | 33966 | 0.2082 |
109
+ | 0.1208 | 11.0035 | 34595 | 0.2093 |
110
+ | 0.1181 | 11.2036 | 35224 | 0.2118 |
111
+ | 0.1154 | 11.4036 | 35853 | 0.2144 |
112
+ | 0.115 | 11.6037 | 36482 | 0.2130 |
113
+ | 0.1156 | 11.8038 | 37111 | 0.2115 |
114
 
115
 
116
  ### Framework versions
adapter_config.json CHANGED
@@ -23,8 +23,8 @@
23
  "rank_pattern": {},
24
  "revision": null,
25
  "target_modules": [
26
- "v_proj",
27
- "q_proj"
28
  ],
29
  "task_type": "CAUSAL_LM",
30
  "use_dora": false,
 
23
  "rank_pattern": {},
24
  "revision": null,
25
  "target_modules": [
26
+ "q_proj",
27
+ "v_proj"
28
  ],
29
  "task_type": "CAUSAL_LM",
30
  "use_dora": false,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4ce2053ad077db869539c5c08a26aa44feec65cfde1c470aeb7db1903450f803
3
  size 6304096
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c97fdca213661a02ef2feacd877e07554c1d6ae0118ca7ae5f3a126ebca35a21
3
  size 6304096
loss_plot.png CHANGED
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:53ab949c10ae7ba63431c493210572475108b01f1d56e20fd56bc36852046090
3
  size 5496
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1afd4fd260729fd9d9f186d503014df58543245587b3d8c90834f4f67408923a
3
  size 5496