Hugging Face's logo

saneowl
/
phi-3-mini-LoRA

PEFT
Safetensors
trl
sft
Generated from Trainer
Model card Files
xet
Community
Oscar Wang commited on
Commit
31d6382
·
verified ·
1 Parent(s): 9bd7ba5

HuggingFaceUser/adapter-name

Browse files
Files changed (4) hide show
  1. README.md +54 -32
  2. adapter_config.json +3 -3
  3. adapter_model.safetensors +1 -1
  4. training_args.bin +1 -1
README.md CHANGED
@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.8285
22
 
23
  ## Model description
24
 
@@ -53,37 +53,59 @@ The following hyperparameters were used during training:
53
 
54
  | Training Loss | Epoch | Step | Validation Loss |
55
  |:-------------:|:------:|:----:|:---------------:|
56
- | 1.0352 | 0.0945 | 100 | 0.9701 |
57
- | 0.9209 | 0.1889 | 200 | 0.8977 |
58
- | 0.9124 | 0.2834 | 300 | 0.8801 |
59
- | 0.8715 | 0.3779 | 400 | 0.8686 |
60
- | 0.8671 | 0.4724 | 500 | 0.8635 |
61
- | 0.8636 | 0.5668 | 600 | 0.8580 |
62
- | 0.8573 | 0.6613 | 700 | 0.8544 |
63
- | 0.8698 | 0.7558 | 800 | 0.8529 |
64
- | 0.8456 | 0.8503 | 900 | 0.8493 |
65
- | 0.8388 | 0.9447 | 1000 | 0.8466 |
66
- | 0.8302 | 1.0392 | 1100 | 0.8451 |
67
- | 0.8486 | 1.1337 | 1200 | 0.8435 |
68
- | 0.8223 | 1.2282 | 1300 | 0.8418 |
69
- | 0.8291 | 1.3226 | 1400 | 0.8402 |
70
- | 0.8461 | 1.4171 | 1500 | 0.8385 |
71
- | 0.8315 | 1.5116 | 1600 | 0.8377 |
72
- | 0.8248 | 1.6060 | 1700 | 0.8365 |
73
- | 0.8239 | 1.7005 | 1800 | 0.8356 |
74
- | 0.8217 | 1.7950 | 1900 | 0.8348 |
75
- | 0.8108 | 1.8895 | 2000 | 0.8336 |
76
- | 0.8344 | 1.9839 | 2100 | 0.8326 |
77
- | 0.8074 | 2.0784 | 2200 | 0.8320 |
78
- | 0.8317 | 2.1729 | 2300 | 0.8316 |
79
- | 0.8042 | 2.2674 | 2400 | 0.8311 |
80
- | 0.8086 | 2.3618 | 2500 | 0.8308 |
81
- | 0.8214 | 2.4563 | 2600 | 0.8301 |
82
- | 0.8159 | 2.5508 | 2700 | 0.8297 |
83
- | 0.8179 | 2.6453 | 2800 | 0.8292 |
84
- | 0.8115 | 2.7397 | 2900 | 0.8289 |
85
- | 0.8263 | 2.8342 | 3000 | 0.8286 |
86
- | 0.812 | 2.9287 | 3100 | 0.8285 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
 
88
 
89
  ### Framework versions
 
18
 
19
  This model is a fine-tuned version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.8252
22
 
23
  ## Model description
24
 
 
53
 
54
  | Training Loss | Epoch | Step | Validation Loss |
55
  |:-------------:|:------:|:----:|:---------------:|
56
+ | 1.0774 | 0.0558 | 100 | 1.0228 |
57
+ | 0.9983 | 0.1117 | 200 | 0.9202 |
58
+ | 0.9227 | 0.1675 | 300 | 0.8962 |
59
+ | 0.8892 | 0.2234 | 400 | 0.8891 |
60
+ | 0.8833 | 0.2792 | 500 | 0.8780 |
61
+ | 0.9019 | 0.3351 | 600 | 0.8726 |
62
+ | 0.879 | 0.3909 | 700 | 0.8685 |
63
+ | 0.8604 | 0.4467 | 800 | 0.8648 |
64
+ | 0.8793 | 0.5026 | 900 | 0.8613 |
65
+ | 0.8608 | 0.5584 | 1000 | 0.8586 |
66
+ | 0.863 | 0.6143 | 1100 | 0.8564 |
67
+ | 0.8464 | 0.6701 | 1200 | 0.8538 |
68
+ | 0.88 | 0.7260 | 1300 | 0.8514 |
69
+ | 0.8533 | 0.7818 | 1400 | 0.8498 |
70
+ | 0.8532 | 0.8376 | 1500 | 0.8488 |
71
+ | 0.871 | 0.8935 | 1600 | 0.8468 |
72
+ | 0.8546 | 0.9493 | 1700 | 0.8456 |
73
+ | 0.853 | 1.0052 | 1800 | 0.8437 |
74
+ | 0.8427 | 1.0610 | 1900 | 0.8437 |
75
+ | 0.834 | 1.1169 | 2000 | 0.8423 |
76
+ | 0.8443 | 1.1727 | 2100 | 0.8410 |
77
+ | 0.8487 | 1.2285 | 2200 | 0.8401 |
78
+ | 0.8257 | 1.2844 | 2300 | 0.8387 |
79
+ | 0.835 | 1.3402 | 2400 | 0.8382 |
80
+ | 0.8331 | 1.3961 | 2500 | 0.8375 |
81
+ | 0.8369 | 1.4519 | 2600 | 0.8363 |
82
+ | 0.8209 | 1.5077 | 2700 | 0.8352 |
83
+ | 0.8443 | 1.5636 | 2800 | 0.8350 |
84
+ | 0.8505 | 1.6194 | 2900 | 0.8352 |
85
+ | 0.8284 | 1.6753 | 3000 | 0.8339 |
86
+ | 0.8544 | 1.7311 | 3100 | 0.8325 |
87
+ | 0.8429 | 1.7870 | 3200 | 0.8321 |
88
+ | 0.8223 | 1.8428 | 3300 | 0.8312 |
89
+ | 0.8259 | 1.8986 | 3400 | 0.8303 |
90
+ | 0.8379 | 1.9545 | 3500 | 0.8300 |
91
+ | 0.8407 | 2.0103 | 3600 | 0.8294 |
92
+ | 0.8331 | 2.0662 | 3700 | 0.8294 |
93
+ | 0.8279 | 2.1220 | 3800 | 0.8294 |
94
+ | 0.8321 | 2.1779 | 3900 | 0.8289 |
95
+ | 0.819 | 2.2337 | 4000 | 0.8284 |
96
+ | 0.8289 | 2.2895 | 4100 | 0.8277 |
97
+ | 0.8222 | 2.3454 | 4200 | 0.8275 |
98
+ | 0.8039 | 2.4012 | 4300 | 0.8273 |
99
+ | 0.8015 | 2.4571 | 4400 | 0.8271 |
100
+ | 0.8148 | 2.5129 | 4500 | 0.8267 |
101
+ | 0.8217 | 2.5688 | 4600 | 0.8263 |
102
+ | 0.8005 | 2.6246 | 4700 | 0.8262 |
103
+ | 0.8146 | 2.6804 | 4800 | 0.8258 |
104
+ | 0.8229 | 2.7363 | 4900 | 0.8258 |
105
+ | 0.8266 | 2.7921 | 5000 | 0.8256 |
106
+ | 0.8222 | 2.8480 | 5100 | 0.8254 |
107
+ | 0.8238 | 2.9038 | 5200 | 0.8253 |
108
+ | 0.8296 | 2.9597 | 5300 | 0.8252 |
109
 
110
 
111
  ### Framework versions
adapter_config.json CHANGED
@@ -20,11 +20,11 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "o_proj",
24
- "v_proj",
25
  "gate_proj",
 
 
26
  "k_proj",
27
- "up_proj",
28
  "down_proj",
29
  "q_proj"
30
  ],
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
+ "up_proj",
 
24
  "gate_proj",
25
+ "v_proj",
26
+ "o_proj",
27
  "k_proj",
 
28
  "down_proj",
29
  "q_proj"
30
  ],
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6d31d1e65b3fe58a9a37830eae1350aaf50c30efa305fa0d5b809d8710f5c653
3
  size 35668592
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d969cf83007e2f38afa917447d506bb73c1306e8f99471b4528746df4bda9fb
3
  size 35668592
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e9a4c1475d4cecd74622b72c919effef5b0e17bb6835fe1fa8541e3e43c5074e
3
  size 5432
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ccd41c3d8514bb34a59303e4c8d864f117f2394c4cfcda1fefca3c1887041a45
3
  size 5432