Hugging Face's logo

thrunlab
/
sparse_llama_7b_hf_refined_web_50p_2024-03-24

Text Generation
Transformers
Safetensors
sparse_llama
Generated from Trainer
custom_code
Model card Files Community
lukeleeai commited on
Commit
487290b
·
verified ·
1 Parent(s): 28fb8fb

End of training

Browse files
Files changed (4) hide show
  1. README.md +10 -2
  2. model-00001-of-00003.safetensors +1 -1
  3. model-00002-of-00003.safetensors +1 -1
  4. model-00003-of-00003.safetensors +1 -1
README.md CHANGED
@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
14
 
15
  This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on the None dataset.
16
  It achieves the following results on the evaluation set:
17
- - Loss: 2.1427
18
 
19
  ## Model description
20
 
@@ -44,7 +44,7 @@ The following hyperparameters were used during training:
44
  - total_eval_batch_size: 4
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
- - training_steps: 200
48
 
49
  ### Training results
50
 
@@ -58,6 +58,14 @@ The following hyperparameters were used during training:
58
  | 2.1753 | 0.05 | 150 | 2.4039 |
59
  | 2.1433 | 0.06 | 175 | 2.3947 |
60
  | 2.3055 | 0.06 | 200 | 2.3859 |
 
 
 
 
 
 
 
 
61
 
62
 
63
  ### Framework versions
 
14
 
15
  This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on the None dataset.
16
  It achieves the following results on the evaluation set:
17
+ - Loss: 2.1220
18
 
19
  ## Model description
20
 
 
44
  - total_eval_batch_size: 4
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
+ - training_steps: 400
48
 
49
  ### Training results
50
 
 
58
  | 2.1753 | 0.05 | 150 | 2.4039 |
59
  | 2.1433 | 0.06 | 175 | 2.3947 |
60
  | 2.3055 | 0.06 | 200 | 2.3859 |
61
+ | 2.2679 | 0.07 | 225 | 2.3842 |
62
+ | 2.2177 | 0.08 | 250 | 2.3817 |
63
+ | 2.1572 | 0.09 | 275 | 2.3830 |
64
+ | 2.1926 | 0.1 | 300 | 2.3829 |
65
+ | 2.2406 | 0.1 | 325 | 2.3817 |
66
+ | 2.21 | 0.11 | 350 | 2.3771 |
67
+ | 2.1296 | 0.12 | 375 | 2.3797 |
68
+ | 2.232 | 0.13 | 400 | 2.3764 |
69
 
70
 
71
  ### Framework versions
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:012374630d39b8a757514b733372df8fab30fda05802d8be2cc35a482c2eb450
3
  size 4938985352
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e029e1f68013a12923fadd920b746a7c7ca85626ed8681299a21f207ced72307
3
  size 4938985352
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:56f06eb5e2acbccc85cb92e9d75991830eaac4577b5076ec89caa937d225d973
3
  size 4947390880
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1843fb8bee355f8b9b610196d874ef458240eb65ce84555b6aa4862db0a54d5e
3
  size 4947390880
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c164b9a9166186e4eee313cb3703baf5ec6a3fbf32ef08fc2cdbe33b3d328bad
3
  size 3590488816
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2cf27e89867640243ec1430eff0b6e75ea870b2158523458c590d2774e88f37b
3
  size 3590488816