Hugging Face's logo

Heejindo
/
rationale_model_e10_save5000

Text Generation
Transformers
Safetensors
llama
trl
sft
Generated from Trainer
text-generation-inference
Model card Files Community
Heejindo commited on
Commit
83808c0
·
verified ·
1 Parent(s): 679b0e9

training_output/rationale_model_e10_save5000/

Browse files
Files changed (2) hide show
  1. README.md +15 -25
  2. model.safetensors +1 -1
README.md CHANGED
@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [meta-llama/Llama-3.2-1B](https://huggingface.co/meta-llama/Llama-3.2-1B) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 2.0836
22
 
23
  ## Model description
24
 
@@ -38,8 +38,8 @@ More information needed
38
 
39
  The following hyperparameters were used during training:
40
  - learning_rate: 1e-05
41
- - train_batch_size: 4
42
- - eval_batch_size: 4
43
  - seed: 42
44
  - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
  - lr_scheduler_type: linear
@@ -47,28 +47,18 @@ The following hyperparameters were used during training:
47
 
48
  ### Training results
49
 
50
- | Training Loss | Epoch | Step | Validation Loss |
51
- |:-------------:|:------:|:------:|:---------------:|
52
- | 1.0612 | 0.4769 | 5000 | 2.0836 |
53
- | 0.3248 | 0.9538 | 10000 | 2.7452 |
54
- | 0.1833 | 1.4308 | 15000 | 2.9985 |
55
- | 0.1503 | 1.9077 | 20000 | 3.1807 |
56
- | 0.1285 | 2.3846 | 25000 | 3.3188 |
57
- | 0.1204 | 2.8615 | 30000 | 3.4080 |
58
- | 0.1049 | 3.3384 | 35000 | 3.5038 |
59
- | 0.102 | 3.8153 | 40000 | 3.5999 |
60
- | 0.0865 | 4.2923 | 45000 | 3.7076 |
61
- | 0.0844 | 4.7692 | 50000 | 3.7838 |
62
- | 0.0664 | 5.2461 | 55000 | 3.9114 |
63
- | 0.0671 | 5.7230 | 60000 | 4.0266 |
64
- | 0.051 | 6.1999 | 65000 | 4.1414 |
65
- | 0.0515 | 6.6768 | 70000 | 4.2532 |
66
- | 0.0422 | 7.1538 | 75000 | 4.3879 |
67
- | 0.0409 | 7.6307 | 80000 | 4.4931 |
68
- | 0.0352 | 8.1076 | 85000 | 4.6297 |
69
- | 0.0356 | 8.5845 | 90000 | 4.7444 |
70
- | 0.0322 | 9.0614 | 95000 | 4.8618 |
71
- | 0.0328 | 9.5383 | 100000 | 4.9179 |
72
 
73
 
74
  ### Framework versions
 
18
 
19
  This model is a fine-tuned version of [meta-llama/Llama-3.2-1B](https://huggingface.co/meta-llama/Llama-3.2-1B) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 2.6975
22
 
23
  ## Model description
24
 
 
38
 
39
  The following hyperparameters were used during training:
40
  - learning_rate: 1e-05
41
+ - train_batch_size: 8
42
+ - eval_batch_size: 8
43
  - seed: 42
44
  - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
  - lr_scheduler_type: linear
 
47
 
48
  ### Training results
49
 
50
+ | Training Loss | Epoch | Step | Validation Loss |
51
+ |:-------------:|:------:|:-----:|:---------------:|
52
+ | 0.3986 | 0.9538 | 5000 | 2.6975 |
53
+ | 0.1468 | 1.9077 | 10000 | 3.2221 |
54
+ | 0.1156 | 2.8615 | 15000 | 3.4922 |
55
+ | 0.0981 | 3.8153 | 20000 | 3.6490 |
56
+ | 0.0847 | 4.7692 | 25000 | 3.8345 |
57
+ | 0.0704 | 5.7230 | 30000 | 3.9968 |
58
+ | 0.0551 | 6.6768 | 35000 | 4.2504 |
59
+ | 0.0433 | 7.6307 | 40000 | 4.5271 |
60
+ | 0.0354 | 8.5845 | 45000 | 4.7534 |
61
+ | 0.0317 | 9.5383 | 50000 | 4.9696 |
 
 
 
 
 
 
 
 
 
 
62
 
63
 
64
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:db60b6ff3e9ba57f952db8a25071b16b580a952c56dae2960635b25db012ef6c
3
  size 4943274328
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1ef7f2d1334d8ffb94dd219cd72b507f7388478a04e9730bb1ca1c813996c1e
3
  size 4943274328