End of training
Browse files
README.md
CHANGED
@@ -60,7 +60,7 @@ wandb_log_model:
|
|
60 |
|
61 |
gradient_accumulation_steps: 4
|
62 |
micro_batch_size: 1
|
63 |
-
num_epochs:
|
64 |
optimizer: adamw_torch
|
65 |
lr_scheduler: cosine
|
66 |
learning_rate: 0.0002
|
@@ -99,7 +99,7 @@ special_tokens:
|
|
99 |
|
100 |
</details><br>
|
101 |
|
102 |
-
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/afrias5/Code32bTDualGPU/runs/
|
103 |
# codellama-cont2
|
104 |
|
105 |
This model is a fine-tuned version of [codellama/CodeLlama-34b-Python-hf](https://huggingface.co/codellama/CodeLlama-34b-Python-hf) on the None dataset.
|
@@ -133,7 +133,7 @@ The following hyperparameters were used during training:
|
|
133 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
134 |
- lr_scheduler_type: cosine
|
135 |
- lr_scheduler_warmup_steps: 10
|
136 |
-
- num_epochs:
|
137 |
|
138 |
### Training results
|
139 |
|
|
|
60 |
|
61 |
gradient_accumulation_steps: 4
|
62 |
micro_batch_size: 1
|
63 |
+
num_epochs: 8
|
64 |
optimizer: adamw_torch
|
65 |
lr_scheduler: cosine
|
66 |
learning_rate: 0.0002
|
|
|
99 |
|
100 |
</details><br>
|
101 |
|
102 |
+
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/afrias5/Code32bTDualGPU/runs/oh019yiu)
|
103 |
# codellama-cont2
|
104 |
|
105 |
This model is a fine-tuned version of [codellama/CodeLlama-34b-Python-hf](https://huggingface.co/codellama/CodeLlama-34b-Python-hf) on the None dataset.
|
|
|
133 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
134 |
- lr_scheduler_type: cosine
|
135 |
- lr_scheduler_warmup_steps: 10
|
136 |
+
- num_epochs: 8
|
137 |
|
138 |
### Training results
|
139 |
|