File size: 638 Bytes
d5dfbac 1d10463 d5dfbac 1d10463 e8854f3 1d10463 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
---
license: apache-2.0
datasets:
- vicgalle/alpaca-gpt4
language:
- en
---
This is a finetuned version microsoft phi - 2. This is finetuned using the alpaca dataset.
## Training arguments
num_train_epochs=1
per_device_train_batch_size=1
gradient_accumulation_steps=8
optim="paged_adamw_32bit"
logging_steps=25
learning_rate=2e-4
weight_decay=0.001
max_grad_norm=0.3
warmup_ratio=0.03
lr_scheduler_type="cosine"
## Prompt format:
Use the below format
```
Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction:{prompt} ### Response:{generated_output}
```
|