|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- vicgalle/alpaca-gpt4 |
|
language: |
|
- en |
|
--- |
|
This is a finetuned version microsoft phi - 2. This is finetuned using the alpaca dataset. |
|
|
|
|
|
## Training arguments |
|
|
|
num_train_epochs=1 |
|
|
|
per_device_train_batch_size=1 |
|
|
|
gradient_accumulation_steps=8 |
|
|
|
optim="paged_adamw_32bit" |
|
|
|
logging_steps=25 |
|
|
|
learning_rate=2e-4 |
|
|
|
weight_decay=0.001 |
|
|
|
max_grad_norm=0.3 |
|
|
|
warmup_ratio=0.03 |
|
|
|
lr_scheduler_type="cosine" |
|
|
|
## Prompt format: |
|
|
|
Use the below format |
|
|
|
``` |
|
Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction:{prompt} ### Response:{generated_output} |
|
``` |
|
|
