README.md

# phi-1_5-qlora-alpaca-instruction Model Card

## Model Description

This model is a causal language model based on the `microsoft/phi-1_5` and has been finetuned using QLORA technology on the `vicgalle/alpaca-gpt4` dataset.

## Fine-tuning Details
- **Base Model**: `microsoft/phi-1_5`
- **Fine-tuning Dataset**: `vicgalle/alpaca-gpt4`
- **Hardware**: NVIDIA 3090ti
- **Training Duration**: 8 hours
- **VRAM Consumption**: Approx. 20 GB for 14 hours
- **Token Max Length**: 2048
- **Model Size**: 1.5billion + qlora weights merged

### Hyperparameters

```python
# Lora Configuration
config = LoraConfig(
    r=16,
    lora_alpha=16,
    target_modules=["Wqkv", "out_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

# Training Hyperparameters
training_arguments = TrainingArguments(
        output_dir=f"{local_path}/output_dir",
        per_device_train_batch_size=4,
        gradient_accumulation_steps=6,
        learning_rate=2e-4,
        lr_scheduler_type="cosine",
        evaluation_strategy = "steps",
        eval_steps=500,
        save_strategy="epoch",
        logging_steps=100,
        num_train_epochs=6,
        report_to = 'wandb',
        run_name = run_name
    )

```

## Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "nps798/phi-1_5-qlora-alpaca-instruction"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map={"": 0},
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    model_name,
    trust_remote_code=True
)

prompt= """Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Choose three places you would like to visit and explain why.

### Response:"""
inputs = tokenizer(prompt, return_tensors="pt", return_attention_mask=False)
outputs = model.generate(**inputs, max_length=500)
text = tokenizer.batch_decode(outputs)[0]
print(text)

```

## License
Because the base model is microsoft phi-1.5b model, this fine-tuned model is provided under the MICROSOFT RESEARCH LICENSE and is meant for non-commercial use only.

## Author
I am a medical doctor interested in ML/NLP field. 
If you have any advice, suggestions, or opportunities, or simply want to discuss the fascinating intersection of medicine and technology, please don't hesitate to reach out.