---
language:
- en
tags:
- llama-2
- peft
- qlora
- legal-ai
- fine-tuning
license: apache-2.0
datasets:
- nisaar/LLAMA2_Legal_Dataset_4.4k_Instructions
base_model: meta-llama/Llama-3.2-3B-Instruct
model-index:
- name: Legal-Llama-3.2-3B-Instruct
  results: []
pipeline_tag: text-generation
---
# sartajbhuvaji/Legal-Llama-3.2-3B-Instruct

This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct using QLoRA.

## Model description
Fine-tuned Llama 2 model for legal tasks

## Training Details
- Base Model: meta-llama/Llama-3.2-3B-Instruct
- Training Method: QLoRA (Quantized Low-Rank Adaptation)
- Framework: PEFT (Parameter-Efficient Fine-Tuning)
- Dataset: nisaar/LLAMA2_Legal_Dataset_4.4k_Instructions
- Training Date: 2024-12-29

## Intended Uses
This model is designed for legal domain tasks and should be used in accordance with the base model's intended use cases and limitations.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# Load adapter
model = PeftModel.from_pretrained(
    model,
    "sartajbhuvaji/Legal-Llama-3.2-3B-Instruct"
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# Format prompt
prompt = "<s> [INST] Your prompt here [/INST]"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
outputs = model.generate(**inputs, max_length=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Limitations
- This is a fine-tuned model and inherits the limitations of the base model
- The model's performance is limited to the scope and quality of the training dataset
- Legal advice generated by the model should not be considered as professional legal counsel

## Training Hyperparameters
- Learning rate: 2e-4
- Epochs: 3
- Batch size: 4
- LoRA rank: 32
- LoRA alpha: 16
- Gradient accumulation steps: 4

## Citation

```bibtex
@misc{your-model-name,
  author = {Your Name},
  title = {Your Model Title},
  year = {2024},
  publisher = {HuggingFace},
  journal = {HuggingFace Hub},
  howpublished = {\url{https://huggingface.co/sartajbhuvaji/Legal-Llama-3.2-3B-Instruct}}
}
```