README.md

---
language:
- en
- ko
tags:
- qwen
- lora
- rag
- instruction-tuning
- email
- qwen-2.5
- peft
- question-answering
library_name: peft
pipeline_tag: text-generation
license: mit
---

# Qwen-RAG-LoRA

This repository contains LoRA weights for [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), fine-tuned for email-based question answering tasks. The model has been trained to handle both English and Korean queries about email content.

## Model Description

- **Base Model:** Qwen/Qwen2.5-7B-Instruct
- **Training Type:** LoRA (Low-Rank Adaptation)
- **Checkpoint:** checkpoint-600
- **Languages:** English and Korean
- **Task:** Email-based Question Answering
- **Domain:** Email Content

## Training Details

### LoRA Configuration
```python
lora_config = LoraConfig(
    r=8,
    lora_alpha=32,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_dropout=0.1,
    bias="none",
    task_type="CAUSAL_LM"
)
```

## Usage with vLLM

```python
from vllm import LLM, SamplingParams
from vllm.lora.request import LoRARequest

# Initialize LLM with LoRA support
llm = LLM(
    model="Qwen/Qwen2.5-7B-Instruct",
    tensor_parallel_size=2,
    enable_lora=True
)

sampling_params = SamplingParams(temperature=0.0, max_tokens=50)

# Create LoRA request
lora_request = LoRARequest(
    "rag_adapter",
    1,
    "doubleyyh/qwen-rag-lora"  # HuggingFace repo name
)

# Example prompt
prompt = """Using the context provided below, answer the question concisely. Respond in Korean if the question is in Korean, and in English if the question is in English.

Context: subject: Meeting Schedule Update
from: [['John Smith', '[email protected]']]
to: [['Team', '[email protected]']]
text_body: The project review meeting is rescheduled to 3 PM tomorrow.

Question: When is the meeting rescheduled to?

Answer: """

# Generate with LoRA
outputs = llm.generate([prompt], sampling_params, lora_request=lora_request)
print(outputs[0].outputs[0].text)
```

## Example Input/Output

```
# English Query
Q: When is the project review scheduled?
A: The project review meeting is rescheduled to 3 PM tomorrow.

# Korean Query
Q: 프로젝트 미팅이 언제로 변경되었나요?
A: 내일 오후 3시로 변경되었습니다.
```

## Limitations

- The model is specifically trained for email-related queries
- Performance might vary between English and Korean
- Optimal results when used with email content in standard format
- Limited to the capabilities of the base Qwen model

## Citation

```bibtex
@misc{qwen-rag-lora,
  author = {doubleyyh},
  title = {Qwen-RAG-LoRA: Fine-tuned LoRA Weights for Email QA},
  year = {2024},
  publisher = {Hugging Face}
}
```

## License

This model follows the same license as Qwen2.5-7B-Instruct.