README.md

---
base_model: meta-llama/Llama-3.2-1B-Instruct
tags:
- ellora
- lora
- tool-calling
- function-calling
- code-exploration
- magpie
- peft
- llama
library_name: peft
license: apache-2.0
language:
- en
pipeline_tag: text-generation
inference: true
model_type: llama
---
# codelion/Llama-3.2-1B-Instruct-tool-calling-lora

## 🛠️ Tool Calling LoRA with Magpie

This LoRA adapter enhances meta-llama/Llama-3.2-1B-Instruct with tool calling capabilities for code exploration and manipulation. Trained using a hybrid Magpie + real execution approach on diverse coding scenarios.

## 🎯 Key Features

- **Tool Calling**: Teaches models to use development tools effectively
- **Code Exploration**: Navigate and understand unfamiliar codebases
- **Real Execution**: Training data generated from actual tool execution
- **OpenAI Format**: Compatible with OpenAI function calling format
- **Multi-Tool Sequences**: Learns to chain multiple tools for complex tasks

## 📊 Performance Metrics

- **Base Model**: meta-llama/Llama-3.2-1B-Instruct
- **Training Method**: Standard LoRA fine-tuning
- **LoRA Rank**: 64
- **LoRA Alpha**: 128
- **Training Samples**: 1000
- **Sequence Success Rate**: 80.0%
- **Tool Call Accuracy**: 80.0%
- **Average Tools per Sequence**: 4.0
## 🔧 Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-1B-Instruct",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct")

# Load tool calling LoRA adapter
model = PeftModel.from_pretrained(model, "codelion/Llama-3.2-1B-Instruct-tool-calling-lora")

# Example: Use with tool calling prompt
prompt = '''You have access to the following tools:
- list_directory: List contents of a directory
- search_files: Search for files containing specific content
- read_file: Read a single file's contents
- get_file_info: Get file metadata

User: Help me understand how user authentication works in this Flask application
Response:

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```
## 📈 Expected Output Format

The model will generate tool calling sequences in OpenAI format:

```json
{
  "tool_calls": [
    {
      "id": "call_1",
      "type": "function",
      "function": {
        "name": "search_files",
        "arguments": "{\"query\": \"auth\", \"file_types\": [\".py\"]}"
      }
    },
    {
      "id": "call_2",
      "type": "function",
      "function": {
        "name": "read_file",
        "arguments": "{\"path\": \"app.py\"}"
      }
    }
  ]
}
```
## 🧪 Training Details

- **Method**: Standard LoRA fine-tuning with tool calling data
- **Data Generation**: Magpie scenarios + real tool execution
- **Tool Execution**: Safe sandbox environment for code exploration
- **Scenario Types**: Code exploration, bug hunting, feature addition, refactoring
- **Quality Validation**: Minimum tool usage and success rate thresholds

## 📚 Available Tools

The model is trained to use these development tools:

1. **list_directory**: Browse project structure
   - Parameters: `path` (directory to list)

2. **search_files**: Find files containing specific content
   - Parameters: `query`, `path`, `file_types`, `regex`

3. **read_file**: Read complete file contents
   - Parameters: `path` (file to read)

4. **read_multiple_files**: Read multiple files at once
   - Parameters: `paths` (list of files)

5. **get_file_info**: Get file metadata
   - Parameters: `path` (file or directory)

6. **create_file**: Create new files (if safety mode disabled)
   - Parameters: `path`, `content`

7. **edit_file**: Modify existing files (if safety mode disabled)
   - Parameters: `path`, `changes`

## 🎭 Tool Usage Patterns Learned

- **Exploration First**: Start with `list_directory` to understand structure
- **Search Before Read**: Use `search_files` to find relevant files
- **Batch Operations**: Use `read_multiple_files` for related files
- **Progressive Refinement**: Start broad, then focus on specific files
## 🔬 Evaluation

The adapter was evaluated on diverse coding scenarios:
- Sequence success rate: 80.0%
- Tool call accuracy: 80.0%
- Average tools per sequence: 4.0
- Successful executions: 4/5

## 🏷️ Tool Usage Distribution

Most frequently used tools during evaluation:
- **get_file_info**: 5 uses
- **list_directory**: 5 uses
- **read_multiple_files**: 5 uses
- **search_files**: 5 uses

## 🏷️ Related

- **Dataset**: [codelion/Llama-3.2-1B-Instruct-magpie-tool-calling](https://huggingface.co/datasets/codelion/Llama-3.2-1B-Instruct-magpie-tool-calling)
- **Base Model**: [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
- **Framework**: [PEFT](https://github.com/huggingface/peft)
- **Training Approach**: Magpie + Real Execution

---

*This adapter is part of the [Ellora project](https://github.com/codelion/ellora) - standardized recipes for enhancing LLM capabilities.*