File size: 5,213 Bytes
57b6f1a 55b3d89 57b6f1a 55b3d89 0a3a3a1 55b3d89 0a3a3a1 55b3d89 0a3a3a1 55b3d89 0a3a3a1 55b3d89 57b6f1a 55b3d89 57b6f1a 55b3d89 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 |
---
base_model: meta-llama/Llama-3.2-1B-Instruct
tags:
- ellora
- lora
- tool-calling
- function-calling
- code-exploration
- magpie
- peft
- llama
library_name: peft
license: apache-2.0
language:
- en
pipeline_tag: text-generation
inference: true
model_type: llama
---
# codelion/Llama-3.2-1B-Instruct-tool-calling-lora
## 🛠️ Tool Calling LoRA with Magpie
This LoRA adapter enhances meta-llama/Llama-3.2-1B-Instruct with tool calling capabilities for code exploration and manipulation. Trained using a hybrid Magpie + real execution approach on diverse coding scenarios.
## 🎯 Key Features
- **Tool Calling**: Teaches models to use development tools effectively
- **Code Exploration**: Navigate and understand unfamiliar codebases
- **Real Execution**: Training data generated from actual tool execution
- **OpenAI Format**: Compatible with OpenAI function calling format
- **Multi-Tool Sequences**: Learns to chain multiple tools for complex tasks
## 📊 Performance Metrics
- **Base Model**: meta-llama/Llama-3.2-1B-Instruct
- **Training Method**: Standard LoRA fine-tuning
- **LoRA Rank**: 64
- **LoRA Alpha**: 128
- **Training Samples**: 1000
- **Sequence Success Rate**: 80.0%
- **Tool Call Accuracy**: 80.0%
- **Average Tools per Sequence**: 4.0
## 🔧 Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Load base model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-3.2-1B-Instruct",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct")
# Load tool calling LoRA adapter
model = PeftModel.from_pretrained(model, "codelion/Llama-3.2-1B-Instruct-tool-calling-lora")
# Example: Use with tool calling prompt
prompt = '''You have access to the following tools:
- list_directory: List contents of a directory
- search_files: Search for files containing specific content
- read_file: Read a single file's contents
- get_file_info: Get file metadata
User: Help me understand how user authentication works in this Flask application
Response:
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```
## 📈 Expected Output Format
The model will generate tool calling sequences in OpenAI format:
```json
{
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {
"name": "search_files",
"arguments": "{\"query\": \"auth\", \"file_types\": [\".py\"]}"
}
},
{
"id": "call_2",
"type": "function",
"function": {
"name": "read_file",
"arguments": "{\"path\": \"app.py\"}"
}
}
]
}
```
## 🧪 Training Details
- **Method**: Standard LoRA fine-tuning with tool calling data
- **Data Generation**: Magpie scenarios + real tool execution
- **Tool Execution**: Safe sandbox environment for code exploration
- **Scenario Types**: Code exploration, bug hunting, feature addition, refactoring
- **Quality Validation**: Minimum tool usage and success rate thresholds
## 📚 Available Tools
The model is trained to use these development tools:
1. **list_directory**: Browse project structure
- Parameters: `path` (directory to list)
2. **search_files**: Find files containing specific content
- Parameters: `query`, `path`, `file_types`, `regex`
3. **read_file**: Read complete file contents
- Parameters: `path` (file to read)
4. **read_multiple_files**: Read multiple files at once
- Parameters: `paths` (list of files)
5. **get_file_info**: Get file metadata
- Parameters: `path` (file or directory)
6. **create_file**: Create new files (if safety mode disabled)
- Parameters: `path`, `content`
7. **edit_file**: Modify existing files (if safety mode disabled)
- Parameters: `path`, `changes`
## 🎭 Tool Usage Patterns Learned
- **Exploration First**: Start with `list_directory` to understand structure
- **Search Before Read**: Use `search_files` to find relevant files
- **Batch Operations**: Use `read_multiple_files` for related files
- **Progressive Refinement**: Start broad, then focus on specific files
## 🔬 Evaluation
The adapter was evaluated on diverse coding scenarios:
- Sequence success rate: 80.0%
- Tool call accuracy: 80.0%
- Average tools per sequence: 4.0
- Successful executions: 4/5
## 🏷️ Tool Usage Distribution
Most frequently used tools during evaluation:
- **get_file_info**: 5 uses
- **list_directory**: 5 uses
- **read_multiple_files**: 5 uses
- **search_files**: 5 uses
## 🏷️ Related
- **Dataset**: [codelion/Llama-3.2-1B-Instruct-magpie-tool-calling](https://huggingface.co/datasets/codelion/Llama-3.2-1B-Instruct-magpie-tool-calling)
- **Base Model**: [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
- **Framework**: [PEFT](https://github.com/huggingface/peft)
- **Training Approach**: Magpie + Real Execution
---
*This adapter is part of the [Ellora project](https://github.com/codelion/ellora) - standardized recipes for enhancing LLM capabilities.* |