---
tags:
- unsloth
---

# Llama 3.1 Merged Adapters

## Model Description
This is a merged model combining multiple fine-tuned LoRA adapters using TIES (TRIM, ELECT SIGN & MERGE) merging technique. The model combines the strengths of multiple specialized adapters into a single model.

## Base Model
- [unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit](https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit)

## Merged Adapters
The following adapters were merged:
- llama337 - Specialized for rewrite and revision
- llama338 - Specialized for rewrite and revision
- llama340 - Specialized for instruction following, code, writing, accessiblity
- llama344 - Specialized for CoT, reasoning, code, math
- llama345 - Specialized for CoT, reasoning, code, math
- llama346 - Specialized for conversational abilities
- llama349 - Specialized for problem solving
- llama350 - Specialized for structured output

## Merging Parameters
- **Merging Method**: TIES (TRIM, ELECT SIGN & MERGE)
- **Density**: 0.2 (controls parameter sparsity)
- **Weights**: Equal weighting (1.0 for each adapter)
- **Merge Date**: 2025-03-09

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained("kevin009/llama3-merged-adapters")
tokenizer = AutoTokenizer.from_pretrained("kevin009/llama3-merged-adapters")

# Example usage
prompt = "Write a short story about a robot learning to paint."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Model Performance
This merged model combines the capabilities of multiple specialized adapters, resulting in improved performance across a variety of tasks compared to individual adapters.

## Limitations
- The model inherits limitations from the base Llama 3.1 model
- May produce inconsistent outputs for certain edge cases
- As with all language models, can produce incorrect or misleading information

## License
This model is subject to the license of the original Llama 3.1 model.