File size: 1,962 Bytes
1d3b826 a54e2bf 0fa6f49 a54e2bf 1d3b826 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
tags:
- merge
- mergekit
- lazymergekit
- KoboldAI/LLaMA2-13B-Tiefighter
- DavidAU/D_AU-Tiefighter-Giraffe-13B-32k-slerp
base_model:
- KoboldAI/LLaMA2-13B-Tiefighter
- DavidAU/D_AU-Tiefighter-Giraffe-13B-32k-slerp
---
Version 2:
Attempt to use linear "retraining" to fix issues with orginal model (D_AU-Tiefighter-Giraffe-13B-32k-slerp)
merge from step 1.
Seems to be successful.
Model is working correctly and GGUFs are also working correctly with context at 32768.
More testing required to see if context upgrade holds.
Imatrix Plus GGUFs upload to follow shortly.
# D_AU-Tiefighter-Plus-Giraffe-13B-32k-slerp
D_AU-Tiefighter-Plus-Giraffe-13B-32k-slerp is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [KoboldAI/LLaMA2-13B-Tiefighter](https://huggingface.co/KoboldAI/LLaMA2-13B-Tiefighter)
* [DavidAU/D_AU-Tiefighter-Giraffe-13B-32k-slerp](https://huggingface.co/DavidAU/D_AU-Tiefighter-Giraffe-13B-32k-slerp)
## 🧩 Configuration
```yaml
models:
- model: KoboldAI/LLaMA2-13B-Tiefighter
parameters:
weight: 0.8
- model: DavidAU/D_AU-Tiefighter-Giraffe-13B-32k-slerp
parameters:
weight: 0.2
merge_method: linear
dtype: bfloat16
```
## 💻 Usage
```python
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "DavidAU/D_AU-Tiefighter-Plus-Giraffe-13B-32k-slerp"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
``` |