ChatHercules-2.5-Mistral-7B-DPO

image/png

ChatHercules-2.5-Mistral-7B is a merge of the following models using LazyMergekit:

I then use DPO to fine-tune the product of the merge on 20% of argilla/distilabel-intel-orca-dpo-pairs

🧩 Configuration

slices:
  - sources:
      - model: Locutusque/Hercules-2.5-Mistral-7B
        layer_range: [0, 32]
      - model: openchat/openchat-3.5-0106
        layer_range: [0, 32]
merge_method: slerp
base_model: Locutusque/Hercules-2.5-Mistral-7B
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

πŸ’» Usage

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "hydra-project/ChatHercules-2.5-Mistral-7B"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

Evaluation results

image/png

image/png

Downloads last month
19
Safetensors
Model size
7.24B params
Tensor type
BF16
Β·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for Locutusque/ChatHercules-2.5-Mistral-7B-DPO

Merges
1 model
Quantizations
3 models

Dataset used to train Locutusque/ChatHercules-2.5-Mistral-7B-DPO

Spaces using Locutusque/ChatHercules-2.5-Mistral-7B-DPO 7