Breezeblossom-v4-mistral-2x7B

Breezeblossom-v4-mistral-2x7B is a Mixure of Experts (MoE) made with the following models using LazyMergekit:

🧩 Configuration

base_model: MediaTek-Research/Breeze-7B-Instruct-v0_1
gate_mode: hidden
dtype: float16
experts:
  - source_model: MediaTek-Research/Breeze-7B-Instruct-v0_1
    positive_prompts: [ "<s>You are a helpful AI assistant built by MediaTek Research. The user you are helping speaks Traditional Chinese and comes from Taiwan.   [INST] 你好,請問你可以完成什麼任務? [/INST] "]
  - source_model: Azure99/blossom-v4-mistral-7b
    positive_prompts: ["A chat between a human and an artificial intelligence bot. The bot gives helpful, detailed, and polite answers to the human's questions. \n|Human|: hello\n|Bot|: "]

💻 Usage

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "sam-ezai/Breezeblossom-v4-mistral-2x7B"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Downloads last month
8
Safetensors
Model size
13.1B params
Tensor type
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for sam-ezai/Breezeblossom-v4-mistral-2x7B