CrystalMistral-26b

CrystalMistral-26b is a Mixure of Experts (MoE) made with the following models using LazyMergekit:

🧩 Configuration

base_model: Crystalcareai/CrystalMistral-13b
gate_mode: random
dtype: bfloat16 # output dtype (float32, float16, or bfloat16)
experts_per_token: 2
experts:
  - source_model: Crystalcareai/CrystalMistral
    positive_prompts:
      - You are an helpful general-purpose assistant"
  - source_model: Crystalcareai/CrystalMistral
    positive_prompts:
      - "You are an expert in providing detailed technical explanations."
  - source_model: Crystalcareai/CrystalMistral
    positive_prompts:
      - "You are an expert in providing detailed code."
  - source_model: Crystalcareai/CrystalMistral
    positive_prompts:
      - "You are an expert in providing emotionally emotional support."
      

πŸ’» Usage

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "Crystalcareai/CrystalMistral-26b"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Downloads last month
16
Safetensors
Model size
24.2B params
Tensor type
FP16
Β·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for Crystalcareai/CrystalMistral-26b

Finetuned
(1)
this model
Adapters
1 model
Quantizations
2 models