MoonRide/Llama-3.2-3B-Khelavaster

Intro

Experimental merge of multiple Llama 3.2 3B models, guided by MoonRide-Index-v7. Created with mergekit.

Merge Details

Merge Method

This model was merged using the SCE merge method using meta-llama/Llama-3.2-3B as a base.

Models Merged

The following models were included in the merge:

• bunnycore/Llama-3.2-3B-Sci-Think
• FuseAI/FuseChat-Llama-3.2-3B-Instruct
• theprint/ReWiz-Llama-3.2-3B
• bunnycore/Llama-3.2-3B-Mix-Skill

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: bunnycore/Llama-3.2-3B-Mix-Skill
  - model: bunnycore/Llama-3.2-3B-Sci-Think
  - model: FuseAI/FuseChat-Llama-3.2-3B-Instruct
  - model: theprint/ReWiz-Llama-3.2-3B
base_model: meta-llama/Llama-3.2-3B
tokenizer:
  source: meta-llama/Llama-3.2-3B-Instruct
merge_method: sce
parameters:
  normalize: true
dtype: float32
out_dtype: float16