MrRobotoAI/ResplendentAI-ChattyMix-8b-64k

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Stock merge method using MrRobotoAI/10 as a base.

Models Merged

The following models were included in the merge:

• ResplendentAI/Kei_Llama3_8B
• ResplendentAI/Rawr_Llama3_8B
• ResplendentAI/Nymph_8B
• ResplendentAI/Aurora_l3_8B
• ResplendentAI/SOVL_Llama3_8B
• ResplendentAI/Aura_Uncensored_l3_8B

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: ResplendentAI/Nymph_8B
  - model: ResplendentAI/Rawr_Llama3_8B
  - model: ResplendentAI/SOVL_Llama3_8B
  - model: ResplendentAI/Kei_Llama3_8B
  - model: ResplendentAI/Aurora_l3_8B
  - model: ResplendentAI/Aura_Uncensored_l3_8B
merge_method: model_stock
base_model: MrRobotoAI/10
normalize: false
dtype: float16

Models Merged

The following models were included in the merge:

• MrRobotoAI/3 + ResplendentAI/Llama3_Aesir_Preview_LoRA_128
• MrRobotoAI/3 + ResplendentAI/Smarts_Llama3
• MrRobotoAI/3 + ResplendentAI/Theory_of_Mind_Llama3
• MrRobotoAI/3 + ResplendentAI/BlueMoon_Llama3
• MrRobotoAI/3 + ResplendentAI/Aura_Llama3
• MrRobotoAI/3 + ResplendentAI/Luna_Llama3
• MrRobotoAI/3 + ResplendentAI/NoWarning_Llama3

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: MrRobotoAI/3+ResplendentAI/Theory_of_Mind_Llama3
  - model: MrRobotoAI/3+ResplendentAI/Aura_Llama3
  - model: MrRobotoAI/3+ResplendentAI/Luna_Llama3
  - model: MrRobotoAI/3+ResplendentAI/NoWarning_Llama3
  - model: MrRobotoAI/3+ResplendentAI/BlueMoon_Llama3
  - model: MrRobotoAI/3+ResplendentAI/Smarts_Llama3
  - model: MrRobotoAI/3+ResplendentAI/Llama3_Aesir_Preview_LoRA_128
merge_method: model_stock
base_model: MrRobotoAI/10
normalize: false
dtype: float16

Models Merged

The following models were included in the merge:

• MrRobotoAI/ResplendentAI-ChattyMix-8b

Configuration

The following YAML configuration was used to produce this model:

merge_method: dare_linear
models:
  - model: MrRobotoAI/10
    parameters:
      weight:
        - filter: v_proj
          value: [0.8, 0.8, 0.75, 0.55, 0.35, 0.15, 0.35, 0.55, 0.75, 0.8, 0.8]
        - filter: o_proj
          value: [0.8, 0.8, 0.75, 0.55, 0.35, 0.15, 0.35, 0.55, 0.75, 0.8, 0.8]
        - filter: up_proj
          value: [0.8, 0.8, 0.75, 0.55, 0.35, 0.15, 0.35, 0.55, 0.75, 0.8, 0.8]
        - filter: gate_proj
          value: [0.8, 0.8, 0.75, 0.55, 0.35, 0.15, 0.35, 0.55, 0.75, 0.8, 0.8]
        - filter: down_proj
          value: [0.8, 0.8, 0.75, 0.55, 0.35, 0.15, 0.35, 0.55, 0.75, 0.8, 0.8]
        - value: 1
  - model: MrRobotoAI/ResplendentAI-ChattyMix-8b
    parameters:
      weight:
        - filter: v_proj
          value: [0.2, 0.2, 0.25, 0.45, 0.65, 0.85, 0.65, 0.45, 0.25, 0.2, 0.2]
        - filter: o_proj
          value: [0.2, 0.2, 0.25, 0.45, 0.65, 0.85, 0.65, 0.45, 0.25, 0.2, 0.2]
        - filter: up_proj
          value: [0.2, 0.2, 0.25, 0.45, 0.65, 0.85, 0.65, 0.45, 0.25, 0.2, 0.2]
        - filter: gate_proj
          value: [0.2, 0.2, 0.25, 0.45, 0.65, 0.85, 0.65, 0.45, 0.25, 0.2, 0.2]
        - filter: down_proj
          value: [0.2, 0.2, 0.25, 0.45, 0.65, 0.85, 0.65, 0.45, 0.25, 0.2, 0.2]
        - value: 0
base_model: MrRobotoAI/10
tokenizer_source: base
dtype: bfloat16