kdt3/gemma-merge

output-model

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the breadcrumbs_ties merge method using lemon07r/Gemma-2-Ataraxy-9B as a base.

Models Merged

The following models were included in the merge:

• rtzr/ko-gemma-2-9b-it + kdt3/1101_3_tab_v2.2.split.2_512
• wzhouad/gemma-2-9b-it-WPO-HB
• rtzr/ko-gemma-2-9b-it + ghost613/gemma9_on_korean_summary_events

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: lemon07r/Gemma-2-Ataraxy-9B
        layer_range: [0, 42]
        parameters:
            weight: 1
            density: 0.7
            gamma: 0.03
      - model: wzhouad/gemma-2-9b-it-WPO-HB
        layer_range: [0, 42]
        parameters:
            weight: 1
            density: 0.42
            gamma: 0.03
      - model: rtzr/ko-gemma-2-9b-it+kdt3/1101_3_tab_v2.2.split.2_512
        layer_range: [0, 42]
        parameters:
            weight: 1
            density: 0.42
            gamma: 0.03
      - model: rtzr/ko-gemma-2-9b-it+ghost613/gemma9_on_korean_summary_events # lora model loading
        layer_range: [0, 42]
        parameters:
            weight: 1
            density: 0.42
            gamma: 0.03
merge_method: breadcrumbs_ties
base_model: lemon07r/Gemma-2-Ataraxy-9B
dtype: bfloat16