metadata

base_model:
  - TheDrummer/Cydonia-22B-v1.3
  - concedo/Beepo-22B
  - SvdH/RPLament-22B
library_name: transformers
tags:
  - mergekit
  - merge
license: other
license_name: mrl
license_link: https://mistral.ai/licenses/MRL-0.1.md

Vengeance

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the DARE TIES merge method using SvdH/RPLament-22B as a base.

Models Merged

The following models were included in the merge:

TheDrummer/Cydonia-22B-v1.3
concedo/Beepo-22B

Configuration

The following YAML configuration was used to produce this model:

merge_method: dare_ties
base_model: SvdH/RPLament-22B
parameters:
  int8_mask: true
dtype: bfloat16
models:
  - model: SvdH/RPLament-22B
    use_cache: true
    parameters:
      weight: 0.60
      density: 0.78
  - model: TheDrummer/Cydonia-22B-v1.3
    use_cache: true
    parameters:
      weight: 0.20
      density: 0.66
  - model: concedo/Beepo-22B
    use_cache: true
    parameters:
      weight: 0.20
      density: 0.54