---
base_model:
- Qwen/Qwen2.5-7B-Instruct-1M
- Sakalti/SJT-7B-1M
- Triangle104/Q2.5-Instruct-1M_Harmony
- bunnycore/Qwen2.5-7B-RRP-1M
- huihui-ai/Qwen2.5-7B-Instruct-1M-abliterated
library_name: transformers
tags:
- mergekit
- merge

---
# merge

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [Qwen/Qwen2.5-7B-Instruct-1M](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-1M) as a base.

### Models Merged

The following models were included in the merge:
* [Sakalti/SJT-7B-1M](https://huggingface.co/Sakalti/SJT-7B-1M)
* [Triangle104/Q2.5-Instruct-1M_Harmony](https://huggingface.co/Triangle104/Q2.5-Instruct-1M_Harmony)
* [bunnycore/Qwen2.5-7B-RRP-1M](https://huggingface.co/bunnycore/Qwen2.5-7B-RRP-1M)
* [huihui-ai/Qwen2.5-7B-Instruct-1M-abliterated](https://huggingface.co/huihui-ai/Qwen2.5-7B-Instruct-1M-abliterated)

### Configuration

The following YAML configuration was used to produce this model:

```yaml

# Merge configuration for ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M using Model Stock

base_model: Qwen/Qwen2.5-7B-Instruct-1M
dtype: bfloat16
merge_method: model_stock
models:
  - model: Qwen/Qwen2.5-7B-Instruct-1M
  - model: Triangle104/Q2.5-Instruct-1M_Harmony
  - model: Sakalti/SJT-7B-1M
  - model: bunnycore/Qwen2.5-7B-RRP-1M
  - model: huihui-ai/Qwen2.5-7B-Instruct-1M-abliterated
tokenizer_source: Qwen/Qwen2.5-7B-Instruct-1M


```