---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
- Locutusque/Hercules-6.1-Llama-3.1-8B
- Sao10K/Llama-3.1-8B-Stheno-v3.4
base_model:
- Locutusque/Hercules-6.1-Llama-3.1-8B
---
README.md

# ZeroXClem/Stheno-Hercules-3.1-8B

ZeroXClem/Stheno-Hercules-3.1-8B is an advanced model merge, combining the strengths of two state-of-the-art models using the powerful [mergekit](https://github.com/cg123/mergekit) framework. This model is designed to maximize performance by blending different architecture layers and leveraging cutting-edge interpolation techniques, bringing together the best of both worlds: **Hercules** and **Stheno**.

## 🚀 Merged Models

This model merge incorporates the following:

- [**Locutusque/Hercules-6.1-Llama-3.1-8B**](https://huggingface.co/Locutusque/Hercules-6.1-Llama-3.1-8B): Known for its powerful attention mechanisms and deep neural layers, Hercules-6.1 serves as the base for this merge.
- [**Sao10K/Llama-3.1-8B-Stheno-v3.4**](https://huggingface.co/Sao10K/Llama-3.1-8B-Stheno-v3.4): Complementing Hercules, Stheno-v3.4 contributes its refined, balanced network architecture for added depth and flexibility.

## 🧩 Merge Configuration

The configuration below outlines how the models are merged using **spherical linear interpolation (SLERP)**, which allows for smooth transitions between the layers of both models, ensuring an optimal blend of their unique attributes:

```yaml
slices:
  - sources:
      - model: Locutusque/Hercules-6.1-Llama-3.1-8B
        layer_range: [0, 32]
      - model: Sao10K/Llama-3.1-8B-Stheno-v3.4
        layer_range: [0, 32]
merge_method: slerp
base_model: Locutusque/Hercules-6.1-Llama-3.1-8B
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]  # Controls the blending of self-attention layers
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]  # Adjusts the blending across the MLP layers
    - value: 0.5  # Global merge weight for layers not specified by filters
dtype: bfloat16  # Optimized for efficiency and performance

```

### Key Parameters

- **Self-Attention Filtering** (`self_attn`): Controls the extent of blending across self-attention layers, ranging from full to partial utilization from both models at various levels.
- **MLP Filtering** (`mlp`): Similar to self-attention, this filter applies to the Multi-Layer Perceptrons, fine-tuning the neural network’s layer balance.
- **Global Weight (`t.value`)**: A general interpolation factor for all layers not explicitly defined by the filters, set at 0.5 for an equal contribution from both models.
- **Data Type (`dtype`)**: Uses `bfloat16` to maintain computational efficiency while ensuring a high level of precision.

## 🎯 Use Case & Applications

**ZeroXClem/Stheno-Hercules-3.1-8B** is where **imagination meets intelligence**, a model built to seamlessly weave together the **art of roleplay** and the **precision of science**. With the raw power of Hercules fueling your creations and Stheno’s delicate balance guiding every interaction, this model thrives in:

- **Immersive storytelling and dynamic roleplaying**: Craft rich, believable characters and worlds with unparalleled depth, emotional nuance, and narrative flow.
- **Scientific exploration and discovery**: Unleash your mind’s full potential for complex problem-solving, hypothesis testing, and advanced AI-driven research.
- **Blending creativity and logic**: A harmonious fusion of heart and intellect, this model handles anything from playful creativity to rigorous scientific applications.

## 📜 License

This model is open-sourced under the **Apache-2.0 License**.

## 💡 Tags

- `merge`
- `mergekit`
- `lazymergekit`
- `Locutusque/Hercules-6.1-Llama-3.1-8B`
- `Sao10K/Llama-3.1-8B-Stheno-v3.4`