README.md

---
license: apache-2.0
library_name: transformers
tags:
- mergekit
- merge
- llama-cpp
- gguf-my-repo
base_model: arcee-ai/SuperNova-Medius
model-index:
- name: SuperNova-Medius
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 55.6
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/SuperNova-Medius
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 49.3
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/SuperNova-Medius
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 32.48
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/SuperNova-Medius
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 17.9
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/SuperNova-Medius
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 19.19
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/SuperNova-Medius
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 48.83
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/SuperNova-Medius
      name: Open LLM Leaderboard
---

# Triangle104/SuperNova-Medius-Q6_K-GGUF
This model was converted to GGUF format from [`arcee-ai/SuperNova-Medius`](https://huggingface.co/arcee-ai/SuperNova-Medius) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/arcee-ai/SuperNova-Medius) for more details on the model.

---
Model details:
-
Arcee-SuperNova-Medius is a 14B parameter language model developed by Arcee.ai, built on the Qwen2.5-14B-Instruct architecture. This unique model is the result of a cross-architecture distillation pipeline, combining knowledge from both the Qwen2.5-72B-Instruct model and the Llama-3.1-405B-Instruct model. By leveraging the strengths of these two distinct architectures, SuperNova-Medius achieves high-quality instruction-following and complex reasoning capabilities in a mid-sized, resource-efficient form.

SuperNova-Medius is designed to excel in a variety of business use cases, including customer support, content creation, and technical assistance, while maintaining compatibility with smaller hardware configurations. It’s an ideal solution for organizations looking for advanced capabilities without the high resource requirements of larger models like our SuperNova-70B.

Distillation Overview
-
The development of SuperNova-Medius involved a sophisticated multi-teacher, cross-architecture distillation process, with the following key steps:

    Logit Distillation from Llama 3.1 405B:
        We distilled the logits of Llama 3.1 405B using an offline approach.
        The top K logits for each token were stored to capture most of the probability mass while managing storage requirements.

    Cross-Architecture Adaptation:
        Using mergekit-tokensurgeon, we created a version of Qwen2.5-14B that uses the vocabulary of Llama 3.1 405B.
        This allowed for the use of Llama 3.1 405B logits in training the Qwen-based model.

    Distillation to Qwen Architecture:
        The adapted Qwen2.5-14B model was trained using the stored 405B logits as the target.

    Parallel Qwen Distillation:
        In a separate process, Qwen2-72B was distilled into a 14B model.

    Final Fusion and Fine-Tuning:
        The Llama-distilled Qwen model's vocabulary was reverted to Qwen vocabulary.
        After re-aligning the vocabularies, a final fusion and fine-tuning step was conducted, using a specialized dataset from EvolKit to ensure that SuperNova-Medius maintained coherence, fluency, and context understanding across a broad range of tasks.

Performance Evaluation
-
Below are the benchmark results of SuperNova-Medius compared to similar models in its class:
Model 	Average 	IFEval 	BBH 	GPQA 	MMLU Pro 	MuSR 	Math Level 5
Mistral-Small 2409 	0.423 	0.628 	0.581 	0.333 	0.410 	0.406 	0.181
Supernova-Lite 	0.427 	0.786 	0.511 	0.306 	0.388 	0.415 	0.155
Qwen2.5-14B-Instruct 	0.450 	0.827 	0.623 	0.358 	0.490 	0.403 	0.000
Supernova-Medius 	0.480 	0.832 	0.631 	0.359 	0.502 	0.402 	0.152

SuperNova-Medius performs exceptionally well in instruction-following (IFEval) and complex reasoning tasks (BBH), demonstrating its capability to handle a variety of real-world scenarios. It outperforms Qwen2.5-14B and SuperNova-Lite in multiple benchmarks, making it a powerful yet efficient choice for high-quality generative AI applications.

Model Use Cases
-
Arcee-SuperNova-Medius is suitable for a range of applications, including:

    Customer Support: With its robust instruction-following and dialogue management capabilities, SuperNova-Medius can handle complex customer interactions, reducing the need for human intervention.
    Content Creation: The model’s advanced language understanding and generation abilities make it ideal for creating high-quality, coherent content across diverse domains.
    Technical Assistance: SuperNova-Medius has a deep reservoir of technical knowledge, making it an excellent assistant for programming, technical documentation, and other expert-level content creation.

Deployment Options
-
SuperNova-Medius is available for use under the Apache-2.0 license. For those who need even higher performance, the full-size 70B SuperNova model can be accessed via an Arcee-hosted API or for local deployment. To learn more or explore deployment options, please reach out to [email protected].
Technical Specifications

    Model Architecture: Qwen2.5-14B-Instruct
    Distillation Sources: Qwen2.5-72B-Instruct, Llama-3.1-405B-Instruct
    Parameter Count: 14 billion
    Training Dataset: Custom instruction dataset generated with EvolKit
    Distillation Technique: Multi-architecture offline logit distillation with cross-architecture vocabulary alignment.

Summary
-
Arcee-SuperNova-Medius provides a unique balance of power, efficiency, and versatility. By distilling knowledge from two top-performing teacher models into a single 14B parameter model, SuperNova-Medius achieves results that rival larger models while maintaining a compact size ideal for practical deployment. Whether for customer support, content creation, or technical assistance, SuperNova-Medius is the perfect choice for organizations looking to leverage advanced language model capabilities in a cost-effective and accessible form.
Open LLM Leaderboard Evaluation Results

Detailed results can be found here
Metric 	Value
Avg. 	37.22
IFEval (0-Shot) 	55.60
BBH (3-Shot) 	49.30
MATH Lvl 5 (4-Shot) 	32.48
GPQA (0-shot) 	17.90
MuSR (0-shot) 	19.19
MMLU-PRO (5-shot) 	48.83

---
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo Triangle104/SuperNova-Medius-Q6_K-GGUF --hf-file supernova-medius-q6_k.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Triangle104/SuperNova-Medius-Q6_K-GGUF --hf-file supernova-medius-q6_k.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Triangle104/SuperNova-Medius-Q6_K-GGUF --hf-file supernova-medius-q6_k.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Triangle104/SuperNova-Medius-Q6_K-GGUF --hf-file supernova-medius-q6_k.gguf -c 2048
```