---
license: mit
language:
- en
base_model:
- microsoft/phi-4-gguf
pipeline_tag: text-generation
tags:
- phi4
- gguf-connector
---

# GGUF quantized and bug fixed version of **phi4**

### review
- bug fixed for: "ResponseError: llama runner process has terminated: GGML_ASSERT(hparams.n_swa > 0) failed"
- define the architecture (from none) to llama; all works right away

### run the model
use any gguf connector to interact with gguf file(s), i.e., [connector](https://pypi.org/project/gguf-connector/)

### reference
- base model: microsoft/[phi-4](https://huggingface.co/microsoft/phi-4)
- bug fixed following the guide written by [unsloth](https://unsloth.ai/blog/phi4)
- tool used for quantization: [cutter](https://pypi.org/project/gguf-cutter)

### citation
[Phi-4 Technical Report](https://arxiv.org/pdf/2412.08905)

### appendices: model summary and quality (written by microsoft)

#### model summary

|                         |                                                                               |     
|-------------------------|-------------------------------------------------------------------------------|
| **Developers**          | Microsoft Research                                                            |
| **Description**         | `phi-4` is a state-of-the-art open model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning.<br><br>`phi-4` underwent a rigorous enhancement and alignment process, incorporating both supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures                |
| **Architecture**        | 14B parameters, dense decoder-only Transformer model                          |
| **Inputs**              | Text, best suited for prompts in the chat format                              |
| **Context length**      | 16K tokens                                                                    |
| **GPUs**                | 1920 H100-80G                                                                 |
| **Training time**       | 21 days                                                                       |
| **Training data**       | 9.8T tokens                                                                   |
| **Outputs**             | Generated text in response to input                                           |
| **Dates**               | October 2024 – November 2024                                                  |
| **Status**              | Static model trained on an offline dataset with cutoff dates of June 2024 and earlier for publicly available data                                                                               |
| **Release date**        | December 12, 2024                                                             |
| **License**             | MIT                                                                         |

#### model quality

to understand the capabilities, we (here refer to microsoft side) compare `phi-4` with a set of models over OpenAI’s SimpleEval benchmark; at the high-level overview of the model quality on representative benchmarks; for the table below, higher numbers indicate better performance: 

| **Category**                 | **Benchmark** | **phi-4** (14B) | **phi-3** (14B) | **Qwen 2.5** (14B instruct) | **GPT-4o-mini** | **Llama-3.3** (70B instruct) | **Qwen 2.5** (72B instruct) | **GPT-4o** |
|------------------------------|---------------|-----------|-----------------|----------------------|----------------------|--------------------|-------------------|-----------------|
| Popular Aggregated Benchmark | MMLU          | 84.8      | 77.9            | 79.9                 | 81.8                 | 86.3               | 85.3              | **88.1**            |
| Science                      | GPQA          | **56.1**      | 31.2            | 42.9                 | 40.9                 | 49.1               | 49.0              | 50.6            |
| Math                         | MGSM<br>MATH  | 80.6<br>**80.4** | 53.5<br>44.6 | 79.6<br>75.6 | 86.5<br>73.0 | 89.1<br>66.3* | 87.3<br>80.0              | **90.4**<br>74.6            |
| Code Generation              | HumanEval     | 82.6      | 67.8            | 72.1                 | 86.2                 | 78.9*               | 80.4              | **90.6**            |
| Factual Knowledge            | SimpleQA      | 3.0       | 7.6            | 5.4                 | 9.9                  | 20.9               | 10.2              | **39.4**             |
| Reasoning                    | DROP          | 75.5      | 68.3            | 85.5                 | 79.3                 | **90.2**               | 76.7              | 80.9            |

\* these scores are lower than those reported by Meta, perhaps because simple-evals has a strict formatting requirement that Llama models have particular trouble following.