---
base_model:
- Qwen/Qwen2.5-Math-7B-Instruct
- nvidia/AceMath-7B-Instruct
- open-r1/OpenR1-Qwen-7B
library_name: transformers
tags:
- mergekit
- merge

---
![MITA Logo](logo.png)  

# Formulae/MITA-V1.2-7B-2-24-2025  

## Overview  
Formulae/MITA-V1.2-7B is designed primarily for **mathematics**, with some capability in **coding**. Built using the **Linear DARE merge method**, this model blends powerful mathematical reasoning with computational accuracy.  

| Rank | Type | Model                        | Average  | IFEval  | BBH     | MATH    | GPQA    | MUSR    | MMLU-PRO | CO₂ Cost |
|------|------|-----------------------------|----------|---------|---------|---------|---------|---------|---------|---------|
| 914  | 🤝   | formulae/mita-v1.1-7b-2-24-2025 | **29.48 %** | **34.12 %** | **35.44 %** | 43.50 % | **8.61 %** | **16.06 %** | **39.15 %** | 0.67 kg  |
| 1403 | 🤝   | formulae/mita-v1.2-7b-2-24-2025 | 24.86 %  | 25.64 %  | 28.41 %  | **48.79 %** | 7.49 %  | 12.63 %  | 26.21 %  | **0.64 kg** |


## Merge Details  

- **Base Model:** [Qwen/Qwen2.5-Math-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Math-7B-Instruct)  
- **Merged Models:**  
  - [nvidia/AceMath-7B-Instruct](https://huggingface.co/nvidia/AceMath-7B-Instruct)  
  - [open-r1/OpenR1-Qwen-7B](https://huggingface.co/open-r1/OpenR1-Qwen-7B)  
- **Merge Method:** [Linear DARE](https://arxiv.org/abs/2311.03099)  
- **Data Type:** bfloat16  
- **Merge Parameters:**  
  - Density & Weight: 0.5 for AceMath & OpenR1  
  - Normalization: Disabled  
  - Int8 Masking: Enabled  

## What is DARE?  
DARE (**Density-Aware Residual Estimation**) is an advanced model merging technique designed to **preserve task-specific knowledge**. Unlike simple model averaging, DARE adjusts parameter density to ensure that merged models **retain their specialized skills** while improving general performance.  

📖 **Reference:** [DARE Paper](https://arxiv.org/abs/2311.03099)  

This merge is also inspired by **task arithmetic**, which shows that models can be **linearly combined** to enhance capabilities in specialized domains.  

📖 **Reference:** [Task Arithmetic Paper](https://arxiv.org/abs/2212.04089)  

## Capabilities  
✅ **Advanced Mathematics** – Strong problem-solving, algebra, calculus, and theorem applications.  
✅ **Limited Coding Support** – Can handle basic programming tasks but is not optimized for complex software development.  

## Limitations & Risks  

⚠ **Hallucinations in Code** – Not a coding-specialized model, may produce incorrect or insecure implementations.  
⚠ **Arithmetic Errors** – While highly capable, the model still makes occasional miscalculations.  

## Usage Disclaimer  
Formulae/MITA-V1.2-7B is an **experimental mathematical model**. For verified accuracy, always cross-check results with reliable tools.  

## Contribute  
We welcome **contributions**, including quantizations, fine-tuning, and further enhancements.  

💡 **Support Us:** [Buy Me a Coffee](https://www.buymeacoffee.com/marcusidia)  

📩 **Contact:** formulaeresearch@gmail.com  

---  

## Future Development  
This is part of the **MITA series**. Future iterations will integrate **MoE (Mixture of Experts)** for even more specialized reasoning across multiple domains.  

**Made possible with [MergeKit](https://github.com/arcee-ai/mergekit).**