---
base_model:
- agentica-org/DeepScaleR-1.5B-Preview
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
- trl
- reasoning
- finetune
- edge-device
- research
license: apache-2.0
language:
- en
datasets:
- bespokelabs/Bespoke-Stratos-17k
- simplescaling/s1K
- cognitivecomputations/dolphin-r1
- openai/gsm8k
- PrimeIntellect/NuminaMath-QwQ-CoT-5M
library_name: transformers
---

![image](./image.jpeg)

# **mini-Cogito-R1**

## **Overview**
The **mini-Cogito-R1** is a lightweight, high-performance language model fine-tuned for **text generation**, **mathematical reasoning**, and **edge-device optimization**. Developed by **Daemontatox**, this model is based on the **unsloth/deepscaler-1.5b-preview** architecture and fine-tuned using the **Unsloth** framework and Huggingface's **TRL** library, achieving **2x faster training speeds** without compromising performance.

---

## **Key Features**
- **Efficient Training:** Leverages [Unsloth](https://github.com/unslothai/unsloth) for faster and more efficient fine-tuning.  
- **Optimized for Edge Devices:** Designed to run efficiently on resource-constrained devices, making it ideal for edge computing applications.  
- **Mathematical Reasoning:** Excels in tasks requiring logical and mathematical reasoning.  
- **Text Generation:** Capable of generating high-quality, coherent text for a variety of applications.  
- **Lightweight:** Despite its compact size (1.5B parameters), it delivers robust performance.  

---

## **Model Details**
- **Developed by:** Daemontatox  
- **Model Name:** mini-Cogito-R1  
- **License:** Apache-2.0  
- **Base Model:** unsloth/deepscaler-1.5b-preview  
- **Fine-Tuned From:** unsloth/deepscaler-1.5b-preview-unsloth-bnb-4bit  
- **Framework:** Unsloth + Huggingface TRL  
- **Language:** English  

---

## **Training Datasets**
The **mini-Cogito-R1** model was fine-tuned on a diverse set of high-quality datasets to enhance its reasoning, mathematical, and text-generation capabilities. These datasets include:

1. **PrimeIntellect/NuminaMath-QwQ-CoT-5M**  
   - A large-scale dataset focused on mathematical reasoning and chain-of-thought (CoT) problem-solving.  

2. **openai/gsm8k**  
   - A dataset of grade-school math problems designed to test mathematical reasoning and problem-solving skills.  

3. **cognitivecomputations/dolphin-r1**  
   - A dataset for instruction-following and reasoning tasks, enhancing the model's ability to understand and execute complex instructions.  

4. **simplescaling/s1K**  
   - A lightweight dataset for general-purpose text generation and reasoning tasks.  

5. **bespokelabs/Bespoke-Stratos-17k**  
   - A dataset tailored for edge-device optimization and efficient text generation.  

---

## **Use Cases**
- **Edge Computing:** Deploy on edge devices for real-time text generation and reasoning tasks.  
- **Educational Tools:** Assist in solving mathematical problems and logical reasoning exercises.  
- **Content Creation:** Generate high-quality text for blogs, articles, and creative writing.  
- **Research:** Explore efficient training techniques and lightweight model architectures.  

---

## **Performance**
The **mini-Cogito-R1** was fine-tuned **2x faster** using Unsloth's optimized training pipeline, making it a cost-effective solution for developers and researchers. It maintains high accuracy and efficiency, particularly in mathematical reasoning and text generation tasks.

---

## **How to Use**
You can load and use the model with Huggingface's `transformers` library:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Daemontatox/mini-Cogito-R1"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

inputs = tokenizer("Solve for x: 2x + 5 = 15", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))
```

---

## **Acknowledgments**
- **Unsloth Team:** For their groundbreaking work on efficient model training.  
- **Huggingface:** For providing the TRL library and ecosystem.  
- **Open Source Community:** For continuous support and contributions.  

---

## **License**
This model is licensed under the **Apache-2.0** license. For more details, see the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) file.

---

## **Connect with the Developer**
- **GitHub:** [Daemontatox](https://github.com/Daemontatox)  
- **Huggingface Model Hub:** [mini-Cogito-R1](https://huggingface.co/Daemontatox/mini-Cogito-R1)  

---

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)  

---

### **Dataset References**
- **NuminaMath-QwQ-CoT-5M:** [PrimeIntellect](https://huggingface.co/datasets/PrimeIntellect/NuminaMath-QwQ-CoT-5M)  
- **GSM8K:** [OpenAI](https://huggingface.co/datasets/openai/gsm8k)  
- **Dolphin-R1:** [Cognitive Computations](https://huggingface.co/datasets/cognitivecomputations/dolphin-r1)  
- **S1K:** [Simple Scaling](https://huggingface.co/datasets/simplescaling/s1K)  
- **Bespoke-Stratos-17k:** [Bespoke Labs](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k)  

---