|
--- |
|
base_model: |
|
- agentica-org/DeepScaleR-1.5B-Preview |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- unsloth |
|
- qwen2 |
|
- trl |
|
- reasoning |
|
- finetune |
|
- edge-device |
|
- research |
|
license: apache-2.0 |
|
language: |
|
- en |
|
datasets: |
|
- bespokelabs/Bespoke-Stratos-17k |
|
- simplescaling/s1K |
|
- cognitivecomputations/dolphin-r1 |
|
- openai/gsm8k |
|
- PrimeIntellect/NuminaMath-QwQ-CoT-5M |
|
library_name: transformers |
|
--- |
|
|
|
 |
|
|
|
# **mini-Cogito-R1** |
|
|
|
## **Overview** |
|
The **mini-Cogito-R1** is a lightweight, high-performance language model fine-tuned for **text generation**, **mathematical reasoning**, and **edge-device optimization**. Developed by **Daemontatox**, this model is based on the **unsloth/deepscaler-1.5b-preview** architecture and fine-tuned using the **Unsloth** framework and Huggingface's **TRL** library, achieving **2x faster training speeds** without compromising performance. |
|
|
|
--- |
|
|
|
## **Key Features** |
|
- **Efficient Training:** Leverages [Unsloth](https://github.com/unslothai/unsloth) for faster and more efficient fine-tuning. |
|
- **Optimized for Edge Devices:** Designed to run efficiently on resource-constrained devices, making it ideal for edge computing applications. |
|
- **Mathematical Reasoning:** Excels in tasks requiring logical and mathematical reasoning. |
|
- **Text Generation:** Capable of generating high-quality, coherent text for a variety of applications. |
|
- **Lightweight:** Despite its compact size (1.5B parameters), it delivers robust performance. |
|
|
|
--- |
|
|
|
## **Model Details** |
|
- **Developed by:** Daemontatox |
|
- **Model Name:** mini-Cogito-R1 |
|
- **License:** Apache-2.0 |
|
- **Base Model:** unsloth/deepscaler-1.5b-preview |
|
- **Fine-Tuned From:** unsloth/deepscaler-1.5b-preview-unsloth-bnb-4bit |
|
- **Framework:** Unsloth + Huggingface TRL |
|
- **Language:** English |
|
|
|
--- |
|
|
|
## **Training Datasets** |
|
The **mini-Cogito-R1** model was fine-tuned on a diverse set of high-quality datasets to enhance its reasoning, mathematical, and text-generation capabilities. These datasets include: |
|
|
|
1. **PrimeIntellect/NuminaMath-QwQ-CoT-5M** |
|
- A large-scale dataset focused on mathematical reasoning and chain-of-thought (CoT) problem-solving. |
|
|
|
2. **openai/gsm8k** |
|
- A dataset of grade-school math problems designed to test mathematical reasoning and problem-solving skills. |
|
|
|
3. **cognitivecomputations/dolphin-r1** |
|
- A dataset for instruction-following and reasoning tasks, enhancing the model's ability to understand and execute complex instructions. |
|
|
|
4. **simplescaling/s1K** |
|
- A lightweight dataset for general-purpose text generation and reasoning tasks. |
|
|
|
5. **bespokelabs/Bespoke-Stratos-17k** |
|
- A dataset tailored for edge-device optimization and efficient text generation. |
|
|
|
--- |
|
|
|
## **Use Cases** |
|
- **Edge Computing:** Deploy on edge devices for real-time text generation and reasoning tasks. |
|
- **Educational Tools:** Assist in solving mathematical problems and logical reasoning exercises. |
|
- **Content Creation:** Generate high-quality text for blogs, articles, and creative writing. |
|
- **Research:** Explore efficient training techniques and lightweight model architectures. |
|
|
|
--- |
|
|
|
## **Performance** |
|
The **mini-Cogito-R1** was fine-tuned **2x faster** using Unsloth's optimized training pipeline, making it a cost-effective solution for developers and researchers. It maintains high accuracy and efficiency, particularly in mathematical reasoning and text generation tasks. |
|
|
|
--- |
|
|
|
## **How to Use** |
|
You can load and use the model with Huggingface's `transformers` library: |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
model_name = "Daemontatox/mini-Cogito-R1" |
|
model = AutoModelForCausalLM.from_pretrained(model_name) |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
|
inputs = tokenizer("Solve for x: 2x + 5 = 15", return_tensors="pt") |
|
outputs = model.generate(**inputs) |
|
print(tokenizer.decode(outputs[0])) |
|
``` |
|
|
|
--- |
|
|
|
## **Acknowledgments** |
|
- **Unsloth Team:** For their groundbreaking work on efficient model training. |
|
- **Huggingface:** For providing the TRL library and ecosystem. |
|
- **Open Source Community:** For continuous support and contributions. |
|
|
|
--- |
|
|
|
## **License** |
|
This model is licensed under the **Apache-2.0** license. For more details, see the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) file. |
|
|
|
--- |
|
|
|
## **Connect with the Developer** |
|
- **GitHub:** [Daemontatox](https://github.com/Daemontatox) |
|
- **Huggingface Model Hub:** [mini-Cogito-R1](https://huggingface.co/Daemontatox/mini-Cogito-R1) |
|
|
|
--- |
|
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth) |
|
|
|
--- |
|
|
|
### **Dataset References** |
|
- **NuminaMath-QwQ-CoT-5M:** [PrimeIntellect](https://huggingface.co/datasets/PrimeIntellect/NuminaMath-QwQ-CoT-5M) |
|
- **GSM8K:** [OpenAI](https://huggingface.co/datasets/openai/gsm8k) |
|
- **Dolphin-R1:** [Cognitive Computations](https://huggingface.co/datasets/cognitivecomputations/dolphin-r1) |
|
- **S1K:** [Simple Scaling](https://huggingface.co/datasets/simplescaling/s1K) |
|
- **Bespoke-Stratos-17k:** [Bespoke Labs](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k) |
|
|
|
--- |
