File size: 5,232 Bytes
18220c5 7170a98 18220c5 7170a98 18220c5 db600ce 18220c5 7d86cfe 18220c5 db600ce 18220c5 db600ce 18220c5 db600ce |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 |
---
base_model:
- agentica-org/DeepScaleR-1.5B-Preview
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
- trl
- reasoning
- finetune
- edge-device
- research
license: apache-2.0
language:
- en
datasets:
- bespokelabs/Bespoke-Stratos-17k
- simplescaling/s1K
- cognitivecomputations/dolphin-r1
- openai/gsm8k
- PrimeIntellect/NuminaMath-QwQ-CoT-5M
library_name: transformers
---
# **mini-Cogito-R1**
## **Overview**
The **mini-Cogito-R1** is a lightweight, high-performance language model fine-tuned for **text generation**, **mathematical reasoning**, and **edge-device optimization**. Developed by **Daemontatox**, this model is based on the **unsloth/deepscaler-1.5b-preview** architecture and fine-tuned using the **Unsloth** framework and Huggingface's **TRL** library, achieving **2x faster training speeds** without compromising performance.
---
## **Key Features**
- **Efficient Training:** Leverages [Unsloth](https://github.com/unslothai/unsloth) for faster and more efficient fine-tuning.
- **Optimized for Edge Devices:** Designed to run efficiently on resource-constrained devices, making it ideal for edge computing applications.
- **Mathematical Reasoning:** Excels in tasks requiring logical and mathematical reasoning.
- **Text Generation:** Capable of generating high-quality, coherent text for a variety of applications.
- **Lightweight:** Despite its compact size (1.5B parameters), it delivers robust performance.
---
## **Model Details**
- **Developed by:** Daemontatox
- **Model Name:** mini-Cogito-R1
- **License:** Apache-2.0
- **Base Model:** unsloth/deepscaler-1.5b-preview
- **Fine-Tuned From:** unsloth/deepscaler-1.5b-preview-unsloth-bnb-4bit
- **Framework:** Unsloth + Huggingface TRL
- **Language:** English
---
## **Training Datasets**
The **mini-Cogito-R1** model was fine-tuned on a diverse set of high-quality datasets to enhance its reasoning, mathematical, and text-generation capabilities. These datasets include:
1. **PrimeIntellect/NuminaMath-QwQ-CoT-5M**
- A large-scale dataset focused on mathematical reasoning and chain-of-thought (CoT) problem-solving.
2. **openai/gsm8k**
- A dataset of grade-school math problems designed to test mathematical reasoning and problem-solving skills.
3. **cognitivecomputations/dolphin-r1**
- A dataset for instruction-following and reasoning tasks, enhancing the model's ability to understand and execute complex instructions.
4. **simplescaling/s1K**
- A lightweight dataset for general-purpose text generation and reasoning tasks.
5. **bespokelabs/Bespoke-Stratos-17k**
- A dataset tailored for edge-device optimization and efficient text generation.
---
## **Use Cases**
- **Edge Computing:** Deploy on edge devices for real-time text generation and reasoning tasks.
- **Educational Tools:** Assist in solving mathematical problems and logical reasoning exercises.
- **Content Creation:** Generate high-quality text for blogs, articles, and creative writing.
- **Research:** Explore efficient training techniques and lightweight model architectures.
---
## **Performance**
The **mini-Cogito-R1** was fine-tuned **2x faster** using Unsloth's optimized training pipeline, making it a cost-effective solution for developers and researchers. It maintains high accuracy and efficiency, particularly in mathematical reasoning and text generation tasks.
---
## **How to Use**
You can load and use the model with Huggingface's `transformers` library:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Daemontatox/mini-Cogito-R1"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
inputs = tokenizer("Solve for x: 2x + 5 = 15", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))
```
---
## **Acknowledgments**
- **Unsloth Team:** For their groundbreaking work on efficient model training.
- **Huggingface:** For providing the TRL library and ecosystem.
- **Open Source Community:** For continuous support and contributions.
---
## **License**
This model is licensed under the **Apache-2.0** license. For more details, see the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) file.
---
## **Connect with the Developer**
- **GitHub:** [Daemontatox](https://github.com/Daemontatox)
- **Huggingface Model Hub:** [mini-Cogito-R1](https://huggingface.co/Daemontatox/mini-Cogito-R1)
---
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
---
### **Dataset References**
- **NuminaMath-QwQ-CoT-5M:** [PrimeIntellect](https://huggingface.co/datasets/PrimeIntellect/NuminaMath-QwQ-CoT-5M)
- **GSM8K:** [OpenAI](https://huggingface.co/datasets/openai/gsm8k)
- **Dolphin-R1:** [Cognitive Computations](https://huggingface.co/datasets/cognitivecomputations/dolphin-r1)
- **S1K:** [Simple Scaling](https://huggingface.co/datasets/simplescaling/s1K)
- **Bespoke-Stratos-17k:** [Bespoke Labs](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k)
--- |