File size: 2,044 Bytes
6104480 9aa863b 6104480 3a48e91 ec1c470 3a48e91 2317517 3a48e91 3c54d1e 3a48e91 2317517 e34e816 2317517 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
---
license: apache-2.0
datasets:
- open-thoughts/OpenThoughts-114k
- prithivMLmods/Deepthink-Reasoning-Ins
base_model:
- XeTute/SaplingDream_V0.5-0.5B
---
# Model Training Update
We are currently in the process of training our model, with an official release scheduled for **February 22, 2025 at 17:00 according to the timezone of the Islamic Republic of Pakistan**.
Introducing **SaplingDream**, a compact GPT model with 0.5 billion parameters, based on the [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) architecture. This model has been fine-tuned on reasoning datasets with meticulous attention to detail, ensuring the highest quality—hence the name "SaplingDream." See this as advanced "instruction" tuning for the base model to support reasoning to make up for its size efficiently.
To enhance generalization, we are fine-tuning the base model using Stochastic Gradient Descent (SGD) alongside a "Polynomial" learning rate scheduler, starting with a learning rate of 1e-4. Our goal is to ensure that the model not only learns the tokens but also develops the ability to reason through problems effectively.
For training, we are utilizing the [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k) and [prithivMLmods/Deepthink-Reasoning-Ins](https://huggingface.co/datasets/prithivMLmods/Deepthink-Reasoning-Ins) datasets across the entire epoch.
**Stay tuned! Until training is complete, we will be uploading every (last three) 200th checkpoint from a total of 14,275 optimization steps. Check out the `Files and versions` section for updates!**
---
# Our Apps & Socials
[Chat with our Assistant](https://xetute.com/) | [Support us Financially](https://ko-fi.com/XeTute) | [Visit our GitHub](https://github.com/XeTute)
Long live the Islamic Republic of Pakistan; Glory to the Islamic Republic of Pakistan 🇵🇰
 |