Update README.md

README.md

@@ -9,33 +9,51 @@ library_name: transformers
 tags:
 - mergekit
 - merge
-
+license: mit
 ---
-# merge
+# ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M
 
-This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+**ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M** is a custom merged language model based on **Qwen2.5-7B** with enhanced reasoning, roleplaying, and long-context capabilities. This model supports up to **1 million token** context lengths, making it ideal for ultra-long text processing, deep reasoning tasks, and immersive roleplay interactions.
 
-## Merge Details
-### Merge Method
+---
 
-This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [Qwen/Qwen2.5-7B-Instruct-1M](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-1M) as a base.
+## 🔧 **Model Details**
+- **Base Model**: `Qwen/Qwen2.5-7B-Instruct-1M`
+- **Models Used in Merge**:
+  - `Qwen/Qwen2.5-7B-Instruct-1M`
+  - `bunnycore/Qwen2.5-7B-RRP-1M`
+  - `Triangle104/Q2.5-Instruct-1M_Harmony`
+- **Merge Method**: `MODEL_STOCK` (Optimized layer-wise weight averaging)
 
-### Models Merged
+---
 
-The following models were included in the merge:
-* [Sakalti/SJT-7B-1M](https://huggingface.co/Sakalti/SJT-7B-1M)
-* [Triangle104/Q2.5-Instruct-1M_Harmony](https://huggingface.co/Triangle104/Q2.5-Instruct-1M_Harmony)
-* [bunnycore/Qwen2.5-7B-RRP-1M](https://huggingface.co/bunnycore/Qwen2.5-7B-RRP-1M)
-* [huihui-ai/Qwen2.5-7B-Instruct-1M-abliterated](https://huggingface.co/huihui-ai/Qwen2.5-7B-Instruct-1M-abliterated)
+## 📖 **Overview**
+**Qwen2.5-7B-CelestialHarmony-1M** enhances the **Qwen2.5-7B series** with a fine-tuned balance of roleplaying dynamics, structured reasoning, and long-context memory. The model is particularly well-suited for:
+- **Roleplaying** 🧝‍♂️: Immersive character-based storytelling with deep contextual awareness.
+- **Reasoning & Thought Processing** 🧠: Capable of structured logical thinking, especially when prompted with `<think>` tags.
+- **Ultra-Long Context Handling** 📜: Efficient processing of sequences up to **1,010,000 tokens** using optimized sparse attention.
 
-### Configuration
+---
 
-The following YAML configuration was used to produce this model:
+## ⚙️ **Technical Specifications**
+| Specification  | Value |
+|--------------|---------|
+| **Model Type** | Causal Language Model |
+| **Parameters** | 7.61B |
+| **Non-Embedding Parameters** | 6.53B |
+| **Layers** | 28 |
+| **Attention Heads (GQA)** | 28 (Q), 4 (KV) |
+| **Max Context Length** | 1,010,000 tokens |
+| **Max Generation Length** | 8,192 tokens |
+| **Merge Method** | Model Stock|
 
-```yaml
+---
 
-# Merge configuration for ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M using Model Stock
+## 🔬 **Merging Details**
+This model was merged using the **Model Stock** method, which optimally averages weights from multiple fine-tuned models to create a more efficient, balanced, and performant model.
 
+### **Merge YAML Configuration**
+```yaml
 base_model: Qwen/Qwen2.5-7B-Instruct-1M
 dtype: bfloat16
 merge_method: model_stock
@@ -46,6 +64,79 @@ models:
   - model: bunnycore/Qwen2.5-7B-RRP-1M
   - model: huihui-ai/Qwen2.5-7B-Instruct-1M-abliterated
 tokenizer_source: Qwen/Qwen2.5-7B-Instruct-1M
+```
 
+---
 
+## 🚀 **Quickstart**
+### **Install Required Packages**
+Ensure you have the latest `transformers` library installed:
+```bash
+pip install transformers torch accelerate
 ```
+
+### **Load and Use the Model**
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "Tell me a short story about an ancient celestial warrior."
+messages = [
+    {"role": "system", "content": "You are a wise celestial storyteller."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(**model_inputs, max_new_tokens=512)
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+
+print(response)
+```
+
+---
+
+## ⚡ **Optimized Deployment with vLLM**
+For long-context inference, use **vLLM**:
+```bash
+git clone -b dev/dual-chunk-attn [email protected]:QwenLM/vllm.git
+cd vllm
+pip install -e . -v
+```
+Run the model:
+```bash
+vllm serve ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M \
+  --tensor-parallel-size 4 \
+  --max-model-len 1010000 \
+  --enable-chunked-prefill --max-num-batched-tokens 131072 \
+  --enforce-eager \
+  --max-num-seqs 1
+```
+
+---
+
+## 🎯 **Model Capabilities**
+✅ **Roleplay & Storytelling** – Designed for engaging interactions.  
+✅ **Long-Context Awareness** – Handles texts up to **1M tokens**.  
+✅ **Logical Thinking & Reasoning** – Supports `<think>` tag to enhance thought structuring.  
+��� **Optimized Merge Strategy** – Uses `Model Stock` for superior generalization.  
+
+---
+
+## 📜 **Acknowledgments**
+This model is built on top of **Qwen2.5-7B**, with contributions from **bunnycore, Triangle104, and Sakalti**, leveraging the **Model Stock** merging methodology.
+
+For further details, see:
+- 📄 [Qwen2.5-7B Technical Report](https://arxiv.org/abs/2501.15383)
+- 📖 [MergeKit Documentation](https://github.com/mlfoundations/mergekit)
+- 🚀 [vLLM for Long-Context Inference](https://github.com/QwenLM/vllm)
+
+---