Update README.md

README.md

@@ -1,35 +1,53 @@
 ---
 license: other
 license_name: tencent-hunyuan-a13b
-license_link: LICENSE
+license_link: https://github.com/Tencent-Hunyuan/Hunyuan-A13B/blob/main/LICENSE
 ---
 
+<p align="left">
+    <a href="README_CN.md">中文</a>&nbsp ｜ English</a>
+</p>
+<br><br>
+
 
 
 <p align="center">
  <img src="https://dscache.tencent-cloud.cn/upload/uploader/hunyuan-64b418fd052c033b228e04bc77bbc4b54fd7f5bc.png" width="400"/> <br>
 </p><p></p>
 
+
+<p align="center">
+    🫣&nbsp;<a href="https://huggingface.co/tencent/Hunyuan-A13B-Instruct"><b>Hugging Face</b></a>&nbsp;&nbsp;|&nbsp;&nbsp;
+    🖥️&nbsp;<a href="https://llm.hunyuan.tencent.com/" style="color: red;"><b>Official Website</b></a>&nbsp;&nbsp;|&nbsp;&nbsp;
+    🕖&nbsp;<a href="https://cloud.tencent.com/product/hunyuan"><b>HunyuanAPI</b></a>&nbsp;&nbsp;|&nbsp;&nbsp;
+    🕹️&nbsp;<a href="https://hunyuan.tencent.com/?model=hunyuan-a13b"><b>Demo</b></a>&nbsp;&nbsp;|&nbsp;&nbsp;
+    <img src="https://avatars.githubusercontent.com/u/109945100?s=200&v=4" width="16"/>&nbsp;<a href="https://modelscope.cn/models/Tencent-Hunyuan/Hunyuan-A13B-Instruct"><b>ModelScope</b></a>
+</p>
+
 <p align="center">
-    &nbsp<a href="https://github.com/Tencent/Hunyuan-A13B"><b>GITHUB</b></a>&nbsp&nbsp
+    <a href="https://github.com/Tencent/Hunyuan-A13B"><b>GITHUB</b></a>
+</p>
+
 
   
-Welcome to the official repository of **Hunyuan-A13B**, an innovative and open-source large language model (LLM) built on a fine-grained Mixture-of-Experts (MoE) architecture. Designed for efficiency and scalability, Hunyuan-A13B delivers cutting-edge performance with minimal computational overhead, making it an ideal choice for advanced reasoning and general-purpose applications, especially in resource-constrained environments.
+Welcome to the official repository of **Hunyuan-A13B**, an innovative and open-source large language model (LLM) built on a fine-grained Mixture-of-Experts (MoE) architecture. Designed for efficiency and scalability, Hunyuan-A13B delivers cutting-edge performance with minimal computational overhead, making it an ideal choice for advanced reasoning and general-purpose applications, especially in resource-constrained environments.
+
+## Model Introduction
 
-## Key Features and Highlights
+With the rapid advancement of artificial intelligence technology, large language models (LLMs) have achieved remarkable progress in natural language processing, computer vision, and scientific tasks. However, as model scales continue to expand, optimizing resource consumption while maintaining high performance has become a critical challenge. To address this, we have explored Mixture of Experts (MoE) architectures. The newly introduced Hunyuan-A13B model features a total of 80 billion parameters with 13 billion active parameters. It not only delivers high-performance results but also achieves optimal resource efficiency, successfully balancing computational power and resource utilization.
 
-- **High Performance with Fewer Parameters**: With only 13B active parameters (out of a total of 80B), Hunyuan-A13B achieves competitive results compared to much larger models across diverse benchmark tasks.
-- **Robust Pre-Training and Optimization**: Trained on a massive 20TB high-quality dataset, the model benefits from structured supervised fine-tuning and reinforcement learning strategies to enhance its reasoning, language comprehension, and general knowledge capabilities. 
-- **Dual-Mode Chain-of-Thought (CoT) Framework**: This unique feature allows dynamic adjustment of reasoning depth, balancing computational efficiency with accuracy. It supports both concise responses for simple tasks and in-depth reasoning for complex challenges.
-- **Exceptional Long-Context Understanding**: Hunyuan-A13B natively supports a 256K context window, maintaining robust performance in long-text tasks. 
+### Key Features and Advantages
 
-- **Advanced Agent-Oriented Capabilities**: Tailored optimizations enable effective handling of complex decision-making, with leading performance on agent benchmarks such as BFCL-v3  and τ-Bench.
-- **Superior Inference Efficiency**: Architectural innovations, including Grouped Query Attention (GQA) and support for multiple quantization formats , result in exceptional inference speed.
+- **Compact yet Powerful**: With only 13 billion active parameters (out of a total of 80 billion), the model delivers competitive performance on a wide range of benchmark tasks, rivaling much larger models.
+- **Hybrid Inference Support**: Supports both fast and slow thinking modes, allowing users to flexibly choose according to their needs.
+- **Ultra-Long Context Understanding**: Natively supports a 256K context window, maintaining stable performance on long-text tasks.
+- **Enhanced Agent Capabilities**: Optimized for agent tasks, achieving leading results on benchmarks such as BFCL-v3 and τ-Bench.
+- **Efficient Inference**: Utilizes Grouped Query Attention (GQA) and supports multiple quantization formats, enabling highly efficient inference.
 
-## Why Choose Hunyuan-A13B?
+### Why Choose Hunyuan-A13B?
+
+As a powerful yet computationally efficient large model, Hunyuan-A13B is an ideal choice for researchers and developers seeking high performance under resource constraints. Whether for academic research, cost-effective AI solution development, or innovative application exploration, this model provides a robust foundation for advancement.
 
-Hunyuan-A13B stands out as a powerful, scalable, and computationally efficient LLM, perfectly suited for researchers and developers seeking high performance without the burden of excessive resource demands. Whether you're working on academic research, building cost-effective AI solutions, or exploring novel applications, Hunyuan-A13B provides a versatile foundation to build upon.
-    
 &nbsp;
 
 ## Related News
@@ -60,146 +78,120 @@ Note: The following benchmarks are evaluated by TRT-LLM-backend
 | MMMLU            | 76.89          | 79.28 *         | 83.83          | 86.70        | 84.68          |
 
 
-&nbsp;
 
+Hunyuan-A13B-Instruct has achieved highly competitive performance across multiple benchmarks, particularly in mathematics, science, agent domains, and more. We compared it with several powerful models, and the results are shown below.
 
-| Topic               | Bench                         | OpenAI-o1-1217 | DeepSeek R1 | Qwen3-A22B | Hunyuan-A13B-Instruct |
-|:-------------------:|:-----------------------------:|:-------------:|:------------:|:-----------:|:---------------------:|
-| **Mathematics**     | AIME 2024<br>AIME 2025<br>MATH | 74.3<br>79.2<br>96.4 | 79.8<br>70<br>94.9 | 85.7<br>81.5<br>94.0 | 87.3<br>76.8<br>94.3 |
-| **Science**         | GPQA-Diamond<br>OlympiadBench | 78<br>83.1 | 71.5<br>82.4 | 71.1<br>85.7 | 71.2<br>82.7 |
-| **Coding**          | Livecodebench<br>Fullstackbench<br>ArtifactsBench | 63.9<br>64.6<br>38.6 | 65.9<br>71.6<br>44.6 | 70.7<br>65.6<br>44.6 | 63.9<br>67.8<br>43 |
-| **Reasoning**       | BBH<br>DROP<br>ZebraLogic    | 80.4<br>90.2<br>81 | 83.7<br>92.2<br>78.7 | 88.9<br>90.3<br>80.3 | 89.1<br>91.1<br>84.7 |
-| **Instruction<br>Following** | IF-Eval<br>SysBench  | 91.8<br>82.5 | 88.3<br>77.7 | 83.4<br>74.2 | 84.7<br>76.1 |
-| **Text<br>Creation**| LengthCtrl<br>InsCtrl       | 60.1<br>74.8 | 55.9<br>69 | 53.3<br>73.7 | 55.4<br>71.9 |
-| **NLU**             | ComplexNLU<br>Word-Task     | 64.7<br>67.1 | 64.5<br>81.8 | 59.8<br>56.4 | 61.2<br>62.9 |
-| **Agent**           | BDCL v3<br> $\tau$-bench<br>ComplexFuncBench<br> $C^3$-Bench | 67.8<br>60.4<br>47.6<br>58.8 | 63.8<br>58.7<br>n/a<br>55.3 | 70.8<br>46.7<br>n/a<br>51.7 | 78.3<br>54.7<br>51.2<br>63.5 |
-| **Average**         | -                            | n/a | n/a | n/a | n/a |
+| Topic               |                            Bench                             | OpenAI-o1-1217 | DeepSeek R1 | Qwen3-A22B | Hunyuan-A13B-Instruct |
+|:-------------------:|:------------------------------------------------------------:|:-------------:|:------------:|:-----------:|:---------------------:|
+| **Mathematics**     |                AIME 2024<br>AIME 2025<br>MATH                | 74.3<br>79.2<br>96.4 | 79.8<br>70<br>94.9 | 85.7<br>81.5<br>94.0 | 87.3<br>76.8<br>94.3 |
+| **Science**         |                GPQA-Diamond<br>OlympiadBench                 | 78<br>83.1 | 71.5<br>82.4 | 71.1<br>85.7 | 71.2<br>82.7 |
+| **Coding**          |      Livecodebench<br>Fullstackbench<br>ArtifactsBench       | 63.9<br>64.6<br>38.6 | 65.9<br>71.6<br>44.6 | 70.7<br>65.6<br>44.6 | 63.9<br>67.8<br>43 |
+| **Reasoning**       |                  BBH<br>DROP<br>ZebraLogic                   | 80.4<br>90.2<br>81 | 83.7<br>92.2<br>78.7 | 88.9<br>90.3<br>80.3 | 89.1<br>91.1<br>84.7 |
+| **Instruction<br>Following** |                     IF-Eval<br>SysBench                      | 91.8<br>82.5 | 88.3<br>77.7 | 83.4<br>74.2 | 84.7<br>76.1 |
+| **Text<br>Creation**|                    LengthCtrl<br>InsCtrl                     | 60.1<br>74.8 | 55.9<br>69 | 53.3<br>73.7 | 55.4<br>71.9 |
+| **NLU**             |                   ComplexNLU<br>Word-Task                    | 64.7<br>67.1 | 64.5<br>81.8 | 59.8<br>56.4 | 61.2<br>62.9 |
+| **Agent**           | BFCL v3<br> $\tau$-bench<br>ComplexFuncBench<br> $C^3$-Bench | 67.8<br>60.4<br>47.6<br>58.8 | 63.8<br>58.7<br>n/a<br>55.3 | 70.8<br>46.7<br>n/a<br>51.7 | 78.3<br>54.7<br>51.2<br>63.5 |
+| **Average**         |                              -                               | n/a | n/a | n/a | n/a |
 
 
+&nbsp;
 
-
-
-## Quick Start
-
-You can refer to the content in [Hunyuan-A13B](https://github.com/Tencent-Hunyuan/Hunyuan-A13B) to get started quickly. The training and inference code can use the version provided in this github repository.
-
-
-### Transformer
+## Use with transformers
+The following code snippet shows how to use the transformers library to load and apply the model. It also demonstrates how to enable and disable the reasoning mode , and how to parse the reasoning process along with the final output.
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 import os
+import re
+
+model_name_or_path = os.environ['MODEL_PATH']
+# model_name_or_path = "tencent/Hunyuan-A13B-Instruct"
 
+tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto",trust_remote_code=True)  # You may want to use bfloat16 and/or move to GPU here
+messages = [
+    {"role": "user", "content": "Write a short summary of the benefits of regular exercise"},
+]
+tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True,return_tensors="pt",
+                                                enable_thinking=True # Toggle thinking mode (default: True)
+                                                )
+                                                
+outputs = model.generate(tokenized_chat.to(model.device), max_new_tokens=4096)
 
-def main():
-    model_name_or_path = os.environ['MODEL_PATH']
-    
-
-    tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
-    model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto",
-                                                 trust_remote_code=True)  # You may want to use bfloat16 and/or move to GPU here
-    for name, param in model.named_parameters():
-        print(f"{name}: {param.size()}")
-    messages = [
-        {
-            "role": "system",
-            "content": "You are a helpful assistant.",
-        },
-        {"role": "user", "content": "Write a short summary of the benefits of regular exercise."}, 
-    ]
-    tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
-    outputs = model.generate(tokenized_chat.to(model.device), max_new_tokens=100,do_sample=True)            
-    print(tokenizer.decode(outputs[0]))
-
-if __name__ == '__main__':
-    main()
+output_text = tokenizer.decode(outputs[0])
 
+think_pattern = r'<think>(.*?)</think>'
+think_matches = re.findall(think_pattern, output_text, re.DOTALL)
+
+answer_pattern = r'<answer>(.*?)</answer>'
+answer_matches = re.findall(answer_pattern, output_text, re.DOTALL)
+
+think_content = [match.strip() for match in think_matches][0]
+answer_content = [match.strip() for match in answer_matches][0]
+print(f"thinking_content:{think_content}\n\n")
+print(f"answer_content:{answer_content}\n\n")
 ```
 
 
-## Deployment
+## Deployment   
 
-For deployment, you can use frameworks such as *vLLM*, *SGLang*, or *TensorRT-LLM* to serve the model and create an OpenAI-compatible API endpoint.
+For deployment, you can use frameworks such as **vLLM**, **SGLang**, or **TensorRT-LLM** to serve the model and create an OpenAI-compatible API endpoint.
 
 
 ### vllm
 
 #### Docker Image
-We provide a pre-built Docker image containing vLLM 0.8.5 with full support for this model. The official support is currently under development.
-
+We provide a pre-built Docker image containing vLLM 0.8.5 with full support for this model. The official vllm release is currently under development， **note: cuda 12.8 is require for this docker**.
 
 - To get started:
-```
-Pull the Docker image:docker pull xxx
-```
 
-- Start the API server:
+https://hub.docker.com/r/hunyuaninfer/hunyuan-large/tags 
 
 ```
-docker start xxx
-```
-
-
-#### Source Code
-
-Support for this model has been added via  this PR:  (https://github.com/vllm-project/vllm/pull/20114 )in the vLLM project. 
-You can build and run vLLM from source after merging this pull request into your local repository.
-
-After applying the changes, you can start the API server by following the standard vLLM setup instructions.
-
-
-### SGLlang
-
-#### Docker Image 
-
-We also provide a pre-built Docker image based on the latest version of SGLang.
-
-To get started:
-
-- Pull the Docker image
+docker pull hunyuaninfer/hunyuan-a13b:hunyuan-moe-A13B-vllm
 
 ```
-docker pull xxx
-```
+
+- Download Model file: 
+  - Huggingface:  will download automicly by vllm.
+  - ModelScope: `modelscope download --model Tencent-Hunyuan/Hunyuan-A13B-Instruct`
+ 
 
 - Start the API server:
 
+model download by huggingface:
 ```
-docker run --gpus all \
-    --shm-size 32g \
-    -p 30000:30000 \
-    --ipc=host \
-    xxx \
-    python3 -m sglang.launch_server --model-path hunyuan/huanyuan_A13B --tp 4 --trust-remote-code --host 0.0.0.0 --port 30000
-```
-
-
-#### Source Code
+docker run  --privileged --user root  --net=host --ipc=host \
+        -v ~/.cache:/root/.cache/ \
+        --gpus=all -it --entrypoint python  hunyuaninfer/hunyuan-a13b:hunyuan-moe-A13B-vllm
+ \
+         -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --port 8000 \
+         --tensor-parallel-size 4 --model tencent/Hunyuan-A13B-Instruct --trust-remote-code 
 
-The necessary integration has already been merged into the main branch via this PR(https://github.com/sgl-project/sglang/pull/7549 ).
-Once you have cloned or updated your local SGLang repository, you can build and run the API server using the standard SGLang setup process.
-
-After applying the changes, you can start the API server by following the standard SGLang setup instructions.
+``` 
 
+model downloaded by modelscope:
 ```
-python3 -m sglang.launch_server --model-path hunyuan/huanyuan_A13B --tp 4 --trust-remote-code --host 0.0.0.0 --port 30000
+docker run  --privileged --user root  --net=host --ipc=host \
+        -v ~/.cache/modelscope:/root/.cache/modelscope \
+        --gpus=all -it --entrypoint python   hunyuaninfer/hunyuan-a13b:hunyuan-moe-A13B-vllm \
+         -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --tensor-parallel-size 4 --port 8000 \ 
+         --model /root/.cache/modelscope/hub/models/Tencent-Hunyuan/Hunyuan-A13B-Instruct/ --trust_remote_code  
 ```
 
 
-
-###  TensorRT-LLM
-
+### SGLang
 
 #### Docker Image 
 
-We also provide a pre-built Docker image based on the latest version of TensorRT-LLM.
+We also provide a pre-built Docker image based on the latest version of SGLang.
 
 To get started:
 
 - Pull the Docker image
 
 ```
-docker pull xxx
+docker pull tiacc-test.tencentcloudcr.com/tiacc/sglang:0.4.7
 ```
 
 - Start the API server:
@@ -209,48 +201,10 @@ docker run --gpus all \
     --shm-size 32g \
     -p 30000:30000 \
     --ipc=host \
-    xxx \
-    python3 -m sglang.launch_server --model-path hunyuan/huanyuan_A13B --tp 4 --trust-remote-code --host 0.0.0.0 --port 30000
+    tiacc-test.tencentcloudcr.com/tiacc/sglang:0.4.7 \
+    -m sglang.launch_server --model-path hunyuan/huanyuan_A13B --tp 4 --trust-remote-code --host 0.0.0.0 --port 30000
 ```
 
-#### Source Code
-
-The necessary integration has already been merged into the main branch via this PR(xxx ).
-Once you have cloned or updated your local TensorRT-LLM. repository, you can build and run the API server using the standard TensorRT-LLM. setup process.
-
-After applying the changes, you can start the API server by following the standard TensorRT-LLM. setup instructions.
-
-
-
-## Inference Performance
-
-This section presents the efficiency test results of deploying various models using vLLM, including inference speed (tokens/s) under different batch sizes.
-    
-    
-Evaluation Script:     
-```python
-python3 benchmark_throughput.py --backend vllm \
-         --input-len 2048 \
-         --output-len 14336 \
-         --model $MODEL_PATH \
-         --tensor-parallel-size $TP \
-         --use-v2-block-manager \
-         --async-engine \
-         --trust-remote-code \
-         --num_prompts $BATCH_SIZE \
-         --max-num-seqs $BATCH_SIZE
-```
-
-| Inference Framework | Model      | Number of GPUs (GPU productA) | input_length | batch=1             | batch=16              | batch=32       |
-|------|-----------------------------|-----------|-------------------------|---------------------|----------------------|----------------------|
-| vLLM | Hunyuan-A13B-Instruct                   |    8     | 2048                  |      190.84     |       1246.54      |       1981.99     |
-| vLLM | Hunyuan-A13B-Instruct                   |    4     | 2048                  |     158.90      |       779.10       |    1301.75        |
-| vLLM | Hunyuan-A13B-Instruct                   |    2     | 2048                  |    111.72       |      327.31        |    346.54         |
-| vLLM | Hunyuan-A13B-Instruct(int8 weight only) |    2      | 2048                  |   109.10       |      444.17        |     721.93        |
-| vLLM | Hunyuan-A13B-Instruct(W8A8C8-FP8)       |    2      | 2048                  |    91.83       |      372.01        |      617.70       |
-| vLLM | Hunyuan-A13B-Instruct(W8A8C8-FP8)       |    1      | 2048                  |     60.07      |         148.80     |      160.41       |
-
-
 
 ## Contact Us