Update docs/function_call_guide.md

docs/function_call_guide.md

@@ -8,9 +8,122 @@ The MiniMax-M1 model supports function calling capabilities, enabling the model
 
 ## 🚀 Quick Start
 
-### Using Chat Template
+### Using vLLM for Function Calls (Recommended)
 
-MiniMax-M1 uses a specific chat template format to handle function calls. The chat template is defined in `tokenizer_config.json`, and you can use it in your code through the template.
+In actual deployment, to support native Function Calling (tool calling) capabilities similar to OpenAI API, the MiniMax-M1 model integrates a dedicated `tool_call_parser=minimax` parser, avoiding additional regex parsing of model output.
+
+#### Environment Setup and vLLM Recompilation
+
+Since this feature has not been officially released in the PyPI version, compilation from source code is required. The following is an example process based on the official vLLM Docker image `vllm/vllm-openai:v0.8.3`:
+
+```bash
+IMAGE=vllm/vllm-openai:v0.8.3
+DOCKER_RUN_CMD="--network=host --privileged --ipc=host --ulimit memlock=-1 --shm-size=32gb --rm --gpus all --ulimit stack=67108864"
+
+# Run docker
+sudo docker run -it -v $MODEL_DIR:$MODEL_DIR \
+                    -v $CODE_DIR:$CODE_DIR \
+                    --name vllm_function_call \
+                    $DOCKER_RUN_CMD \
+                    --entrypoint /bin/bash \
+                    $IMAGE
+```
+
+#### Compiling vLLM Source Code
+
+After entering the container, execute the following commands to get the source code and reinstall:
+
+```bash
+cd $CODE_DIR
+git clone https://github.com/vllm-project/vllm.git
+cd vllm
+pip install -e .
+```
+
+#### Starting vLLM API Service
+
+```bash
+export SAFETENSORS_FAST_GPU=1
+export VLLM_USE_V1=0
+
+python3 -m vllm.entrypoints.openai.api_server \
+--model MiniMax-M1-80k \
+--tensor-parallel-size 8 \
+--trust-remote-code \
+--quantization experts_int8  \
+--enable-auto-tool-choice \
+--tool-call-parser minimax \
+--chat-template vllm/examples/tool_chat_template_minimax_m1.jinja \
+--max_model_len 4096 \
+--dtype bfloat16 \
+--gpu-memory-utilization 0.85
+```
+
+**⚠️ Note:**
+- `--tool-call-parser minimax` is a key parameter for enabling the MiniMax-M1 custom parser
+- `--enable-auto-tool-choice` enables automatic tool selection
+- `--chat-template` template file needs to be adapted for tool calling format
+
+#### Function Call Test Script Example
+
+The following Python script implements a weather query function call example based on OpenAI SDK:
+
+```python
+from openai import OpenAI
+import json
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")
+
+def get_weather(location: str, unit: str):
+    return f"Getting the weather for {location} in {unit}..."
+
+tool_functions = {"get_weather": get_weather}
+
+tools = [{
+    "type": "function",
+    "function": {
+        "name": "get_weather",
+        "description": "Get the current weather in a given location",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "location": {"type": "string", "description": "City and state, e.g., 'San Francisco, CA'"},
+                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
+            },
+            "required": ["location", "unit"]
+        }
+    }
+}]
+
+response = client.chat.completions.create(
+    model=client.models.list().data[0].id,
+    messages=[{"role": "user", "content": "What's the weather like in San Francisco? use celsius."}],
+    tools=tools,
+    tool_choice="auto"
+)
+
+print(response)
+
+tool_call = response.choices[0].message.tool_calls[0].function
+print(f"Function called: {tool_call.name}")
+print(f"Arguments: {tool_call.arguments}")
+print(f"Result: {get_weather(**json.loads(tool_call.arguments))}")
+```
+
+**Output Example:**
+```
+Function called: get_weather
+Arguments: {"location": "San Francisco, CA", "unit": "celsius"}
+Result: Getting the weather for San Francisco, CA in celsius...
+```
+
+### Manual Parsing of Model Output
+
+If you cannot use vLLM's built-in parser, or need to use other inference frameworks (such as transformers, TGI, etc.), you can use the following method to manually parse the model's raw output. This method requires you to parse the XML tag format of the model output yourself.
+
+#### Using Transformers Example
+
+The following is a complete example using the transformers library:
 
 ```python
 from transformers import AutoTokenizer
@@ -18,21 +131,19 @@ from transformers import AutoTokenizer
 def get_default_tools():
     return [
         {
-          {
-            "name": "get_current_weather",
-            "description": "Get the latest weather for a location",
-            "parameters": {
-                "type": "object", 
-                "properties": {
-                    "location": {
-                        "type": "string", 
-                        "description": "A certain city, such as Beijing, Shanghai"
-                    }
-                }, 
-            }
-            "required": ["location"],
-            "type": "object"
+          "name": "get_current_weather",
+          "description": "Get the latest weather for a location",
+          "parameters": {
+              "type": "object", 
+              "properties": {
+                  "location": {
+                      "type": "string", 
+                      "description": "A certain city, such as Beijing, Shanghai"
+                  }
+              }, 
           }
+          "required": ["location"],
+          "type": "object"
         }
     ]
 
@@ -54,6 +165,27 @@ text = tokenizer.apply_chat_template(
     add_generation_prompt=True,
     tools=tools
 )
+
+# Send request (using any inference service here)
+import requests
+payload = {
+    "model": "MiniMaxAI/MiniMax-M1-40k",
+    "prompt": text,
+    "max_tokens": 4000
+}
+response = requests.post(
+    "http://localhost:8000/v1/completions",
+    headers={"Content-Type": "application/json"},
+    json=payload,
+    stream=False,
+)
+
+# Model output needs manual parsing
+raw_output = response.json()["choices"][0]["text"]
+print("Raw output:", raw_output)
+
+# Use the parsing function below to process the output
+function_calls = parse_function_calls(raw_output)
 ```
 
 ## 🛠️ Function Call Definition
@@ -102,22 +234,21 @@ Function calls need to be defined in the `tools` field of the request body. Each
 When processed internally by the model, function definitions are converted to a special format and concatenated to the input text:
 
 ```
-]~!b[]~b]system ai_setting=MiniMax AI
-MiniMax AI is an AI assistant independently developed by MiniMax. [e~[
-]~b]system tool_setting=tools
+<begin_of_document><beginning_of_sentence>system ai_setting=MiniMax AI
+MiniMax AI是由上海稀宇科技有限公司（MiniMax）自主研发的AI助理。<end_of_sentence>
+<beginning_of_sentence>system tool_setting=tools
 You are provided with these tools:
 <tools>
-{"name": "search_web", "description": "Search function.", "parameters": {"properties": {"query_list": {"description": "Keywords for search, with list element count of 1.", "items": {"type": "string"}, "type": "array"}, "query_tag": {"description": "Classification of the query", "items": {"type": "string"}, "type": "array"}}, "required": ["query_list", "query_tag"], "type": "object"}}
+{"name": "search_web", "description": "搜索函数。", "parameters": {"properties": {"query_list": {"description": "进行搜索的关键词，列表元素个数为1。", "items": {"type": "string"}, "type": "array"}, "query_tag": {"description": "query的分类", "items": {"type": "string"}, "type": "array"}}, "required": ["query_list", "query_tag"], "type": "object"}}
 </tools>
-
 If you need to call tools, please respond with <tool_calls></tool_calls> XML tags, and provide tool-name and json-object of arguments, following the format below:
 <tool_calls>
 {"name": <tool-name>, "arguments": <args-json-object>}
 ...
-</tool_calls>[e~[
-]~b]user name=User
-When were the most recent launch events for OpenAI and Gemini?[e~[
-]~b]ai name=MiniMax AI
+</tool_calls><end_of_sentence>
+<beginning_of_sentence>user name=用户
+OpenAI 和 Gemini 的最近一次发布会都是什么时候?<end_of_sentence>
+<beginning_of_sentence>ai name=MiniMax AI
 ```
 
 ### Model Output Format
@@ -134,16 +265,15 @@ Okay, I will search for the OpenAI and Gemini latest release.
 </tool_calls>
 ```
 
-## 📥 Function Call Result Processing
+## 📥 Manual Parsing of Function Call Results
 
 ### Parsing Function Calls
 
-You can use the following code to parse function calls from the model output:
+When manual parsing is required, you need to parse the XML tag format of the model output:
 
 ```python
 import re
 import json
-
 def parse_function_calls(content: str):
     """
     Parse function calls from model output
@@ -193,23 +323,33 @@ def execute_function_call(function_name: str, arguments: dict):
         # Build function execution result
         return {
             "role": "tool", 
-            "name": function_name, 
-            "content": json.dumps({
-                "location": location, 
-                "temperature": "25", 
-                "unit": "celsius", 
-                "weather": "Sunny"
-            }, ensure_ascii=False)
-        }
+            "content": [
+              {
+                "name": function_name,
+                "type": "text",
+                "text": json.dumps({
+                    "location": location, 
+                    "temperature": "25", 
+                    "unit": "celsius", 
+                    "weather": "Sunny"
+                }, ensure_ascii=False)
+              }
+            ] 
+          }
     elif function_name == "search_web":
         query_list = arguments.get("query_list", [])
         query_tag = arguments.get("query_tag", [])
         # Simulate search results
         return {
             "role": "tool",
-            "name": function_name,
-            "content": f"Search keywords: {query_list}, Categories: {query_tag}\nSearch results: Relevant information found"
-        }
+            "content": [
+              {
+                "name": function_name,
+                "type": "text",
+                "text": f"Search keywords: {query_list}, Categories: {query_tag}\nSearch results: Relevant information found"
+              }
+            ]
+          }
     
     return None
 ```
@@ -220,51 +360,65 @@ After successfully parsing function calls, you should add the function execution
 
 #### Single Result
 
-If the model decides to call `search_web`, we suggest you to return the function result in the following format, with the `name` field set to the specific tool name.
+If the model calls the `search_web` function, you can refer to the following format to add execution results, with the `name` field being the specific function name.
 
 ```json
 {
-  "data": [
-     {
-       "role": "tool", 
-       "name": "search_web", 
-       "content": "search_result"
-     }
+  "role": "tool", 
+  "content": [
+    {
+      "name": "search_web",
+      "type": "text",
+      "text": "test_result"
+    }
   ]
 }
 ```
 
 Corresponding model input format:
 ```
-]~b]tool name=search_web
-search_result[e~[
+<beginning_of_sentence>tool name=tools
+tool name: search_web
+tool result: test_result
+<end_of_sentence>
 ```
 
+#### Multiple Results
 
-#### Multiple Result
-If the model decides to call `search_web` and `get_current_weather` at the same time, we suggest you to return the multiple function results in the following format, with the `name` field set to "tools", and use the `content` field to contain multiple results.
-
+If the model calls both `search_web` and `get_current_weather` functions simultaneously, you can refer to the following format to add execution results, with `content` containing multiple results.
 
 ```json
 {
-  "data": [
-     {
-       "role": "tool", 
-       "name": "tools", 
-       "content": "Tool name: search_web\nTool result: test_result1\n\nTool name: get_current_weather\nTool result: test_result2"
-     }
+  "role": "tool", 
+  "content": [
+    {
+      "name": "search_web",
+      "type": "text",
+      "text": "test_result1"
+    },
+    {
+      "name": "get_current_weather",
+      "type": "text",
+      "text": "test_result2"
+    }
   ]
 }
 ```
 
 Corresponding model input format:
 ```
-]~b]tool name=tools
-Tool name: search_web
-Tool result: test_result1
-
-Tool name: get_current_weather
-Tool result: test_result2[e~[
+<beginning_of_sentence>tool name=tools
+tool name: search_web
+tool result: test_result1
+tool name: get_current_weather
+tool result: test_result2<end_of_sentence>
 ```
 
-While we suggest following the above formats, as long as the model input is easy to understand, the specific values of `name` and `content` is entirely up to the caller.
+While we recommend following the above formats, as long as the input returned to the model is easy to understand, the specific content of `name` and `text` is entirely up to you.
+
+## 📚 References
+
+- [MiniMax-M1 Model Repository](https://github.com/MiniMaxAI/MiniMax-M1)
+- [vLLM Project Homepage](https://github.com/vllm-project/vllm)
+- [vLLM Function Calling PR](https://github.com/vllm-project/vllm/pull/20297)
+- [OpenAI Python SDK](https://github.com/openai/openai-python)