Trelis/Qwen1.5-function-calling-chat-template

Qwen1.5 one shot chat template for function calling

This repo contains a tokenizer with a custom chat template in the tokenizer_config.json file.

The custom chat template can be used - via 'tokenizer.apply_chat_template' - to format an array of messages.

For example:

function_metadata = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "This function gets the current weather in a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city, e.g., San Francisco"
                    },
                    "format": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The temperature unit to use."
                    }
                },
                "required": ["city"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_clothes",
            "description": "This function provides a suggestion of clothes to wear based on the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "temperature": {
                        "type": "string",
                        "description": "The temperature, e.g., 15 C or 59 F"
                    },
                    "condition": {
                        "type": "string",
                        "description": "The weather condition, e.g., 'Cloudy', 'Sunny', 'Rainy'"
                    }
                },
                "required": ["temperature", "condition"]
            }
        }
    }    
]

# Comment out later messages to test various stages of generation.

sample_messages = [
    # System messages are not supported by default
    # {
    #     "role": "system",
    #     "content": "you are a helpful assistant"
    # },
    {
        "role": "function_metadata",
        "content": "FUNCTION_METADATA"
    },
    {
        "role": "user",
        "content": "What is the current weather in London?"
    },
    # {
    #     "role": "function_call",
    #     "content": "{\n    \"name\": \"get_current_weather\",\n    \"arguments\": {\n        \"city\": \"London\"\n    }\n}"
    # },
    # {
    #     "role": "function_response",
    #     "content": "{\n    \"temperature\": \"15 C\",\n    \"condition\": \"Cloudy\"\n}"
    # },
    # {
    #     "role": "assistant",
    #     "content": "The current weather in London is Cloudy with a temperature of 15 Celsius.<|end_of_turn|>"
    # },
    # {
    #     "role": "user",
    #     "content": "That's great. Now say hello."
    # },
    # {
    #     "role": "assistant",
    #     "content": "Hello!"
    # }
]

# Iterate through each message in the list
for message in sample_messages:
    if message['role'] == 'function_metadata':
        # Replace 'FUNCTION_METADATA' with 'function_metadata' in the content
        message['content'] = message['content'].replace('FUNCTION_METADATA', json.dumps(function_metadata, indent=4))

# View the template applied without tokenization
prompt = tokenizer.apply_chat_template(sample_messages, tokenize=False, add_generation_prompt=True)
print(prompt)

This will provide a prompt format for doing zero-shot function calling, for example using a TGI api.

Alternatively, when deploying a vLLM endpoint, this repo id may be passed as the tokenizer for a Qwen1.5 chat model, and the chat template will be applied. In this case, you simply need to prepare your array of messages as per above.