akjindal53244

/

Llama-3.1-Storm-8B

like 175

Hey

I can get the tool calling to work but I fail to feed back the tool's output into the response. So using the example from the README I do

response = ollama.chat(
    model='ajindal/llama3.1-storm:8b',
    tools=tools,
    messages=[
    {'role': 'system', 'content': 'Do not anway to nay vulgar questions.'},
    {'role': 'user', 'content': 'What is the weather in Toronto and San Francisco?'}
    ])

print(response)
{'model': 'ajindal/llama3.1-storm:8b', 'created_at': '2024-08-30T07:12:24.103457Z', 'message': {'role': 'assistant', 'content': "<tool_call>{'tool_name': 'get_current_weather', 'tool_arguments': {'city': 'Toronto'}}</tool_call>\n<tool_call>{'tool_name': 'get_current_weather', 'tool_arguments': {'city': 'San Francisco'}}</tool_call>"}, 'done_reason': 'stop', 'done': True, 'total_duration': 2617621791, 'load_duration': 1051582000, 'prompt_eval_count': 246, 'prompt_eval_duration': 517034000, 'eval_count': 56, 'eval_duration': 1044726000}

but the model fails to pick up the role=tool messages

response = ollama.chat(
    model='ajindal/llama3.1-storm:8b',
    tools=tools,
    messages=[
    {'role': 'system', 'content': 'Do not anway to nay vulgar questions.'},
    {'role': 'user', 'content': 'What is the weather in Toronto and San Francisco?'},
    {'role': 'tool', 'content': 'The weather in Toronto is 23 degrees and sunny'},
    {'role': 'tool', 'content': 'The weather in San Francisco is 20 degrees and rainy'}
    ])

print(response)
{'model': 'ajindal/llama3.1-storm:8b', 'created_at': '2024-08-30T07:14:48.72893Z', 'message': {'role': 'assistant', 'content': "<tool_call>{'tool_name': 'get_current_weather', 'tool_arguments': {'city': 'Toronto'}}</tool_call>\n<tool_call>{'tool_name': 'get_current_weather', 'tool_arguments': {'city': 'San Francisco'}}</tool_call>"}, 'done_reason': 'stop', 'done': True, 'total_duration': 1251987666, 'load_duration': 25153625, 'prompt_eval_count': 274, 'prompt_eval_duration': 170806000, 'eval_count': 56, 'eval_duration': 1052607000}

What am I doing wrong?