metadata

base_model: unsloth/Phi-3.5-mini-instruct
language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl

This page is work in progress!

Overview

The Fhi-3.5-mini-instruct is a fine-tuned version of the unsloth/Phi-3.5-mini-instruct model, optimized for function-calling.

Usage

Here’s a basic example of how to use function calling with the Fhi-3.5-mini-instruct model:

def get_current_temperature(location: str) -> float:
    """
    Get the current temperature at a location.
    
    Args:
        location: The location to get the temperature for, in the format "City, Country"
    Returns:
        The current temperature at the specified location in the specified units, as a float.
    """
    return 22.  

# Create the messages list
messages = [
    {"role": "system", "content": "You are a helpful weather assistant."},
    {"role": "user", "content": "What's the current weather in London and New York? Please use Celsius."}
]

# Apply the chat template
prompt = tokenizer.apply_chat_template(
    messages, 
    tools=[get_current_temperature],  # Pass the custom tool
    add_generation_prompt=True, 
    tokenize=False
)

inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, use_cache=True, temperature=0.001, top_p=1, eos_token_id=[32007])
resu = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True)
print(resu)

The result will look like this:

[
    {'name': 'get_current_temperature', 'arguments': {'location': 'London, UK'}},
    {'name': 'get_current_temperature', 'arguments': {'location': 'New York, USA'}}
]

Testing and Benchmarking

This model is still undergoing testing and evaluation. Use it at your own risk until further validation is complete. Performance on benchmarks like MMLU and MMLU-Pro will be updated soon.

Benchmark	Fhi-3.5 Mini-Ins	Phi-3.5 Mini-Ins	Mistral-7B-Instruct-v0.3	Mistral-Nemo-12B-Ins-2407	Llama-3.1-8B-Ins	Gemma-2-9B-Ins	Gemini 1.5 Flash	GPT-4o-mini-2024-07-18 (Chat)
Multilingual MMLU	____	55.4	47.4	58.9	56.2	63.8	77.2	72.9
MMLU (5-shot)	__	69	60.3	67.2	68.1	71.3	78.7	77.2
MMLU-Pro (3-shot, CoT)	__	47.4	18	40.7	44	50.1	57.2	62.8

Credits

Will be updated soon