README.md

---
base_model: unsloth/Phi-3.5-mini-instruct
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
---
# This page is work in progress!

## Overview

The **Fhi-3.5-mini-instruct** is a fine-tuned version of the [unsloth/Phi-3.5-mini-instruct](https://huggingface.co/unsloth/Phi-3.5-mini-instruct) model, optimized for function-calling.

### Usage

Here’s a basic example of how to use function calling with the Fhi-3.5-mini-instruct model:

```python
def get_current_temperature(location: str) -> float:
    """
    Get the current temperature at a location.
    
    Args:
        location: The location to get the temperature for, in the format "City, Country"
    Returns:
        The current temperature at the specified location in the specified units, as a float.
    """
    return 22.  

# Create the messages list
messages = [
    {"role": "system", "content": "You are a helpful weather assistant."},
    {"role": "user", "content": "What's the current weather in London and New York? Please use Celsius."}
]

# Apply the chat template
prompt = tokenizer.apply_chat_template(
    messages, 
    tools=[get_current_temperature],  # Pass the custom tool
    add_generation_prompt=True, 
    tokenize=False
)

inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, use_cache=True, temperature=0.001, top_p=1, eos_token_id=[32007])
resu = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True)
print(resu)
```

The result will look like this:

```python
[
    {'name': 'get_current_temperature', 'arguments': {'location': 'London, UK'}},
    {'name': 'get_current_temperature', 'arguments': {'location': 'New York, USA'}}
]
```

## Testing and Benchmarking
This model is still undergoing testing and evaluation. Use it at your own risk until further validation is complete. Performance on benchmarks like MMLU and MMLU-Pro will be updated soon.

| Benchmark                  | Fhi-3.5 Mini-Ins | Phi-3.5 Mini-Ins | Mistral-7B-Instruct-v0.3 | Mistral-Nemo-12B-Ins-2407 | Llama-3.1-8B-Ins | Gemma-2-9B-Ins | Gemini 1.5 Flash | GPT-4o-mini-2024-07-18 (Chat) |
|----------------------------|------------------|------------------|--------------------------|---------------------------|------------------|----------------|------------------|-------------------------------|
| Multilingual MMLU          | ____             | 55.4             | 47.4                     | 58.9                      | 56.2             | 63.8           | 77.2             | 72.9                          |
| MMLU (5-shot) | __ | 69 | 60.3 | 67.2 | 68.1 | 71.3 | 78.7 | 77.2 |
| MMLU-Pro (3-shot, CoT) |__| 47.4 | 18 | 40.7 | 44 | 50.1 | 57.2 | 62.8 |
<!-- | Multilingual MMLU-Pro      | 30.9             | 30.21                 | 15.0                     | 34.0                      | 21.4             | 43.0           | 57.9             | 53.2                          |
| **Average** | **55.2** | **52.3** | **47.9** | **55.3** | **47.5** | **59.6** | **64.3** | **76.6** |
 -->
<!-- The table below shows Multilingual MMLU scores in some of the supported languages.

| Benchmark | Phi-3.5 Mini-Ins | Phi-3.0-Mini-128k-Instruct (June2024) | Mistral-7B-Instruct-v0.3 | Mistral-Nemo-12B-Ins-2407 | Llama-3.1-8B-Ins | Gemma-2-9B-Ins | Gemini 1.5 Flash | GPT-4o-mini-2024-07-18 (Chat) |
|-----------|------------------|-----------------------|--------------------------|---------------------------|------------------|----------------|------------------|-------------------------------|
| Arabic    | 44.2             | 35.4                  | 33.7                     | 45.3                      | 49.1             | 56.3           | 73.6             | 67.1                          |
| Chinese   | 52.6             | 46.9                  | 45.9                     | 58.2                      | 54.4             | 62.7           | 66.7             | 70.8                          |
| Dutch     | 57.7             | 48.0                  | 51.3                     | 60.1                      | 55.9             | 66.7           | 80.6             | 74.2                          |
| French    | 61.1             | 61.7                  | 53.0                     | 63.8                      | 62.8             | 67.0           | 82.9             | 75.6                          |
| German    | 62.4             | 61.3                  | 50.1                     | 64.5                      | 59.9             | 65.7           | 79.5             | 74.3                          |
| Italian   | 62.8             | 63.1                  | 52.5                     | 64.1                      | 55.9             | 65.7           | 82.6             | 75.9                          |
| Russian   | 50.4             | 45.3                  | 48.9                     | 59.0                      | 57.4             | 63.2           | 78.7             | 72.6                          |
| Spanish   | 62.6             | 61.3                  | 53.9                     | 64.3                      | 62.6             | 66.0           | 80.0             | 75.5                          |
| Ukrainian | 45.2             | 36.7                  | 46.9                     | 56.6                      | 52.9             | 62.0           | 77.4             | 72.6                          |
 -->

## Credits
Will be updated soon