Update README.md

README.md

@@ -16,7 +16,7 @@ tags:
 
 # Breeze-7B-FC-v1_0-GGUF
 
-- Original model: `[Breeze-7B-FC-v1_0](https://huggingface.co/MediaTek-Research/Breeze-7B-FC-v1_0)`
+- Original model: [Breeze-7B-FC-v1_0](https://huggingface.co/MediaTek-Research/Breeze-7B-FC-v1_0)
 
 A conversion of [Breeze-7B-FC-v1_0](https://huggingface.co/MediaTek-Research/Breeze-7B-FC-v1_0) into diffrent quantisation levels via [llama.cpp](https://github.com/ggerganov/llama.cpp).
 
@@ -111,3 +111,120 @@ for text in llm(tokenizer.apply_chat_template(chat, tokenize=False), stream=True
 # 4. 珍珠奶茶 (Bubble tea) - 珍珠奶茶是一種以紅茶為基底的飲品，加入珍珠（Q彈的小湯圓）和鮮奶。它起源於台灣，並迅速成為全球流行的飲料。珍珠奶茶在全台灣都有不少知名品牌，例如茶湯會、五桐號等。
 # 5. 臭豆腐 (Stinky tofu) - 臭豆腐是一種以發酵豆腐為原料製作的傳統小吃。它具有強烈的氣味，但味道獨特且深受台灣人喜愛。臭豆腐通常會搭配多種調味料和配料，例如辣椒醬、蒜泥、酸菜等。臭豆腐在全台灣都有不少知名店家，例如阿宗麵線、大勇街臭豆腐等。
 ```
+
+
+**Instruction following**
+
+```python
+from mtkresearch.llm.prompt import MRPromptV2
+
+sys_prompt = ('You are a helpful AI assistant built by MediaTek Research. '
+  'The user you are helping speaks Traditional Chinese and comes from Taiwan.')
+
+prompt_engine = MRPromptV2()
+
+conversations = [
+    {"role": "system", "content": sys_prompt},
+    {"role": "user", "content": "請問什麼是深度學習？"},
+]
+
+prompt = prompt_engine.get_prompt(conversations)
+
+
+output_str = _inference(prompt, llm, params)
+result = prompt_engine.parse_generated_str(output_str)
+
+print(result)
+# {'role': 'assistant',
+#  'content': '深度學習（Deep Learning）是一種機器學習方法，它模仿人類大腦的神經網路結構來
+#              處理複雜的數據和任務。在深度學習中，模型由多層人工神經元組成，每個神經元之間有
+#              權重連接，並通過非線性轉換進行計算。這些層與層之間的相互作用使模型能夠學習複雜
+#              的函數關係或模式，從而解決各種問題，如圖像識別、自然語言理解、語音辨識等。深度
+#              學習通常需要大量的數據和強大的計算能力，因此經常使用圖形處理器（GPU）或特殊的
+#              加速器來執行。'}
+```
+
+**Function Calling**
+
+```python
+import json
+
+from mtkresearch.llm.prompt import MRPromptV2
+
+functions = [
+    {
+      "name": "get_current_weather",
+      "description": "Get the current weather in a given location",
+      "parameters": {
+        "type": "object",
+        "properties": {
+          "location": {
+            "type": "string",
+            "description": "The city and state, e.g. San Francisco, CA"
+          },
+          "unit": {
+            "type": "string",
+            "enum": ["celsius", "fahrenheit"]
+          }
+        },
+        "required": ["location"]
+      }
+    }
+]
+
+def fake_get_current_weather(location, unit=None):
+    return {'temperature': 30}
+
+mapping = {
+    'get_current_weather': fake_get_current_weather
+}
+
+prompt_engine = MRPromptV2()
+
+# stage 1: query
+conversations = [
+    {"role": "user", "content": "請問台北目前溫度是攝氏幾度？"},
+]
+
+prompt = prompt_engine.get_prompt(conversations, functions=functions)
+
+output_str = _inference(prompt, llm, params)
+result = prompt_engine.parse_generated_str(output_str)
+
+print(result) 
+# {'role': 'assistant', 
+#  'tool_calls': [
+#    {'id': 'call_U9bYCBRAbF639uUqfwehwSbw', 'type': 'function', 
+#     'function': {'name': 'get_current_weather', 'arguments': '{"location": "台北, 台灣", "unit": "celsius"}'}}]}
+
+# stage 2: execute called functions
+conversations.append(result)
+
+tool_call = result['tool_calls'][0]
+func_name = tool_call['function']['name']
+func = mapping[func_name]
+arguments = json.loads(tool_call['function']['arguments'])
+called_result = func(**arguments)
+
+# stage 3: put executed results
+conversations.append(
+    {
+        'role': 'tool',
+        'tool_call_id': tool_call['id'],
+        'name': func_name,
+        'content': json.dumps(called_result)
+    }
+)
+
+prompt = prompt_engine.get_prompt(conversations, functions=functions)
+
+output_str2 = _inference(prompt, llm, params)
+result2 = prompt_engine.parse_generated_str(output_str2)
+print(result2)
+# {'role': 'assistant', 'content': '台北目前的溫度是攝氏30度'}
+```
+
+
+3. Example function calling via `llama.cpp` server:
+
+![Function calling example](https://ik.imagekit.io/project2062/hadaly/Screenshot%202024-10-01%20at%2018.10.19_lh-dgptFf.png?updatedAt=1727777532528)