Update README.md

README.md

@@ -6,16 +6,16 @@ language:
 - en
 ---
 
-# Model Card for Breeze-7B-Instruct-v0.1
+# Model Card for Breeze-7B-Instruct-v0_1
 
 Breeze-7B is a language model family that builds on top of [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-v0.1), specifically intended for Traditional Chinese use.
 
-[Breeze-7B-Base](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0.1) is the base model for the Breeze-7B series. 
+[Breeze-7B-Base](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0_1) is the base model for the Breeze-7B series. 
 It is suitable for use if you have substantial fine-tuning data to tune it for your specific use case.
 
-[Breeze-7B-Instruct](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0.1) derives from the base model Breeze-7B-Base, making the resulting model amenable to be used as-is for commonly seen tasks.
+[Breeze-7B-Instruct](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0_1) derives from the base model Breeze-7B-Base, making the resulting model amenable to be used as-is for commonly seen tasks.
 
-[Breeze-7B-Instruct-64k](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0.1) is a slightly modified version of 
+[Breeze-7B-Instruct-64k](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0_1) is a slightly modified version of 
 Breeze-7B-Instruct to enable a 64k-token context length. Roughly speaking, that is equivalent to 88k Traditional Chinese characters.
 
 The current release version of Breeze-7B is v0.1.
@@ -38,34 +38,33 @@ Performance-wise:
 
 ## Features
 
-- Breeze-7B-Base-v0.1
+- Breeze-7B-Base-v0_1
   - Expanding the vocabulary dictionary size from 32k to 62k to better support Traditional Chinese
   - 8k-token context length
-- Breeze-7B-Instruct-v0.1
+- Breeze-7B-Instruct-v0_1
   - Expanding the vocabulary dictionary size from 32k to 62k to better support Traditional Chinese 
   - 8k-token context length
   - Multi-turn dialogue (without special handling for harmfulness)
-- Breeze-7B-Instruct-64k-v0.1
+- Breeze-7B-Instruct-64k-v0_1
   - Expanding the vocabulary dictionary size from 32k to 62k to better support Traditional Chinese
   - 64k-token context length
   - Multi-turn dialogue (without special handling for harmfulness)
 
 ## Model Details
 
-- Breeze-7B-Base-v0.1
+- Breeze-7B-Base-v0_1
   - Finetuned from: [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)
   - Model type: Causal decoder-only transformer language model
   - Language: English and Traditional Chinese (zh-tw)
-- Breeze-7B-Instruct-v0.1
-  - Finetuned from: [MediaTek-Research/Breeze-7B-Base-v0.1](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0.1)
+- Breeze-7B-Instruct-v0_1
+  - Finetuned from: [MediaTek-Research/Breeze-7B-Base-v0_1](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0_1)
   - Model type: Causal decoder-only transformer language model
   - Language: English and Traditional Chinese (zh-tw)
-- Breeze-7B-Instruct-64k-v0.1
-  - Finetuned from: [MediaTek-Research/Breeze-7B-Instruct-v0.1](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0.1)
+- Breeze-7B-Instruct-64k-v0_1
+  - Finetuned from: [MediaTek-Research/Breeze-7B-Instruct-v0_1](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0_1)
   - Model type: Causal decoder-only transformer language model
   - Language: English and Traditional Chinese (zh-tw)
 
-
 ## Base Model Performance
 
 **TMMLU+**, **DRCD**, and **Table** source from [MediaTek-Research/TCEval-v2](https://huggingface.co/datasets/MediaTek-Research/TCEval-v2).
@@ -82,7 +81,7 @@ Performance-wise:
 | [Qwen-14B](https://huggingface.co/01-ai/Qwen/Qwen-14B)| 14B    | 51.30        | 16.95 *     | 50.69  | 68.83      |
 | [Yi-6B](https://huggingface.co/01-ai/Yi-6B) | 6B     | 49.63        | 76.61       | 34.72  | 65.35      |
 | [Qwen-7B](https://huggingface.co/01-ai/Qwen/Qwen-7B)| 7B     | 42.84        | 0.0 *       | 39.58  | 61.00      |
-| [**Breeze-7B-Base-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0.1)       | 7B     | 40.35        | 81.13        | 28.47  | 61.63      |
+| [**Breeze-7B-Base-v0_1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0_1)       | 7B     | 40.35        | 81.13        | 28.47  | 61.63      |
 | [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)| 7B     | 36.93        | 79.27        | 27.78 | 64.89      |
 
 
@@ -99,15 +98,15 @@ Performance-wise:
  We use the code revised from [fastchat llm_judge](https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge) (GPT4 as judge) to evaluate **MT-Bench-tw** and **MT-Bench**.
 
 
-| Models                                                                                                  |        |↑ MT-Bench-tw (Score)| TMMLU+ (ACC)| TMMLU+ (ACC) | DRCD (EM)   | Table (ACC) | MT-Bench (Score) | MMLU (ACC)  | MMLU (ACC)  | 
+| Models                                                                                                  |        |↑ MT-Bench-tw (Score)| TMMLU+ (ACC) | TMMLU+ (ACC) | DRCD (EM)   | Table (ACC) | MT-Bench (Score) | MMLU (ACC)  | MMLU (ACC)  | 
 |---------------------------------------------------------------------------------------------------------|--------|--------------------|--------------|--------------|-------------|-------------|------------------|-------------|-------------|
 |                                                                                                         |        |TC, Chat            |TC, Knowledge |TC, Knowledge |TC, Reasoning|TC, Reasoning|EN, Chat          |EN, Knowledge|EN, Knowledge|
 |                                                                                                         |        |0 shot              | 0 shot       | 5 shot       | 3 shot      | 0 shot      |0 shot            |  0 shot     | 5 shot      | 
 | [gpt-3.5-turbo](https://openai.com)                                                                     |        |7.1                 | 41.76        |              |             | 40.27       |7.9               |  70.00      |             |    
 | [Yi-34B-Chat](https://huggingface.co/01-ai/Yi-34B-Chat)                                                 | 34B    |6.9                 | 54.87        |              |             | 36.81       |7.6               |   71.04     |             |    
 | [Qwen-14B-Chat](https://huggingface.co/Qwen/Qwen-14B-Chat)                                              | 14B    |6.4                 | 48.41        |              |             | 41.67       |7.2               |    64.91    |             |    
-| [**Breeze-7B-Instruct-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0.1)         | 7B     |5.7                 | 41.61        |              |             | 45.83       |7.1               |    63.26    |             |    
-| [**Breeze-7B-Instruct-64k-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0.1) | 7B     |5.5                 | 40.99        |              |             | 36.11       |7.1               |    63.68    |             |    
+| [**Breeze-7B-Instruct-v0_1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0_1)         | 7B     |5.7                 | 41.61        |              |             | 45.83       |7.1               |    63.26    |             |    
+| [**Breeze-7B-Instruct-64k-v0_1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0_1) | 7B     |5.5                 | 40.99        |              |             | 36.11       |7.1               |    63.68    |             |    
 | [Qwen-7B-Chat](https://huggingface.co/Qwen/Qwen-7B-Chat)                                                | 7B     |5.4                 | 40.02        |              |             | 33.33       |6.2               |    55.94    |             |    
 | [Yi-6B-Chat](https://huggingface.co/01-ai/Yi-6B-Chat)                                                   | 6B     |5.0                 | 44.79        |              |             | 25.69       |6.0               |    59.45    |             |    
 | [Taiwan-LLM-13B-v2.0-chat](https://huggingface.co/yentinglin/Taiwan-LLM-13B-v2.0-chat)                  | 13B    |5.0                 | 29.47        |              |             | 23.61       |-*                |    50.50    |             |     
@@ -121,8 +120,8 @@ Performance-wise:
 | gpt-3.5-turbo                                       |  7.8    |  6.1    |   5.1   |   6.4   |  6.2    |   8.7   |   7.4   |   9.3   |   7.1   |
 | Yi-34B-Chat                                         |  9.0    |  4.8    |   5.7   |   4.0   |  4.7    |   8.5   |   8.7   |   9.8   |   6.9   |
 | Qwen-14B-Chat                                       |  7.6    |  5.7    |   4.5   |   4.2   |  5.3    |   7.5   |   7.3   |   9.1   |   6.4   |
-| **Breeze-7B-Instruct-v0.1**                         |  6.5    |  5.6    |   3.9   |   3.6   |  4.3    |   6.9   |   5.7   |   9.3   |   5.7   |
-| **Breeze-7B-Instruct-64k-v0.1**                     |  6.1    |  5.3    |   3.7   |   2.9   |  4.2    |   7.0   |   6.7   |   8.3   |   5.5   |
+| **Breeze-7B-Instruct-v0_1**                         |  6.5    |  5.6    |   3.9   |   3.6   |  4.3    |   6.9   |   5.7   |   9.3   |   5.7   |
+| **Breeze-7B-Instruct-64k-v0_1**                     |  6.1    |  5.3    |   3.7   |   2.9   |  4.2    |   7.0   |   6.7   |   8.3   |   5.5   |
 | Qwen-7B-Chat                                        |  6.6    |  4.5    |   4.8   |   2.9   |  3.6    |   6.2   |   6.8   |   8.2   |   5.4   |
 | Yi-6B-Chat                                          |  7.3    |  2.7    |   3.1   |   3.3   |  2.3    |   7.2   |   5.2   |   8.8   |   5.0   |
 | Taiwan-LLM-13B-v2.0-chat                            |  6.1    |  3.4    |   4.1   |   2.3   |  3.1    |   7.4   |   6.6   |   6.8   |   5.0   |
@@ -150,8 +149,8 @@ All inferences run on 2 RTX A6000 GPUs (using `vllm`, with a tensor-parallel siz
 | Models                                                             | ↓ Inference Time (sec)|Estimated Max Input Length (Char)|
 |--------------------------------------------------------------------|-------------------|--------------------------|
 | Yi-6B-Chat                                                         |   10.62  |   5.2k                |
-| **Breeze-7B-Instruct-v0.1**                                        |  10.74  |    11.1k                 |
-| **Breeze-7B-Instruct-64k-v0.1**                                    | 10.74       |  88.8k            |
+| **Breeze-7B-Instruct-v0_1**                                        |  10.74  |    11.1k                 |
+| **Breeze-7B-Instruct-64k-v0_1**                                    | 10.74       |  88.8k            |
 | Qwen-7B-Chat                                                       |   10.86         |    9.8k                  |
 | Qwen-14B-Chat                                                      |   18.89  |    9.8k                  |
 | Mistral-7B-v0.1-Instruct                                           |  20.48   |    5.1k                 |
@@ -180,7 +179,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
 import torch
 
 model = AutoModelForCausalLM.from_pretrained(
-    "MediaTek-Research/Breeze-7B-Instruct-v0.1",
+    "MediaTek-Research/Breeze-7B-Instruct-v0_1",
     device_map="auto",
     torch_dtype=torch.bfloat16,
     attn_implementation="flash_attention_2" # optional