tainc commited on
Commit
207e356
1 Parent(s): d8b9211

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -10
README.md CHANGED
@@ -7,11 +7,11 @@ language:
7
  - vi
8
  license: gemma
9
  ---
10
- # Gemma2 9B CPT SEA-LIONv3.0 Instruct
11
 
12
  SEA-LION is a collection of Large Language Models (LLMs) which has been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
13
 
14
- Gemma2 9B CPT SEA-LIONv3.0 Instruct is a multilingual model which has been fine-tuned with around **500,000 English instruction-completion pairs** alongside a larger pool of around **1,000,000 instruction-completion pairs** from other ASEAN languages, such as Indonesian, Thai and Vietnamese.
15
 
16
  SEA-LION stands for _Southeast Asian Languages In One Network_.
17
 
@@ -24,12 +24,12 @@ SEA-LION stands for _Southeast Asian Languages In One Network_.
24
  ## Model Details
25
 
26
  ### Model Description
27
- We performed instruction tuning in English and also in ASEAN languages such as Indonesian, Thai and Vietnamese on our [continued pre-trained Gemma2 9B CPT SEA-LIONv3](https://huggingface.co/aisingapore/gemma2-9b-cpt-sea-lionv3-base), a decoder model using the Gemma2 architecture, to create Gemma2 9B CPT SEA-LIONv3.0 Instruct.
28
 
29
  The model has a context length of 8192.
30
 
31
  ### Benchmark Performance
32
- We evaluated Gemma2 9B CPT SEA-LIONv3.0 Instruct on both general language capabilities and instruction-following capabilities.
33
 
34
  #### General Language Capabilities
35
  For the evaluation of general language capabilities, we employed the [BHASA evaluation benchmark](https://arxiv.org/abs/2309.06085v2) across a variety of tasks.
@@ -41,7 +41,7 @@ The evaluation was done zero-shot with native prompts and only a sample of 100-1
41
 
42
 
43
  #### Instruction-following Capabilities
44
- Since Gemma2 9B CPT SEA-LIONv3.0 Instruct is an instruction-following model, we also evaluated it on instruction-following capabilities with two datasets, [IFEval](https://arxiv.org/abs/2311.07911) and [MT-Bench](https://arxiv.org/abs/2306.05685).
45
 
46
  As these two datasets were originally in English, the linguists and native speakers in the team worked together to filter, localize and translate the datasets into the respective target languages to ensure that the examples remained reasonable, meaningful and natural.
47
 
@@ -55,18 +55,18 @@ IFEval evaluates a model's ability to adhere to constraints provided in the prom
55
  MT-Bench evaluates a model's ability to engage in multi-turn (2 turns) conversations and respond in ways that align with human needs. We use `gpt-4-1106-preview` as the judge model and compare against `gpt-3.5-turbo-0125` as the baseline model. The metric used is the weighted win rate against the baseline model (i.e. average win rate across each category (Math, Reasoning, STEM, Humanities, Roleplay, Writing, Extraction)). A tie is given a score of 0.5.
56
 
57
 
58
- For more details on Gemma2 9B CPT SEA-LIONv3.0 Instruct benchmark performance, please refer to the SEA HELM leaderboard, https://leaderboard.sea-lion.ai/
59
 
60
 
61
  ### Usage
62
- Gemma2 9B CPT SEA-LIONv3.0 Instruct can be run using the 🤗 Transformers library
63
  ```python
64
  # Please use transformers==4.45.2
65
 
66
  import transformers
67
  import torch
68
 
69
- model_id = "aisingapore/gemma2-9b-cpt-sea-lionv3.0-instruct"
70
 
71
  pipeline = transformers.pipeline(
72
  "text-generation",
@@ -95,10 +95,10 @@ Current SEA-LION models, including this commercially permissive release, have no
95
 
96
  ## Technical Specifications
97
  ### Fine-Tuning Details
98
- Gemma2 9B CPT SEA-LIONv3.0 Instruct was built using a combination of a full parameter fine-tune, alignment, alongside model merges of the best performing checkpoints. The training process for fine-tuning was approximately 15 hours, with alignment taking 2 hours on 8x H100-80GB GPUs.
99
 
100
  ## Data
101
- Gemma2 9B CPT SEA-LIONv3.0 Instruct was trained on a wide range of synthetic instructions alongside those hand-curated by the team, with the assistance of native speakers. In addition, special care was taken to ensure that the datasets used had commercially permissive licenses through verification with the original data source.
102
 
103
  ## Call for Contributions
104
  We encourage researchers, developers, and language enthusiasts to actively contribute to the enhancement and expansion of SEA-LION. Contributions can involve identifying and reporting bugs, sharing pre-training, instruction, and preference data, improving documentation usability, proposing and implementing new model evaluation tasks and metrics, or training versions of the model in additional Southeast Asian languages. Join us in shaping the future of SEA-LION by sharing your expertise and insights to make these models more accessible, accurate, and versatile. Please check out our GitHub for further information on the call for contributions.
 
7
  - vi
8
  license: gemma
9
  ---
10
+ # Gemma2 9B CPT SEA-LIONv3 Instruct
11
 
12
  SEA-LION is a collection of Large Language Models (LLMs) which has been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
13
 
14
+ Gemma2 9B CPT SEA-LIONv3 Instruct is a multilingual model which has been fine-tuned with around **500,000 English instruction-completion pairs** alongside a larger pool of around **1,000,000 instruction-completion pairs** from other ASEAN languages, such as Indonesian, Thai and Vietnamese.
15
 
16
  SEA-LION stands for _Southeast Asian Languages In One Network_.
17
 
 
24
  ## Model Details
25
 
26
  ### Model Description
27
+ We performed instruction tuning in English and also in ASEAN languages such as Indonesian, Thai and Vietnamese on our [continued pre-trained Gemma2 9B CPT SEA-LIONv3](https://huggingface.co/aisingapore/gemma2-9b-cpt-sea-lionv3-base), a decoder model using the Gemma2 architecture, to create Gemma2 9B CPT SEA-LIONv3 Instruct.
28
 
29
  The model has a context length of 8192.
30
 
31
  ### Benchmark Performance
32
+ We evaluated Gemma2 9B CPT SEA-LIONv3 Instruct on both general language capabilities and instruction-following capabilities.
33
 
34
  #### General Language Capabilities
35
  For the evaluation of general language capabilities, we employed the [BHASA evaluation benchmark](https://arxiv.org/abs/2309.06085v2) across a variety of tasks.
 
41
 
42
 
43
  #### Instruction-following Capabilities
44
+ Since Gemma2 9B CPT SEA-LIONv3 Instruct is an instruction-following model, we also evaluated it on instruction-following capabilities with two datasets, [IFEval](https://arxiv.org/abs/2311.07911) and [MT-Bench](https://arxiv.org/abs/2306.05685).
45
 
46
  As these two datasets were originally in English, the linguists and native speakers in the team worked together to filter, localize and translate the datasets into the respective target languages to ensure that the examples remained reasonable, meaningful and natural.
47
 
 
55
  MT-Bench evaluates a model's ability to engage in multi-turn (2 turns) conversations and respond in ways that align with human needs. We use `gpt-4-1106-preview` as the judge model and compare against `gpt-3.5-turbo-0125` as the baseline model. The metric used is the weighted win rate against the baseline model (i.e. average win rate across each category (Math, Reasoning, STEM, Humanities, Roleplay, Writing, Extraction)). A tie is given a score of 0.5.
56
 
57
 
58
+ For more details on Gemma2 9B CPT SEA-LIONv3 Instruct benchmark performance, please refer to the SEA HELM leaderboard, https://leaderboard.sea-lion.ai/
59
 
60
 
61
  ### Usage
62
+ Gemma2 9B CPT SEA-LIONv3 Instruct can be run using the 🤗 Transformers library
63
  ```python
64
  # Please use transformers==4.45.2
65
 
66
  import transformers
67
  import torch
68
 
69
+ model_id = "aisingapore/gemma2-9b-cpt-sea-lionv3-instruct"
70
 
71
  pipeline = transformers.pipeline(
72
  "text-generation",
 
95
 
96
  ## Technical Specifications
97
  ### Fine-Tuning Details
98
+ Gemma2 9B CPT SEA-LIONv3 Instruct was built using a combination of a full parameter fine-tune, alignment, alongside model merges of the best performing checkpoints. The training process for fine-tuning was approximately 15 hours, with alignment taking 2 hours on 8x H100-80GB GPUs.
99
 
100
  ## Data
101
+ Gemma2 9B CPT SEA-LIONv3 Instruct was trained on a wide range of synthetic instructions alongside those hand-curated by the team, with the assistance of native speakers. In addition, special care was taken to ensure that the datasets used had commercially permissive licenses through verification with the original data source.
102
 
103
  ## Call for Contributions
104
  We encourage researchers, developers, and language enthusiasts to actively contribute to the enhancement and expansion of SEA-LION. Contributions can involve identifying and reporting bugs, sharing pre-training, instruction, and preference data, improving documentation usability, proposing and implementing new model evaluation tasks and metrics, or training versions of the model in additional Southeast Asian languages. Join us in shaping the future of SEA-LION by sharing your expertise and insights to make these models more accessible, accurate, and versatile. Please check out our GitHub for further information on the call for contributions.