tainc commited on
Commit
992f563
1 Parent(s): 207e356

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -95,10 +95,10 @@ Current SEA-LION models, including this commercially permissive release, have no
95
 
96
  ## Technical Specifications
97
  ### Fine-Tuning Details
98
- Gemma2 9B CPT SEA-LIONv3 Instruct was built using a combination of a full parameter fine-tune, alignment, alongside model merges of the best performing checkpoints. The training process for fine-tuning was approximately 15 hours, with alignment taking 2 hours on 8x H100-80GB GPUs.
99
 
100
  ## Data
101
- Gemma2 9B CPT SEA-LIONv3 Instruct was trained on a wide range of synthetic instructions alongside those hand-curated by the team, with the assistance of native speakers. In addition, special care was taken to ensure that the datasets used had commercially permissive licenses through verification with the original data source.
102
 
103
  ## Call for Contributions
104
  We encourage researchers, developers, and language enthusiasts to actively contribute to the enhancement and expansion of SEA-LION. Contributions can involve identifying and reporting bugs, sharing pre-training, instruction, and preference data, improving documentation usability, proposing and implementing new model evaluation tasks and metrics, or training versions of the model in additional Southeast Asian languages. Join us in shaping the future of SEA-LION by sharing your expertise and insights to make these models more accessible, accurate, and versatile. Please check out our GitHub for further information on the call for contributions.
 
95
 
96
  ## Technical Specifications
97
  ### Fine-Tuning Details
98
+ Gemma2 9B CPT SEA-LIONv3 Instruct was built using a combination of a full parameter fine-tune, on-policy alignment, and model merges of the best performing checkpoints. The training process for fine-tuning was approximately 15 hours, with alignment taking 2 hours, both on 8x H100-80GB GPUs.
99
 
100
  ## Data
101
+ Gemma2 9B CPT SEA-LIONv3 Instruct was trained on a wide range of synthetic instructions, alongside publicly available instructions hand-curated by the team with the assistance of native speakers. In addition, special care was taken to ensure that the datasets used had commercially permissive licenses through verification with the original data source.
102
 
103
  ## Call for Contributions
104
  We encourage researchers, developers, and language enthusiasts to actively contribute to the enhancement and expansion of SEA-LION. Contributions can involve identifying and reporting bugs, sharing pre-training, instruction, and preference data, improving documentation usability, proposing and implementing new model evaluation tasks and metrics, or training versions of the model in additional Southeast Asian languages. Join us in shaping the future of SEA-LION by sharing your expertise and insights to make these models more accessible, accurate, and versatile. Please check out our GitHub for further information on the call for contributions.