Hugging Face's logo

llm-jp
/
llm-jp-13b-v2.0

Text Generation
Transformers
PyTorch
English
Japanese
llama
text-generation-inference
Model card Files Community
3
hkiyomaru commited on
Commit
e51b61c
·
verified ·
1 Parent(s): 71d0e87

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -83,11 +83,11 @@ print(tokenizer.decode(output))
83
  |13b model|13b|40|5120|40|2048|
84
 
85
 
86
- ## Training (To be updated)
87
 
88
  - **Pre-training:**
89
- - **Hardware:** 96 A100 40GB GPUs ([mdx cluster](https://mdx.jp/en/))
90
- - **Software:** Megatron-DeepSpeed
91
 
92
  - **Instruction tuning:**
93
  - **Hardware:** 8 A100 40GB GPUs ([mdx cluster](https://mdx.jp/en/))
 
83
  |13b model|13b|40|5120|40|2048|
84
 
85
 
86
+ ## Training
87
 
88
  - **Pre-training:**
89
+ - **Hardware:** 128 A100 40GB GPUs ([mdx cluster](https://mdx.jp/en/))
90
+ - **Software:** Megatron-LM
91
 
92
  - **Instruction tuning:**
93
  - **Hardware:** 8 A100 40GB GPUs ([mdx cluster](https://mdx.jp/en/))