Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,7 @@ Trained on 100B tokens.
|
|
| 10 |
- 0.1 wd
|
| 11 |
- WSD scheduler with 10% decay
|
| 12 |
- 80% code, 10% NL, 10% instruction data
|
|
|
|
| 13 |
|
| 14 |
|
| 15 |
This is a *base* pretrained model and requires further fine tuning to be useful.
|
|
|
|
| 10 |
- 0.1 wd
|
| 11 |
- WSD scheduler with 10% decay
|
| 12 |
- 80% code, 10% NL, 10% instruction data
|
| 13 |
+
- 8x3090s 110~ hours
|
| 14 |
|
| 15 |
|
| 16 |
This is a *base* pretrained model and requires further fine tuning to be useful.
|