jekunz
/

smollm-360m-cpt-fineweb-icelandic

Text Generation

text-generation-inference

Model card Files Files and versions Community

jekunz commited on Feb 24

Commit

28f5516

·

verified ·

1 Parent(s): 31dd026

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -18,7 +18,7 @@ Training:
 - LR scheduler: Cosine
 - Warmup ratio: 0.05
 - Batch size: 1
-- 4 A100 (80GB) GPUs
 - Gradient accumulation steps: 32
-- Effective batch size: 128
 - Max. context length: 8192 tokens

 - LR scheduler: Cosine
 - Warmup ratio: 0.05
 - Batch size: 1
+- 8 A100 (80GB) GPUs
 - Gradient accumulation steps: 32
+- Effective batch size: 256
 - Max. context length: 8192 tokens