Update README.md
Browse files
README.md
CHANGED
@@ -28,7 +28,7 @@ We adopted exactly the same architecture and tokenizer as Llama 2. This means Ti
|
|
28 |
|
29 |
In this project, rather than only training a single TinyLlama model, we first train TinyLlama on a corpus of 1.5 trillion tokens to obtain foundational language capabilities. Subsequently, we take this model and turn it into three different models by continual pre-training with three distinct data sampling. For a visual representation of this process, please refer to the figure below.
|
30 |
|
31 |
-

|
32 |
|
33 |
### Pretraining
|
34 |
|