Update README.md

README.md

@@ -76,7 +76,7 @@ The full list of hyperparameters for each model can be found [here](https://gith
 | Layers                  | 48            |
 | Hidden size             | 8,192         |
 | Attention heads         | 64            |
-| Context length          | 8,192         |
+| Context length          | 4,096         |
 | Vocabulary size         | 256,000       |
 | Precision               | bfloat16      |
 | Embedding type          | RoPE          |