Update README.md

README.md

@@ -22,6 +22,14 @@ language:
 - it
 - cs
 ---
+## Why prune?
+
+Falcon-11B is still undertrained, as can be seen by this graph:
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/660c0a02cf274b3ab77dd6b7/QeaL9bOrPskustzFpjMUP.png)
+This is why the choice is made by prune 50% of the layers.
+Note that \~1B of continued pre-training (\~1M rows of 1k tokens) is still required to restore the perplexity of this model in the desired language.
+I'm planning on doing that for certain languages, depending on how much compute will be available.
+
 # sliced
 
 This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).