princeton-nlp
commited on
Commit
•
ed635e9
1
Parent(s):
a437579
Update README.md
Browse files
README.md
CHANGED
@@ -15,4 +15,8 @@ license: llama2
|
|
15 |
|
16 |
**License**: Must comply with license of Llama2 since it's a model derived from Llama2.
|
17 |
|
18 |
-
Sheared-LLaMA-2.7B-Pruned is the model pruned from [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) **without continued pre-training**.
|
|
|
|
|
|
|
|
|
|
15 |
|
16 |
**License**: Must comply with license of Llama2 since it's a model derived from Llama2.
|
17 |
|
18 |
+
Sheared-LLaMA-2.7B-Pruned is the model pruned from [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) **without continued pre-training**.
|
19 |
+
We used roughly 0.4B tokens to perform the pruning experiment. This model could be a good use to study
|
20 |
+
- effective data mixtures for continued pre-training
|
21 |
+
- comparisons to other pruning techniques
|
22 |
+
- extensive evaluations to understand how pruning affects knowledge and reasoning capabilities of LLMs
|