JunxiongWang
commited on
Commit
•
1bf7360
1
Parent(s):
808b3f3
Update README.md
Browse files
README.md
CHANGED
@@ -60,3 +60,15 @@ The following hyperparameters were used during training:
|
|
60 |
- Pytorch 2.1.0+cu118
|
61 |
- Datasets 2.20.0
|
62 |
- Tokenizers 0.19.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
60 |
- Pytorch 2.1.0+cu118
|
61 |
- Datasets 2.20.0
|
62 |
- Tokenizers 0.19.1
|
63 |
+
|
64 |
+
|
65 |
+
[MambaInLlama](arxiv.org/abs/2408.15237)
|
66 |
+
|
67 |
+
```
|
68 |
+
@article{junxiongdaniele2024mambainllama,
|
69 |
+
title = {The Mamba in the Llama: Distilling and Accelerating Hybrid Models},
|
70 |
+
author = {Junxiong Wang and Daniele Paliotta and Avner May and Alexander M. Rush and Tri Dao},
|
71 |
+
journal = {arXiv preprint arXiv:2408.15237},
|
72 |
+
year = {2024}
|
73 |
+
}
|
74 |
+
```
|