JunxiongWang
commited on
Commit
•
b616ed5
1
Parent(s):
43044c5
Update README.md
Browse files
README.md
CHANGED
@@ -76,3 +76,15 @@ The following hyperparameters were used during training:
|
|
76 |
- Pytorch 2.1.1+cu118
|
77 |
- Datasets 2.20.0
|
78 |
- Tokenizers 0.19.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
76 |
- Pytorch 2.1.1+cu118
|
77 |
- Datasets 2.20.0
|
78 |
- Tokenizers 0.19.1
|
79 |
+
|
80 |
+
|
81 |
+
[MambaInLlama](arxiv.org/abs/2408.15237)
|
82 |
+
|
83 |
+
```
|
84 |
+
@article{junxiongdaniele2024mambainllama,
|
85 |
+
title = {The Mamba in the Llama: Distilling and Accelerating Hybrid Models},
|
86 |
+
author = {Junxiong Wang and Daniele Paliotta and Avner May and Alexander M. Rush and Tri Dao},
|
87 |
+
journal = {arXiv preprint arXiv:2408.15237},
|
88 |
+
year = {2024}
|
89 |
+
}
|
90 |
+
```
|