Upload MOJO
Browse files
mojo.py
CHANGED
@@ -16,7 +16,7 @@ class RotaryEmbeddingConfig:
|
|
16 |
Parameters to initialize the RotaryEmbedding layer. The rescaling factor allows
|
17 |
to adapt the rotary embeddings to larger lengths than what was used for training.
|
18 |
One of this strategy is presented in the Yarn paper: https://arxiv.org/pdf/2309.00071.pdf. # noqa
|
19 |
-
Args:
|
20 |
"""
|
21 |
|
22 |
rescaling_factor: Optional[float]
|
|
|
16 |
Parameters to initialize the RotaryEmbedding layer. The rescaling factor allows
|
17 |
to adapt the rotary embeddings to larger lengths than what was used for training.
|
18 |
One of this strategy is presented in the Yarn paper: https://arxiv.org/pdf/2309.00071.pdf. # noqa
|
19 |
+
Args:b
|
20 |
"""
|
21 |
|
22 |
rescaling_factor: Optional[float]
|