Hugging Face's logo

mosaicml
/
mpt-7b

Text Generation
Transformers
PyTorch
mpt
Composer
MosaicML
llm-foundry
StreamingDatasets
custom_code
text-generation-inference
Model card Files Community
95

Change `wte` to use shared embedding

#54
by bcui19 - opened
base: refs/heads/main
←
from: refs/pr/54
Discussion Files changed
+3
-2
bcui19

As title, this will help if, when we need to wrap self.wte with FSDP

Change `wte` to use shared embedding55e39ede
bcui19

Need https://huggingface.co/mosaicml/mpt-7b/discussions/56

bcui19 changed pull request status to closed

Sign up or log in to comment