Adjust the weights key [HF staff request]

#89
by cyrilvallez HF staff - opened
No description provided.
cyrilvallez changed pull request title from Upload model.safetensors with huggingface_hub to Adjust the weights to contain the correct tied_weight key

Hey!
With recent changes in model loading logic, we noticed that this checkpoint has the wrong key saved. Indeed, 4 weights are tied: model.shared, model.encoder.embed_tokens, model.decoder.embed_tokens, and lm_head. The model expects the weights to actually reside in model.shared to correctly tie them to the other weights at load time, however in this checkpoint only the key model.decoder.embed_tokens is present. We patched it in https://github.com/huggingface/transformers/pull/36572 (which is also a good source of information if I'm not being clear in this message!), but as it looks like it's an isolated case only for this checkpoint, I would appreciate if you could merge this PR directly, so that we can revert the changes in the codebase to avoid code changes directly in Transformers!

So, the only change here in the weights I'm uploading is switching the key from model.decoder.embed_tokens.weight to model.shared.weight for the tied weights πŸ€—

cyrilvallez changed pull request title from Adjust the weights to contain the correct tied_weight key to Adjust the weights key [HF staff request]
Ready to merge
This branch is ready to get merged automatically.
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment