facebook

/

MobileLLM-125M

like 108

Follow

AI at Meta 5.25k

You missed

!pip install --upgrade transformers

even though the configuration states:
"transformers_version": "4.41.2"
https://huggingface.co/facebook/MobileLLM-125M/blob/main/config.json

and colab has 4.42.2

import transformers
transformers.__version__

you have update it to the newest one.

I went down this same path, however running with the newest transformers the tokenizer gets returned as a bool object:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 tokenizer.add_special_tokens(
      2     {
      3         "eos_token": "</s>",
      4         "bos_token": "<s>",
      5         "unk_token": "<unk>",
      6     }
      7 )

AttributeError: 'bool' object has no attribute 'add_special_tokens'

Also see this post of the same issue on the 1B

There's a typo on model card. Please use this command instead:
AutoTokenizer.from_pretrained("facebook/MobileLLM-125M", use_fast=False)

Some weights of the model checkpoint at facebook/MobileLLM-125M were not used when initializing MobileLLMForCausalLM: ['lm_head.weight']

• This IS expected if you are initializing MobileLLMForCausalLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing MobileLLMForCausalLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  Some weights of MobileLLMForCausalLM were not initialized from the model checkpoint at facebook/MobileLLM-125M and are newly initialized: ['model.embed_tokens.weight']
  You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

hi, I load this model using latest transformers(4.47.0) but get this message. What can I do to load model successfully?

You can ignore the warning. The ['lm_head.weight'] is not used because MobileLLM use embedding sharing. So lm_head.weight = embed_tokens.weight.clone()

Hello! Running this with the current version of Transformers gives me the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-5a5008a62dec> in <cell line: 0>()
      1 model_inputs = tokenizer(["A list of colors: red, blue"], return_tensors="pt")
----> 2 generated_ids = model.generate(**model_inputs)
      3 tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

3 frames
~/.cache/huggingface/modules/transformers_modules/facebook/MobileLLM-125M/e09efb976ae4a767cf27a4bf277b7aade91dbd3a/modeling_mobilellm.py in prepare_inputs_for_generation(self, input_ids, past_key_values, attention_mask, inputs_embeds, cache_position, use_cache, **kwargs)
   1277             max_cache_length = (
   1278                 torch.tensor(past_key_values.get_max_length(), device=input_ids.device)
-> 1279                 if past_key_values.get_max_length() is not None
   1280                 else None
   1281             )

AttributeError: 'DynamicCache' object has no attribute 'get_max_length'

It seems like get_max_length was removed in the latest version of transformers and replaced with cache.get_max_cache_shape() instead.