checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture

#1
by joris-sense - opened

The transformers library has version 4.42.4.

I am getting the following error when using the "use this model" code generated by transformers in Colab:

/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:99: UserWarning: 
Error while fetching `HF_TOKEN` secret value from your vault: 'Requesting secret HF_TOKEN timed out. Secrets can only be fetched when running from the Colab UI.'.
You are not authenticated with the Hugging Face Hub in this notebook.
If the error persists, please let us know by opening an issue on GitHub (https://github.com/huggingface/huggingface_hub/issues/new).
  warnings.warn(
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    992             try:
--> 993                 config_class = CONFIG_MAPPING[config_dict["model_type"]]
    994             except KeyError:

3 frames
KeyError: 'llava_llama'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    993                 config_class = CONFIG_MAPPING[config_dict["model_type"]]
    994             except KeyError:
--> 995                 raise ValueError(
    996                     f"The checkpoint you are trying to load has model type `{config_dict['model_type']}` "
    997                     "but Transformers does not recognize this architecture. This could be because of an "

ValueError: The checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
AImageLab org

Hi @joris-sense ,
to use the LLaVA-MORE model, you can run the script available at the following link: https://github.com/aimagelab/LLaVA-MORE/blob/main/llava/eval/run_llava.py
Additionally, you'll find more useful information in the repository!

The transformers version to use is '4.43.1'.

Hey, thanks! I had some trouble with that script as well, I created a PR documenting what I did/could have done.

Sign up or log in to comment