After fine tuning, there is a problem for using it.

#50
by SalmanFaroz - opened

I've fine tuned the model with DocVQA Dataset, after finetuning pushed to hub

https://huggingface.co/SalmanFaroz/idefics2-8b-DocVQA-SP

transformers version == 4.40.0

When I use it in private, it works as expected.

Screenshot from 2024-05-15 15-02-47.png

however, if I make it public, I receive the error message "config file not found."

Screenshot from 2024-05-15 14-47-13.png

let me know what i'm missing , and how can i load the model ?

thanks in advance

You don't seem to have this file https://huggingface.co/HuggingFaceM4/idefics2-8b/blob/main/config.json on your repo, did you delete it?

@HugoLaurencon No, I haven't removed it. After fine-tuning, there wasn't a file named config.json, but when I use it in private with the hugging face access token, the adaptor_config is downloaded first, followed by config.json. It works properly

but i tried manually copied the config.json from idefics2 to my repo , but still it wasn't working so i've deleted it

Update : @HugoLaurencon now again i tried adding that config.json and then i get

OSError: SalmanFaroz/idefics2-8b-DocVQA-SP does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.

Yes it's because you don't have any model uploaded on your repo, you currently only have the adapters weights (that I suppose you trained with Lora)!
You need to upload the original weights of Idefics2 too

@SalmanFaroz , you should not need the og idefics2 weights, just the adapter weights are sufficient because in the adapter_config.json file, the base model to which the dapter weights is specified.
i am also able to load your (public) adapted model: from transformers import Idefics2ForConditionalGeneration; Idefics2ForConditionalGeneration.from_pretrained("SalmanFaroz/idefics2-8b-DocVQA-SP").
can you try again from scratch? delete your model, and re-upload it to the hub.
for instance, see my debugging model: https://huggingface.co/VictorSanh/idefics2-8b-docvqa-finetuned-tutorial

Sign up or log in to comment