AlekseyCalvin

/

FluxFillDev_fp8_Diffusers

like 0

Hi @AlekseyCalvin , I am trying to load this Flux-Fill-Dev model for FP8 precision but I am getting error of shape mis-match

ValueError: Trying to set a tensor of shape torch.Size([3072, 384]) in "weight" (which has shape torch.Size([3072, 64])), this looks incorrect.

Here is my code

from diffusers import FluxTransformer2DModel, FluxFillPipeline
from transformers import T5EncoderModel
import torch

transformer = FluxTransformer2DModel.from_pretrained("AlekseyCalvin/FluxFillDev_fp8_Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16).to("cuda")
text_encoder_2 = T5EncoderModel.from_pretrained("AlekseyCalvin/FluxFillDev_fp8_Diffusers", subfolder="text_encoder_2", torch_dtype=torch.bfloat16).to("cuda")
pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", transformer=transformer, text_encoder_2=text_encoder_2, torch_dtype=torch.bfloat16).to("cuda")

Even if I try I get the same error.

FluxFillPipeline.from_pretrained("AlekseyCalvin/FluxFillDev_fp8_Diffusers", transformer=transformer, text_encoder_2=text_encoder_2, torch_dtype=torch.bfloat16).to("cuda")

The error occured while loading transformer of FLUX. Can you please help me with this ? Thanks.