raise ValueError(f"Received parameters not in model: {extras}.")
#1
by
AENSaid
- opened
I am trying to load the model using latest from mlx-vlm but getting this error
ValueError: Received parameters not in model: vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.21.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.17.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.5.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.11.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.1.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.1.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.7.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.12.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.11.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.0.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.22.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.15.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.4.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.16.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.18.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.8.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.4.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.21.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.18.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.1.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.12.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.ln_pre.weight vision_tower.vision_model.vision_model.transformer.layers.7.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.1.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.0.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.5.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.3.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.23.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.19.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.2.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.4.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.23.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.8.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.15.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.3.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.6.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.22.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.16.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.14.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.11.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.4.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.15.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.8.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.patch_conv.weight vision_tower.vision_model.vision_model.transformer.layers.6.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.21.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.12.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.8.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.9.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.13.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.1.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.9.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.0.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.9.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.16.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.0.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.19.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.15.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.5.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.20.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.8.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.10.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.3.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.5.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.11.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.1.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.0.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.20.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.0.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.12.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.19.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.14.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.9.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.2.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.15.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.1.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.6.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.0.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.18.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.14.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.12.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.18.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.4.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.18.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.15.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.12.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.20.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.0.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.4.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.10.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.3.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.20.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.11.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.21.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.1.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.18.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.1.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.19.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.2.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.15.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.9.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.2.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.3.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.13.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.4.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.5.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.12.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.11.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.11.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.1.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.13.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.10.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.11.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.2.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.1.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.18.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.6.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.21.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.1.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.11.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.23.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.7.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.6.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.12.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.14.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.19.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.11.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.0.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.21.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.13.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.5.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.0.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.15.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.5.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.10.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.0.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.5.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.18.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.5.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.6.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.7.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.7.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.10.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.23.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.11.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.16.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.17.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.10.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.16.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.12.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.19.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.4.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.4.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.9.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.17.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.19.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.23.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.20.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.17.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.21.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.8.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.6.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.23.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.2.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.15.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.13.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.16.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.6.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.20.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.9.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.21.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.13.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.18.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.8.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.13.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.18.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.5.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.6.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.21.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.19.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.16.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.11.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.3.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.21.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.7.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.8.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.9.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.14.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.21.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.19.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.18.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.0.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.1.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.9.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.5.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.20.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.14.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.4.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.1.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.17.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.22.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.13.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.0.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.21.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.14.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.15.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.attention.o_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.15.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.14.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.5.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.16.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.9.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.5.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.18.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.4.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.18.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.11.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.8.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.13.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.21.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.11.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.2.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.18.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.23.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.16.feed_forward.down_proj.weight vision_tower.vision_model.vision_model.transformer.layers.4.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.17.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.down_proj.biases vision_tower.vision_model.vision_model.transformer.layers.0.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.9.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.22.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.4.attention.k_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.6.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.1.feed_forward.up_proj.scales vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.15.attention.v_proj.scales vision_tower.vision_model.vision_model.transformer.layers.20.attention.k_proj.biases vision_tower.vision_model.vision_model.transformer.layers.3.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.7.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.2.feed_forward.down_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.attention.q_proj.scales vision_tower.vision_model.vision_model.transformer.layers.18.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.14.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.23.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.15.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.3.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.5.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.23.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.7.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.15.feed_forward.up_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.feed_forward.gate_proj.weight vision_tower.vision_model.vision_model.transformer.layers.3.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.10.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.20.feed_forward.up_proj.weight vision_tower.vision_model.vision_model.transformer.layers.12.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.7.feed_forward.gate_proj.biases vision_tower.vision_model.vision_model.transformer.layers.19.attention.o_proj.weight vision_tower.vision_model.vision_model.transformer.layers.21.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.13.attention.k_proj.scales vision_tower.vision_model.vision_model.transformer.layers.0.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.6.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.14.feed_forward.gate_proj.scales vision_tower.vision_model.vision_model.transformer.layers.10.attention_norm.weight vision_tower.vision_model.vision_model.transformer.layers.4.ffn_norm.weight vision_tower.vision_model.vision_model.transformer.layers.22.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.10.attention.o_proj.scales vision_tower.vision_model.vision_model.transformer.layers.15.attention.q_proj.weight vision_tower.vision_model.vision_model.transformer.layers.21.attention.q_proj.biases vision_tower.vision_model.vision_model.transformer.layers.5.attention.v_proj.weight vision_tower.vision_model.vision_model.transformer.layers.11.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.23.attention.v_proj.biases vision_tower.vision_model.vision_model.transformer.layers.4.attention.o_proj.scales.
there was a bug in the quantization for these models. but you can download and convert the source models from the mistral repo and it should work fine:
https://huggingface.co/mistral-community/pixtral-12b/tree/main
Will the model in this repo be replaced by one that works?
I had to change the field model_type
from llava
to pixtral
in config.json
AENSaid
changed discussion status to
closed
Fixed in v0.1.0
https://github.com/Blaizzy/mlx-vlm/pull/41
This is fresh 🔥