ggml-org/gguf-my-repo · Unable to convert Phi 3 small to GGUF

May 22

I get the following error when running this space -

Error: Error converting to fp16: b'INFO:hf-to-gguf:Loading model: Phi-3-small-8k-instruct\nTraceback (most recent call last):\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 2585, in \n main()\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 2563, in main\n model_class = Model.from_model_architecture(hparams["architectures"][0])\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 370, in from_model_architecture\n raise NotImplementedError(f'Architecture {arch!r} not supported!') from None\nNotImplementedError: Architecture 'Phi3SmallForCausalLM' not supported!\n'

Insanelycool

May 22

I got the same (similar) error today on a different model.

reach-vb

ggml.ai org May 27

Hi @aayushg159 - I just tried it, it works for me: https://huggingface.co/reach-vb/Phi-3-mini-4k-instruct-Q8_0-GGUF

Closing this as fixed! (feel free to re-open a new issue)

reach-vb changed discussion status to closed May 27

aayushg159

May 27

Uh, I mentioned Phi-3 small model (link) not mini. Phi-3 mini works as expected. I just ran it again and got the following error -

Error: Error converting to fp16: b'INFO:hf-to-gguf:Loading model: Phi-3-small-8k-instruct\nTraceback (most recent call last):\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 2777, in \n main()\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 2755, in main\n model_class = Model.from_model_architecture(hparams["architectures"][0])\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 369, in from_model_architecture\n raise NotImplementedError(f'Architecture {arch!r} not supported!') from None\nNotImplementedError: Architecture 'Phi3SmallForCausalLM' not supported!\n'