May I ask if there are plans to support Ollama?
#1
by
chuqi2233
- opened
I just ran hf.co/IAILabs/Qwen2.5-VL-7b-Instruct-GGUF:Q8_0 on Ollama, and it seems that the image recognition feature is not yet supported.
Unfortunately there are changes that need to be made in the official llama.cpp repo before anything can be propagated to Ollama.
You can monitor progress here: https://github.com/ollama/ollama/issues/6564#issuecomment-2676430795
The Ollama folks are also working on a new multimodal inference engine, but that can potentially take longer.
We'll publish an API wrapper implementation in the meanwhile.
Although now I checked in LM-studio - it works fine for image recognition.
Many tnx to AIVLAD!