openbmb

/

MiniCPM-Llama3-V-2_5

like 1.37k

Follow

OpenBMB 357

Check this out

https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf

MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail.

and here is the MiniCPM-Llama3-V-2_5-gguf
https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf
@changingshow

Try to import GGUF format model directly into ollama, it can not recognize the image, I don't know what is the problem

@jian2023 Hi! The current way of ollama should not be able to accept the image features, you can wait for our modification or you are welcome to implement it yourself~!

Hi! Thanks for the hardwork! I tried to create a Modelfile in ollama as followed:

FROM ./ggml-model-Q4_K_M.gguf
FROM ./mmproj-model-f16.gguf
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""
TEMPLATE "{{ .System }} USER: {{ .Prompt }} ASSISTANT:"
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
PARAMETER stop "<|reserved_special_token"

If I remove the vision projector, the q4 gguf model works! Ollama core dump though if I combine the projector model. I see that you have forked llama.cpp. It seems that you're providing your own version of CLIP implementation/format for the visual encoder?

https://github.com/OpenBMB/ollama
@y22ma
Here is ollama fork for minicpm-v 2.5
For CLIP, we do make our own adaptations because we have dynamic input for the image

https://ollama.com/ghyghoo8/minicpm-llama3-2_5

I found this, does this support vision?