TheBloke

/

llava-v1.5-13B-AWQ

like 33

👆same question @TheBloke

I think is base64?

@TheBloke Can you provide code for adding image to the inference ?

@kopyl , @shubhamsamant , @pabloce , @Varad1707 , You can use llm-awq for llava awq inference currently i believe. However I believe using llama.cpp or llama-cpp-python with gguf llava would be much faster and take less vram + ram.

Here is the notebook for llava awq inference
https://github.com/mit-han-lab/llm-awq/blob/main/examples/llava_demo.ipynb

@YaTharThShaRma999 thank you :)