CPU optimized version

#3
by gururaser - opened

Hi, can you share any CPU optimized version of this model?

HITsz-Text Machine Group org

Apologies for the delayed response during the holiday period.

We have not yet attempted to convert this model into a CPU-optimized version.
However, we may release an ONNX version of the model in the future for more efficient inference.

HITsz-Text Machine Group org

We have released the GGUF version: HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1.5-GGUF, which was converted using llama.cpp.
Currently, it is functioning properly when deployed via llama.cpp, but it has not yet been tested with LM Studio.
If you have any requirements, you are welcome to try this model.

YanshekWoo changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment