baichuan-inc/Baichuan-M1-14B-Instruct · Requesting Support for GGUF Quantization of Baichuan-M1-14B-Instruct through llama.cpp

Jan 24

Dear Baichuan Team,

I would like to request the support of GGUF quantization through the llama.cpp library.
As this will allow more users to use your new model.
The repo for llama.cpp can be found here: https://github.com/ggerganov/llama.cpp.
Thank you for considering this request.

Doctor-Chad-PhD

10 days ago

Any updates?

J22

10 days ago

Supported by chatllm.cpp.

>python chatllm.py -m :baichuan-m1 -t 0
    ________          __  __    __    __  ___ (百川)
   / ____/ /_  ____ _/ /_/ /   / /   /  |/  /_________  ____  
  / /   / __ \/ __ `/ __/ /   / /   / /|_/ // ___/ __ \/ __ \ 
 / /___/ / / / /_/ / /_/ /___/ /___/ /  / // /__/ /_/ / /_/ / 
 \____/_/ /_/\__,_/\__/_____/_____/_/  /_(_)___/ .___/ .___/  
You are served by Baichuan-M1,                /_/   /_/       
with 14470763520 (14.5B) parameters.

You  > May I ask you some questions about medical knowledge?
A.I. > Of course, I'm here to help. Go ahead and ask your questions.
You  >

Doctor-Chad-PhD

10 days ago

Thank you so much @J22 !