Model not working with llama.cpp
#1
by
virtualspectre
- opened
Hi, The Falcon3-10B not working with llama.cpp
Hi
@virtualspectre
This model is the 1.58bit version of the model, as stated in the model card, you should be running this model with BitNet framework: https://github.com/microsoft/BitNet
Please refer to the instructions presented here: https://huggingface.co/tiiuae/Falcon3-7B-Instruct-1.58bit-GGUF#bitnet for more details