LongWriter-glm4-9b

Original model link: https://huggingface.co/THUDM/LongWriter-glm4-9b

Model by: THUDM

Quants by: QuantPanda

GGUF quantization for llama.cpp and similar applications.

Example:

./llama-cli -m LongWriter-glm4-9B-Q5_K_M.gguf -p "You are a helpful AI assistant." --conversation

If the model takes too long to load you can reduce the context size with --ctx-size

Example with smaller context size:

./llama-cli -m LongWriter-glm4-9B-Q5_K_M.gguf -p "You are a helpful AI assistant." --conversation --ctx-size 4096

Downloads last month
218
GGUF
Model size
9.4B params
Architecture
chatglm

3-bit

4-bit

5-bit

6-bit

8-bit

32-bit

Inference Examples
Unable to determine this model's library. Check the docs .

Model tree for QuantPanda/LongWriter-glm4-9B-GGUF

Quantized
(3)
this model

Dataset used to train QuantPanda/LongWriter-glm4-9B-GGUF