Running with llama.cpp on Macbook M2 Pro

by huksley - opened Jan 31, 2024

Jan 31, 2024

Hi, I am trying to run poro-34b.Q4_K_M.gguf and it gives gibberish and an error "ggml_metal_graph_compute: command buffer 5 failed with status 5"

Which quantization works on MacBook M2 Pro 32Gb?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment