quantization request
#1
by
KT313
- opened
could you maybe also quantize it to something like 4.5bit or 6bit please? 👉👈🥺
I have uploaded some Q4 :) There are going to be more changes to the PR so I'll wait a bit until uploading more
Would a 3-bit quantization fit under 256GB RAM? Is there a quant ?
how 2 run on raspberry pi??
how 2 run on raspberry pi??
ez, just download the 0.000001bpw quant