ARM speedup.
#1
by
Midgardsormr
- opened
Will the Q4_0_X_X quants speedup also work on Apple SoC??
I believe that yes, the Q4_0_4_4 should work on M1 and newer, and Q4_0_4_8 should work on M2 and newer
i just gave it a try on my iPhone. I normally use iQ3_M quants as the Q4 ones run too slow for my liking but with a Q4_0_4_4 it runs actually pretty good. Q4_0_8_8 and Q4_0_4_8 crash the app (CNVRS in testflight).