dicta-il

/

dictalm2.0-instruct-AWQ

like 0

Follow

DICTA: The Israel Center for Text Analysis 41

Hi All,
I have read somewhere that if you don't have a GPU, you can run this model. However I am trying to import:
import intel_extension_for_pytorch as ipex # intel-extension-for-pytorch

and the error is:
"RuntimeError: GPU is required to run AWQ quantized model. You can use IPEX version AWQ if you have an Intel CPU"

Then I have read and found that this package (intel-extension-for-pytorch) is working only on Linux. I even went with it further and installed virtual box. Where I run Ubuntu 24.04. But I get the same error. It might be something with the versions (that should be compatible to my exact CPU).

So my main questions are: should this model work on Windows? And where did I go wrong?

Best,
Eli