srimanth-d

/

GOT_CPU

like 8

Yes, I think it would be possible to do so. I am currently working on that. Please feel free to drop message in here, for any updates.

Do you think it would be possible to support MPS acceleration? (Apple Silicon)

I think I made the necessary changes to the code. Please let me know if you are able to run it on MPS.
@Youcef

If you are able to run this successfully on MPS, I will close this discussion.

@Youcef

mind giving me till friday, a bit busy with exams right now

gonna give some updates since im also using MPS, but I can't run it in MPS, it give me this error

note that I also forcefully make the model to keep running on "mps", so after initiating the model, i wrote model = model.to("mps") -> model = model.eval(), then the normal model.chat()

I am looking into it @srntokyo

I think I am not able to run this model on "mps". So I am sticking with "cpu". I mean, I am dropping the idea of getting this model running on "mps". It will still run perfectly fine on "cuda" and "cpu".
@srntokyo Please let me know how are you able to run it on mps?
If anyone if able to figure out how to run this model on "mps", please let me know.