Mistral-Nemo-Instruct-2407 just hangs on my Mac M3?
#53
by
lancejpollard
- opened
Here is the question I posted on StackOverflow, cross-posting here: https://stackoverflow.com/questions/78801372/huggingface-mistral-nemo-instruct-2407-python-script-for-text-generation-just-ha
It's basically that, when running on my Mac M3 with 4 TB hard drive and 36 GB RAM, it just hangs, I am not sure what's going on. Any ideas how to get this to work?
On my M1 Max with 32 GB I need to use the following command in the terminal:
sudo sysctl iogpu.wired_limit_mb=28000
This will allow more memory to be used for the GPUs. Maybe you will also have to limit the context size as the default size is very large.