Spaces:
Running
Fail early if model requires `trust_remote_code`
That seems like a good idea.. though I'm a little confused why the model would require remote code as the implementations are all in MLX?
Great question, you are right!
It's actually failing when loading the tokenizer, because the repo is using a custom implementation based on tiktoken.
Not sure how to fix this during conversion. For now, perhaps we could relax the error check to be restricted to the tokenizer; the model would load correctly if the architecture is supported in mlx as you said.
Wdyt about passing trust_remote_code = true by default for the tokenizer? I think that's basically what the kwarg in convert does: https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/convert.py#L97
Not sure about the security implementations. I did verify that trust_remote_code is indeed passed when loading the tokenizer, and then we don't do anything with it except saving. But loading the tokenizer may execute arbitrary code during initialization when using trust_remote_code.
I relaxed the test to just check the tokenizer. This shouldn't change the app's behaviour, as conversion will fail eventually; it will just fail faster now.
Another solution would be to ignore the tokenizer (and not save it) if it requires remote code, but still convert the model. I can look into that in a new PR.