Int4 Model works in swift-chat on MBP but not in iOS 18 Simulator
When I use the preview branch of the swift-chat repo, I can load the StatefulMistral7BInstructInt4.mlpackage on my MBP (running Version 15.0 Beta (24A5298h)) without a problem. But when I then compile and deploy to an iPhone 15 Pro Max - iOS 18.0 simulator, I get the following error when I try to load the same model:
Compiling model file:///Users/xxx/Library/Developer/CoreSimulator/Devices/DDBDDAD0-F0A5-4851-AE8C-B2283390919F/data/Containers/Shared/AppGroup/AEE61E30-8F34-40C5-8812-B481A8535C1A/File%20Provider%20Storage/StatefulMistral7BInstructInt4.mlpackage
The view service did terminate with error: Error Domain=_UIViewServiceErrorDomain Code=1 "(null)" UserInfo={Terminated=disconnect method}
E5RT encountered an STL exception. msg = Espresso exception: "Invalid state": MpsGraph backend validation on incompatible OS.
E5RT: Espresso exception: "Invalid state": MpsGraph backend validation on incompatible OS (11)
No model could be loaded: Error Domain=com.apple.CoreML Code=0 "MIL program input, 'keyCache', not found in Core ML model inputs" UserInfo={NSLocalizedDescription=MIL program input, 'keyCache', not found in Core ML model inputs}
Correction - I was using the "main" branch of the swift-chat repo, not the "preview" branch.
Any thoughts as to what the problem might be?
@adenler
I am facing the same error. Were you able to solve it for iOS 18?
Kindly let me know. It would be super helpful!