Phi-3.5-mini-instruct-onnx-cpu Unofficial version

Note: This is unoffical version,just for test and dev.

This is a Phi-3.5-mini-instruct version of ONNX CPU, based on ONNX Runtime for GenAI https://github.com/microsoft/onnxruntime-genai. Convert with the following command

1. Install the SDK


pip install torch transformers onnx onnxruntime


pip install --pre onnxruntime-genai


python3 -m onnxruntime_genai.models.builder -m microsoft/Phi-3.5-mini-instruct  -o ./onnx-cpu -p int4 -e cpu -c ./Phi-3.5-mini-instruct

This is a conversion, but no specific optimization has been done. Please look forward to the official version.