Update README.md
Browse files
README.md
CHANGED
@@ -19,28 +19,51 @@ tags:
|
|
19 |
|
20 |
### Introduction
|
21 |
|
22 |
-
This is an ONNX version of the Phi-4 multimodal model to accelerate inference with ONNX Runtime.
|
23 |
|
24 |
-
|
|
|
25 |
|
26 |
-
|
27 |
|
28 |
-
|
|
|
|
|
29 |
|
30 |
-
|
31 |
-
|
32 |
-
```
|
33 |
|
34 |
-
|
|
|
|
|
|
|
|
|
|
|
35 |
|
36 |
```bash
|
37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
38 |
```
|
39 |
|
40 |
-
|
41 |
|
42 |
```bash
|
43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
44 |
```
|
45 |
|
46 |
You will be prompted to provide any images, audios, and a prompt.
|
|
|
19 |
|
20 |
### Introduction
|
21 |
|
22 |
+
This is an ONNX version of the Phi-4 multimodal model that is quantized to int4 precision to accelerate inference with ONNX Runtime.
|
23 |
|
24 |
+
## Model Run
|
25 |
+
You can see how to run examples with ORT GenAI [here](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/phi-4-multi-modal.md).
|
26 |
|
27 |
+
For CPU: stay tuned!
|
28 |
|
29 |
+
<!-- ```bash
|
30 |
+
# Download the model directly using the Hugging Face CLI
|
31 |
+
huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/* --local-dir .
|
32 |
|
33 |
+
# Install the CPU package of ONNX Runtime GenAI
|
34 |
+
pip install --pre onnxruntime-genai
|
|
|
35 |
|
36 |
+
# Please adjust the model directory (-m) accordingly
|
37 |
+
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi4-mm.py -o phi4-mm.py
|
38 |
+
python phi4-mm.py -m cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4 -e cpu
|
39 |
+
``` -->
|
40 |
+
|
41 |
+
For CUDA:
|
42 |
|
43 |
```bash
|
44 |
+
# Download the model directly using the Hugging Face CLI
|
45 |
+
huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include gpu/* --local-dir .
|
46 |
+
|
47 |
+
# Install the CUDA package of ONNX Runtime GenAI
|
48 |
+
pip install --pre onnxruntime-genai-cuda
|
49 |
+
|
50 |
+
# Please adjust the model directory (-m) accordingly
|
51 |
+
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi4-mm.py -o phi4-mm.py
|
52 |
+
python phi4-mm.py -m gpu/gpu-int4-rtn-block-32 -e cuda
|
53 |
```
|
54 |
|
55 |
+
For DirectML:
|
56 |
|
57 |
```bash
|
58 |
+
# Download the model directly using the Hugging Face CLI
|
59 |
+
huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include gpu/* --local-dir .
|
60 |
+
|
61 |
+
# Install the DML package of ONNX Runtime GenAI
|
62 |
+
pip install --pre onnxruntime-genai-directml
|
63 |
+
|
64 |
+
# Please adjust the model directory (-m) accordingly
|
65 |
+
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi4-mm.py -o phi4-mm.py
|
66 |
+
python phi4-mm.py -m gpu/gpu-int4-rtn-block-32 -e dml
|
67 |
```
|
68 |
|
69 |
You will be prompted to provide any images, audios, and a prompt.
|