kvaishnavi commited on
Commit
5ae86ff
·
verified ·
1 Parent(s): 29e40c6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -11
README.md CHANGED
@@ -19,28 +19,51 @@ tags:
19
 
20
  ### Introduction
21
 
22
- This is an ONNX version of the Phi-4 multimodal model to accelerate inference with ONNX Runtime.
23
 
24
- This model is quantized to int4 precision and runs on GPU devices.
 
25
 
26
- To run this model with ONNX Runtime:
27
 
28
- Download the model:
 
 
29
 
30
- ```bash
31
- git clone https://huggingface.co/microsoft/Phi-4-multimodal-instruct-onnx
32
- ```
33
 
34
- Download the script to run the model:
 
 
 
 
 
35
 
36
  ```bash
37
- curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/refs/heads/main/examples/python/phi4-mm.py -o phi4-mm.py
 
 
 
 
 
 
 
 
38
  ```
39
 
40
- Run the script
41
 
42
  ```bash
43
- python phi4-mm.py -m Phi-4-multimodal-instruct-onnx/gpu/gpu-int4-rtn-block-32 -e cuda
 
 
 
 
 
 
 
 
44
  ```
45
 
46
  You will be prompted to provide any images, audios, and a prompt.
 
19
 
20
  ### Introduction
21
 
22
+ This is an ONNX version of the Phi-4 multimodal model that is quantized to int4 precision to accelerate inference with ONNX Runtime.
23
 
24
+ ## Model Run
25
+ You can see how to run examples with ORT GenAI [here](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/phi-4-multi-modal.md).
26
 
27
+ For CPU: stay tuned!
28
 
29
+ <!-- ```bash
30
+ # Download the model directly using the Hugging Face CLI
31
+ huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/* --local-dir .
32
 
33
+ # Install the CPU package of ONNX Runtime GenAI
34
+ pip install --pre onnxruntime-genai
 
35
 
36
+ # Please adjust the model directory (-m) accordingly
37
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi4-mm.py -o phi4-mm.py
38
+ python phi4-mm.py -m cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4 -e cpu
39
+ ``` -->
40
+
41
+ For CUDA:
42
 
43
  ```bash
44
+ # Download the model directly using the Hugging Face CLI
45
+ huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include gpu/* --local-dir .
46
+
47
+ # Install the CUDA package of ONNX Runtime GenAI
48
+ pip install --pre onnxruntime-genai-cuda
49
+
50
+ # Please adjust the model directory (-m) accordingly
51
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi4-mm.py -o phi4-mm.py
52
+ python phi4-mm.py -m gpu/gpu-int4-rtn-block-32 -e cuda
53
  ```
54
 
55
+ For DirectML:
56
 
57
  ```bash
58
+ # Download the model directly using the Hugging Face CLI
59
+ huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include gpu/* --local-dir .
60
+
61
+ # Install the DML package of ONNX Runtime GenAI
62
+ pip install --pre onnxruntime-genai-directml
63
+
64
+ # Please adjust the model directory (-m) accordingly
65
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi4-mm.py -o phi4-mm.py
66
+ python phi4-mm.py -m gpu/gpu-int4-rtn-block-32 -e dml
67
  ```
68
 
69
  You will be prompted to provide any images, audios, and a prompt.