Update README.md

README.md

@@ -1,3 +1,96 @@
----
-license: mit
----
+---
+license: mit
+---
+
+# **Phi-4-onnx-cpu-int4 Unofficial version**
+
+<b><span style="text-decoration:underline">Note: This is unoffical version,just for test and dev.</span></b>
+
+This is a Phi-4 version of ONNX CPU, based on Olive [https://github.com/microsoft/olive](https://github.com/microsoft/olive). Convert with the following command
+
+
+## **1. Install the SDK**
+
+
+```
+
+pip install olive-ai
+
+pip install transformers==4.44.2
+
+```
+
+
+
+## **2. Convert CPU ONNX Support**
+
+
+```bash
+
+olive auto-opt --model_name_or_path Your Phi-4 location --output_path Your onnx ouput location --device cpu --provider CPUExecutionProvider --precision int4 --use_model_builder --log_level 1
+
+```
+
+This is a conversion, but no specific optimization has been done. Please look forward to the official version.
+
+
+## **Sample - Inference ONNX**
+
+
+```python
+
+
+
+import onnxruntime_genai as og
+import numpy as np
+import os
+
+
+model_folder = "./onnx/model"
+
+
+model = og.Model(model_folder)
+
+
+tokenizer = og.Tokenizer(model)
+tokenizer_stream = tokenizer.create_stream()
+
+
+search_options = {}
+search_options['max_length'] = 2048
+search_options['past_present_share_buffer'] = False
+
+
+chat_template = "<|user|>\n{input}</s>\n<|assistant|>"
+
+
+text = """I have $20,000 in my savings account, where I receive a 4% profit per year and payments twice a year. Can you please tell me how long it will take for me to become a millionaire? Also, can you please explain the math step by step as if you were explaining it to an uneducated person?"""
+
+
+prompt = f'{chat_template.format(input=text)}'
+
+
+input_tokens = tokenizer.encode(prompt)
+
+
+params = og.GeneratorParams(model)
+
+
+params.set_search_options(**search_options)
+params.input_ids = input_tokens
+
+
+generator = og.Generator(model, params)
+
+
+while not generator.is_done():
+      generator.compute_logits()
+      generator.generate_next_token()
+
+      new_token = generator.get_next_tokens()[0]
+      print(tokenizer_stream.decode(new_token), end='', flush=True)
+
+
+
+
+```