{ "cells": [ { "cell_type": "markdown", "id": "fd7a2a4d", "metadata": {}, "source": [ "# Stable Diffusion v2.1 using Optimum-Intel OpenVINO and multiple Intel Hardware\n", "\n", "This notebook will provide you a way to see different precision models performing in different hardware. This notebook was done for showing case the use of Optimum-Intel-OpenVINO and it is not optimized for running multiple times.\n", "\n", "![](https://github.com/openvinotoolkit/openvino_notebooks/assets/10940214/1858dae4-72fd-401e-b055-66d503d82446)\n", "\n", "\n", "#### Table of contents:\n", "\n", "- [Showing Info Available Devices](#Showing-Info-Available-Devices)\n", "- [Configure Inference Pipeline](#Configure-Inference-Pipeline)\n", "- [Using full precision model in choice device with OVStableDiffusionPipeline](#Using-full-precision-model-in-choice-device-with-OVStableDiffusionPipeline)\n", " " ] }, { "cell_type": "markdown", "id": "58fe2d87", "metadata": {}, "source": [ "Optimum Intel is the interface between the Transformers and Diffusers libraries and the different tools and libraries provided by Intel to accelerate end-to-end pipelines on Intel architectures. More details in this [repository](https://github.com/huggingface/optimum-intel#openvino). \n", "\n", "`Note: We suggest you to create a different environment and run the following installation command there.`" ] }, { "cell_type": "code", "execution_count": null, "id": "ec2a1a2a", "metadata": {}, "outputs": [], "source": [ "%pip install -q \"optimum-intel[openvino,diffusers]@git+https://github.com/huggingface/optimum-intel.git\" \"ipywidgets\" \"transformers>=4.33.0\" \"torch>=2.1\" --extra-index-url https://download.pytorch.org/whl/cpu" ] }, { "cell_type": "markdown", "id": "4e5f897a", "metadata": {}, "source": [ "Stable Diffusion pipeline should brings 6 elements together, a text encoder model with a tokenizer, a UNet model with and scheduler, and an Autoencoder with Decoder and Encoder models. \n", "\n", "![image ](https://github.com/openvinotoolkit/openvino_notebooks/assets/10940214/e166f225-1220-44aa-a987-84471e03947d)\n", "\n", "The base model used for this example is the stabilityai/stable-diffusion-2-1-base. This model was converted to OpenVINO format, for accelerated inference on CPU or Intel GPU with OpenVINO's integration into Optimum." ] }, { "cell_type": "code", "execution_count": 1, "id": "6960adc6", "metadata": {}, "outputs": [], "source": [ "import warnings\n", "\n", "warnings.filterwarnings(\"ignore\")" ] }, { "cell_type": "markdown", "id": "b3e8c87b", "metadata": {}, "source": [ "### Showing Info Available Devices\n", "[back to top ⬆️](#Table-of-contents:)\n", "\n", "The `available_devices` property shows the available devices in your system. The \"FULL_DEVICE_NAME\" option to `ie.get_property()` shows the name of the device. Check what is the ID name for the discrete GPU, if you have integrated GPU (iGPU) and discrete GPU (dGPU), it will show `device_name=\"GPU.0\"` for iGPU and `device_name=\"GPU.1\"` for dGPU. If you just have either an iGPU or dGPU that will be assigned to `\"GPU\"`" ] }, { "cell_type": "code", "execution_count": 1, "id": "f71b081b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU: Intel(R) Core(TM) Ultra 7 155H\n", "GPU: Intel(R) Arc(TM) Graphics (iGPU)\n", "NPU: Intel(R) AI Boost\n" ] } ], "source": [ "import openvino as ov\n", "\n", "core = ov.Core()\n", "devices = core.available_devices\n", "\n", "for device in devices:\n", " device_name = core.get_property(device, \"FULL_DEVICE_NAME\")\n", " print(f\"{device}: {device_name}\")" ] }, { "cell_type": "markdown", "id": "4adf107c-57b2-4327-9f5b-1b0569b4b93d", "metadata": {}, "source": [ "### Configure Inference Pipeline\n", "[back to top ⬆️](#Table-of-contents:)\n", "\n", "Select device from dropdown list for running inference using OpenVINO." ] }, { "cell_type": "code", "execution_count": 5, "id": "a331f64a-99a1-4d48-a52f-c231c6fd0560", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "40cd0239ace04eb2bd21888b1c839938", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Dropdown(description='Device:', index=1, options=('CPU', 'GPU', 'NPU', 'AUTO'), value='GPU')" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import ipywidgets as widgets\n", "\n", "device = widgets.Dropdown(\n", " options=core.available_devices + [\"AUTO\"],\n", " value=\"CPU\",\n", " description=\"Device:\",\n", " disabled=False,\n", ")\n", "\n", "device" ] }, { "cell_type": "markdown", "id": "7860bb7f", "metadata": {}, "source": [ "### Using full precision model in choice device with `OVStableDiffusionPipeline`\n", "[back to top ⬆️](#Table-of-contents:)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "f8578ffd", "metadata": {}, "outputs": [], "source": [ "from optimum.intel.openvino import OVStableDiffusionPipeline\n", "\n", "# download the pre-converted SD v2.1 model from Hugging Face Hub\n", "name = \"helenai/stabilityai-stable-diffusion-2-1-base-ov\"\n", "ov_pipe = OVStableDiffusionPipeline.from_pretrained(name, compile=False)\n", "ov_pipe.reshape(batch_size=1, height=512, width=512, num_images_per_prompt=1)\n", "ov_pipe.to(device.value)\n", "ov_pipe.compile()" ] }, { "cell_type": "code", "execution_count": 8, "id": "63b2aecf", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "36f615df1383436f8a322442624bb19e", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/18 [00:00" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import gc\n", "\n", "# Generate an image.\n", "prompt = \"red car in snowy forest, epic vista, beautiful landscape, 4k, 8k\"\n", "output_ov = ov_pipe(prompt, num_inference_steps=17, output_type=\"pil\").images[0]\n", "output_ov.save(\"image.png\")\n", "output_ov" ] }, { "cell_type": "code", "execution_count": null, "id": "7601e800", "metadata": {}, "outputs": [], "source": [ "del ov_pipe\n", "gc.collect()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.10" }, "openvino_notebooks": { "imageUrl": "https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/stable-diffusion-v2/stable-diffusion-v2-optimum-demo.png?raw=true", "tags": { "categories": [ "Model Demos", "AI Trends" ], "libraries": [], "other": [ "Stable Diffusion" ], "tasks": [ "Text-to-Image" ] } }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 5 }