notebooks/model-tools/model-tools.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "artificial-discretion",
   "metadata": {},
   "source": [
    "# Working with Open Model Zoo Models\n",
    "This tutorial shows how to download a model from [Open Model Zoo](https://github.com/openvinotoolkit/open_model_zoo), convert it to OpenVINO™ IR format, show information about the model, and benchmark the model.\n",
    "\n",
    "\n",
    "#### Table of contents:\n",
    "\n",
    "- [OpenVINO and Open Model Zoo Tools](#OpenVINO-and-Open-Model-Zoo-Tools)\n",
    "- [Preparation](#Preparation)\n",
    "    - [Model Name](#Model-Name)\n",
    "    - [Imports](#Imports)\n",
    "    - [Settings and Configuration](#Settings-and-Configuration)\n",
    "- [Download a Model from Open Model Zoo](#Download-a-Model-from-Open-Model-Zoo)\n",
    "- [Convert a Model to OpenVINO IR format](#Convert-a-Model-to-OpenVINO-IR-format)\n",
    "- [Get Model Information](#Get-Model-Information)\n",
    "- [Run Benchmark Tool](#Run-Benchmark-Tool)\n",
    "    - [Benchmark with Different Settings](#Benchmark-with-Different-Settings)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4dda33e",
   "metadata": {},
   "source": [
    "## OpenVINO and Open Model Zoo Tools\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "OpenVINO and Open Model Zoo tools are listed in the table below.\n",
    "\n",
    "| Tool             | Command             | Description                                             |\n",
    "|:-----------------|:--------------------|:--------------------------------------------------------|\n",
    "| Model Downloader | `omz_downloader`      | Download models from Open Model Zoo.                    |\n",
    "| Model Converter  | `omz_converter`       | Convert Open Model Zoo models to OpenVINO's IR format.  |\n",
    "| Info Dumper      | `omz_info_dumper`     | Print information about Open Model Zoo models.          |\n",
    "| Benchmark Tool   | `benchmark_app`       | Benchmark model performance by computing inference time.|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "57288459",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "# Install openvino package\n",
    "%pip install -q \"openvino-dev>=2024.0.0\" torch torchvision --extra-index-url https://download.pytorch.org/whl/cpu"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fec90f61-08f3-4417-8a6f-ec61c9e5955b",
   "metadata": {
    "tags": [
     "hide"
    ]
   },
   "source": [
    "## Preparation\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "### Model Name\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Set `model_name` to the name of the Open Model Zoo model to use in this notebook. Refer to the list of [public](https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/public/index.md) and [Intel](https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/intel/index.md) pre-trained models for a full list of models that can be used. Set `model_name` to the model you want to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "21d6d2be-05a8-4c52-9ad9-af892a76db1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# model_name = \"resnet-50-pytorch\"\n",
    "model_name = \"mobilenet-v2-pytorch\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0206a22-dc33-4666-ab2b-86386b97caca",
   "metadata": {
    "tags": [
     "hide"
    ]
   },
   "source": [
    "### Imports\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "impressed-uncertainty",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from pathlib import Path\n",
    "\n",
    "import openvino as ov\n",
    "from IPython.display import Markdown, display\n",
    "\n",
    "# Fetch `notebook_utils` module\n",
    "import requests\n",
    "\n",
    "r = requests.get(\n",
    "    url=\"https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py\",\n",
    ")\n",
    "\n",
    "open(\"notebook_utils.py\", \"w\").write(r.text)\n",
    "from notebook_utils import DeviceNotFoundAlert, NotebookAlert"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "parental-assets",
   "metadata": {
    "tags": [
     "hide"
    ]
   },
   "source": [
    "### Settings and Configuration\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Set the file and directory paths. By default, this notebook downloads models from Open Model Zoo to the `open_model_zoo_models` directory in your `$HOME` directory. On Windows, the $HOME directory is usually `c:\\users\\username`, on Linux `/home/username`. To change the folder, change `base_model_dir` in the cell below.\n",
    "\n",
    "The following settings can be changed:\n",
    "\n",
    "* `base_model_dir`: Models will be downloaded into the `intel` and `public` folders in this directory.\n",
    "* `omz_cache_dir`: Cache folder for Open Model Zoo. Specifying a cache directory is not required for Model Downloader and Model Converter, but it speeds up subsequent downloads.\n",
    "* `precision`: If specified, only models with this precision will be downloaded and converted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "korean-agency",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "base_model_dir: model, omz_cache_dir: cache, gpu_availble: False\n"
     ]
    }
   ],
   "source": [
    "base_model_dir = Path(\"model\")\n",
    "omz_cache_dir = Path(\"cache\")\n",
    "precision = \"FP16\"\n",
    "\n",
    "# Check if an GPU is available on this system to use with Benchmark App.\n",
    "core = ov.Core()\n",
    "gpu_available = \"GPU\" in core.available_devices\n",
    "\n",
    "print(f\"base_model_dir: {base_model_dir}, omz_cache_dir: {omz_cache_dir}, gpu_availble: {gpu_available}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "judicial-preview",
   "metadata": {},
   "source": [
    "## Download a Model from Open Model Zoo\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rising-interval",
   "metadata": {},
   "source": [
    "Specify, display and run the Model Downloader command to download the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "df2b0446-0c49-41cf-9f3f-dec1232249f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Uncomment the next line to show help in omz_downloader which explains the command-line options.\n",
    "\n",
    "# !omz_downloader --help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "556d0c12-15cf-492d-a1ed-41dff5090eff",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Download command: `omz_downloader --name mobilenet-v2-pytorch --output_dir model --cache_dir cache`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Downloading mobilenet-v2-pytorch..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "################|| Downloading mobilenet-v2-pytorch ||################\n",
      "\n",
      "========== Retrieving model/public/mobilenet-v2-pytorch/mobilenet_v2-b0353104.pth from the cache\n",
      "\n"
     ]
    }
   ],
   "source": [
    "download_command = f\"omz_downloader --name {model_name} --output_dir {base_model_dir} --cache_dir {omz_cache_dir}\"\n",
    "display(Markdown(f\"Download command: `{download_command}`\"))\n",
    "display(Markdown(f\"Downloading {model_name}...\"))\n",
    "! $download_command"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "proprietary-checklist",
   "metadata": {},
   "source": [
    "## Convert a Model to OpenVINO IR format\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Specify, display and run the Model Converter command to convert the model to OpenVINO IR format. Model conversion may take a while. The output of the Model Converter command will be displayed. When the conversion is successful, the last lines of the output will include: `[ SUCCESS ] Generated IR version 11 model.` For downloaded models that are already in OpenVINO IR format, conversion will be skipped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "11fe7461-90db-4585-b55f-b3df42b01274",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Uncomment the next line to show Help in omz_converter which explains the command-line options.\n",
    "\n",
    "# !omz_converter --help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "engaged-academy",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Convert command: `omz_converter --name mobilenet-v2-pytorch --precisions FP16 --download_dir model --output_dir model`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Converting mobilenet-v2-pytorch..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "========== Converting mobilenet-v2-pytorch to ONNX\n",
      "Conversion to ONNX command: /home/ea/work/my_optimum_intel/optimum_env/bin/python -- /home/ea/work/my_optimum_intel/optimum_env/lib/python3.8/site-packages/openvino/model_zoo/internal_scripts/pytorch_to_onnx.py --model-name=mobilenet_v2 --weights=model/public/mobilenet-v2-pytorch/mobilenet_v2-b0353104.pth --import-module=torchvision.models --input-shape=1,3,224,224 --output-file=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx --input-names=data --output-names=prob\n",
      "\n",
      "ONNX check passed successfully.\n",
      "\n",
      "========== Converting mobilenet-v2-pytorch to IR (FP16)\n",
      "Conversion command: /home/ea/work/my_optimum_intel/optimum_env/bin/python -- /home/ea/work/my_optimum_intel/optimum_env/bin/mo --framework=onnx --output_dir=model/public/mobilenet-v2-pytorch/FP16 --model_name=mobilenet-v2-pytorch --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.624,57.12,57.375]' --reverse_input_channels --output=prob --input_model=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 224, 224]' --compress_to_fp16=True\n",
      "\n",
      "[ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False.\n",
      "Find more information about compression to FP16 at https://docs.openvino.ai/2024/openvino-workflow/model-preparation/conversion-parameters.html\n",
      "[ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.\n",
      "Find more information about API v2.0 and IR v11 at https://docs.openvino.ai/2023.3/openvino_2_0_transition_guide.html\n",
      "[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. \n",
      "Find more information about transition from MO to OVC at https://docs.openvino.ai/2024/documentation/legacy-features/transition-legacy-conversion-api.html\n",
      "[ SUCCESS ] Generated IR version 11 model.\n",
      "[ SUCCESS ] XML file: /home/ea/work/openvino_notebooks/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.xml\n",
      "[ SUCCESS ] BIN file: /home/ea/work/openvino_notebooks/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.bin\n",
      "\n"
     ]
    }
   ],
   "source": [
    "convert_command = f\"omz_converter --name {model_name} --precisions {precision} --download_dir {base_model_dir} --output_dir {base_model_dir}\"\n",
    "display(Markdown(f\"Convert command: `{convert_command}`\"))\n",
    "display(Markdown(f\"Converting {model_name}...\"))\n",
    "\n",
    "! $convert_command"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa8d655f-215d-4e3c-adcb-e8fd4a2e8ab4",
   "metadata": {},
   "source": [
    "## Get Model Information\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "The Info Dumper prints the following information for Open Model Zoo models:\n",
    "\n",
    "* Model name\n",
    "* Description\n",
    "* Framework that was used to train the model\n",
    "* License URL\n",
    "* Precisions supported by the model\n",
    "* Subdirectory: the location of the downloaded model\n",
    "* Task type\n",
    "\n",
    "This information can be shown by running `omz_info_dumper --name model_name` in a terminal. The information can also be parsed and used in scripts. \n",
    "\n",
    "In the next cell, run Info Dumper and use `json` to load the information in a dictionary. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b8247daf-d3c5-4420-b4c8-d305ac4ace5b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'name': 'mobilenet-v2-pytorch',\n",
       "  'composite_model_name': None,\n",
       "  'description': 'MobileNet V2 is image classification model pre-trained on ImageNet dataset. This is a PyTorch* implementation of MobileNetV2 architecture as described in the paper \"Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation\" <https://arxiv.org/abs/1801.04381>.\\nThe model input is a blob that consists of a single image of \"1, 3, 224, 224\" in \"RGB\" order.\\nThe model output is typical object classifier for the 1000 different classifications matching with those in the ImageNet database.',\n",
       "  'framework': 'pytorch',\n",
       "  'license_url': 'https://raw.githubusercontent.com/pytorch/vision/master/LICENSE',\n",
       "  'accuracy_config': '/home/ea/work/my_optimum_intel/optimum_env/lib/python3.8/site-packages/openvino/model_zoo/models/public/mobilenet-v2-pytorch/accuracy-check.yml',\n",
       "  'model_config': '/home/ea/work/my_optimum_intel/optimum_env/lib/python3.8/site-packages/openvino/model_zoo/models/public/mobilenet-v2-pytorch/model.yml',\n",
       "  'precisions': ['FP16', 'FP32'],\n",
       "  'quantization_output_precisions': ['FP16-INT8', 'FP32-INT8'],\n",
       "  'subdirectory': 'public/mobilenet-v2-pytorch',\n",
       "  'task_type': 'classification',\n",
       "  'input_info': [{'name': 'data',\n",
       "    'shape': [1, 3, 224, 224],\n",
       "    'layout': 'NCHW'}],\n",
       "  'model_stages': []}]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_info_output = %sx omz_info_dumper --name $model_name\n",
    "model_info = json.loads(model_info_output.get_nlstr())\n",
    "\n",
    "if len(model_info) > 1:\n",
    "    NotebookAlert(\n",
    "        f\"There are multiple IR files for the {model_name} model. The first model in the \"\n",
    "        \"omz_info_dumper output will be used for benchmarking. Change \"\n",
    "        \"`selected_model_info` in the cell below to select a different model from the list.\",\n",
    "        \"warning\",\n",
    "    )\n",
    "\n",
    "model_info"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ea7e868-fd2d-4d11-9c87-7aa1f1301083",
   "metadata": {},
   "source": [
    "Having information of the model in a JSON file enables extraction of the path to the model directory, and building the path to the OpenVINO IR file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "de1a319e-bbef-414c-921d-60938b4a01a8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.xml exists: True\n"
     ]
    }
   ],
   "source": [
    "selected_model_info = model_info[0]\n",
    "model_path = base_model_dir / Path(selected_model_info[\"subdirectory\"]) / Path(f\"{precision}/{selected_model_info['name']}.xml\")\n",
    "print(model_path, \"exists:\", model_path.exists())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54e01154-f700-479f-9111-147c95595d46",
   "metadata": {},
   "source": [
    "## Run Benchmark Tool\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "By default, Benchmark Tool runs inference for 60 seconds in asynchronous mode on CPU. It returns inference speed as latency (milliseconds per image) and throughput values (frames per second). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "282452e8-24c7-49c0-bdb2-10677971c30f",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Uncomment the next line to show Help in benchmark_app which explains the command-line options.\n",
    "# !benchmark_app --help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9812b0c8-8cd0-4840-bca3-a28171d055b7",
   "metadata": {
    "tags": [],
    "test_replace": {
     "-t 15": "-t 3"
    }
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Benchmark command: `benchmark_app -m model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.xml -t 15`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Benchmarking mobilenet-v2-pytorch on CPU with async inference for 15 seconds..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Step 1/11] Parsing and validating input arguments\n",
      "[ INFO ] Parsing input parameters\n",
      "[Step 2/11] Loading OpenVINO Runtime\n",
      "[ INFO ] OpenVINO:\n",
      "[ INFO ] Build ................................. 2024.0.0-14412-faf97c13331\n",
      "[ INFO ] \n",
      "[ INFO ] Device info:\n",
      "[ INFO ] CPU\n",
      "[ INFO ] Build ................................. 2024.0.0-14412-faf97c13331\n",
      "[ INFO ] \n",
      "[ INFO ] \n",
      "[Step 3/11] Setting device configuration\n",
      "[ ERROR ] type object 'openvino._pyopenvino.properties.hint.PerformanceMo' has no attribute 'UNDEFINED'\n",
      "Traceback (most recent call last):\n",
      "  File \"/home/ea/work/my_optimum_intel/optimum_env/lib/python3.8/site-packages/openvino/tools/benchmark/main.py\", line 171, in main\n",
      "    set_performance_hint(device)\n",
      "  File \"/home/ea/work/my_optimum_intel/optimum_env/lib/python3.8/site-packages/openvino/tools/benchmark/main.py\", line 109, in set_performance_hint\n",
      "    perf_hint = properties.hint.PerformanceMode.UNDEFINED\n",
      "AttributeError: type object 'openvino._pyopenvino.properties.hint.PerformanceMo' has no attribute 'UNDEFINED'\n"
     ]
    }
   ],
   "source": [
    "benchmark_command = f\"benchmark_app -m {model_path} -t 15\"\n",
    "display(Markdown(f\"Benchmark command: `{benchmark_command}`\"))\n",
    "display(Markdown(f\"Benchmarking {model_name} on CPU with async inference for 15 seconds...\"))\n",
    "\n",
    "! $benchmark_command"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75891996-cf53-4c76-ad3c-5fb468ccd7bb",
   "metadata": {},
   "source": [
    "### Benchmark with Different Settings\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88d64dd7-789d-4536-ab8f-84999c73afaf",
   "metadata": {},
   "source": [
    "The `benchmark_app` tool displays logging information that is not always necessary. A more compact result is achieved when the output is parsed with `json`.\n",
    "\n",
    "The following cells show some examples of `benchmark_app` with different parameters. Below are some useful parameters:\n",
    "\n",
    "- `-d` A device to use for inference. For example: CPU, GPU, MULTI. Default: CPU.\n",
    "- `-t` Time expressed in number of seconds to run inference. Default: 60.\n",
    "- `-api` Use asynchronous (async) or synchronous (sync) inference. Default: async.\n",
    "- `-b` Batch size. Default: 1.\n",
    "\n",
    "\n",
    "Run `! benchmark_app --help` to get an overview of all possible command-line parameters.\n",
    "\n",
    "In the next cell, define the `benchmark_model()` function that calls `benchmark_app`. This makes it easy to try different combinations. In the cell below that, you display available devices on the system.\n",
    "\n",
    "> **Note**: In this notebook, `benchmark_app` runs for 15 seconds to give a quick indication of performance. For more accurate performance, it is recommended to run inference for at least one minute by setting the `t` parameter to 60 or higher, and run `benchmark_app` in a terminal/command prompt after closing other applications. Copy the **benchmark command** and paste it in a command prompt where you have activated the `openvino_env` environment. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7742390e-df71-45e1-9572-f3cbaa576ec3",
   "metadata": {},
   "outputs": [],
   "source": [
    "def benchmark_model(model_xml, device=\"CPU\", seconds=60, api=\"async\", batch=1):\n",
    "    core = ov.Core()\n",
    "    model_path = Path(model_xml)\n",
    "    if (\"GPU\" in device) and (\"GPU\" not in core.available_devices):\n",
    "        DeviceNotFoundAlert(\"GPU\")\n",
    "    else:\n",
    "        benchmark_command = f\"benchmark_app -m {model_path} -d {device} -t {seconds} -api {api} -b {batch}\"\n",
    "        display(Markdown(f\"**Benchmark {model_path.name} with {device} for {seconds} seconds with {api} inference**\"))\n",
    "        display(Markdown(f\"Benchmark command: `{benchmark_command}`\"))\n",
    "\n",
    "        benchmark_output = %sx $benchmark_command\n",
    "        print(\"command ended\")\n",
    "        benchmark_result = [line for line in benchmark_output if not (line.startswith(r\"[\") or line.startswith(\"      \") or line == \"\")]\n",
    "        print(\"\\n\".join(benchmark_result))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "298904f0-638c-4958-876a-3b8c8bd06518",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU: Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz\n"
     ]
    }
   ],
   "source": [
    "core = ov.Core()\n",
    "\n",
    "# Show devices available for OpenVINO Runtime\n",
    "for device in core.available_devices:\n",
    "    device_name = core.get_property(device, \"FULL_DEVICE_NAME\")\n",
    "    print(f\"{device}: {device_name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3896e4bf-f7d0-4529-be97-921ef548de2a",
   "metadata": {},
   "source": [
    "You can select inference device using device widget"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "7e977f84-a382-467c-a4db-7a7210117a90",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "af7e0ee163054ea3ae8fba6390dd11e4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Device:', options=('CPU', 'AUTO'), value='CPU')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import ipywidgets as widgets\n",
    "\n",
    "device = widgets.Dropdown(\n",
    "    options=core.available_devices + [\"AUTO\"],\n",
    "    value=\"CPU\",\n",
    "    description=\"Device:\",\n",
    "    disabled=False,\n",
    ")\n",
    "\n",
    "device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "486919e1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "**Benchmark mobilenet-v2-pytorch.xml with CPU for 15 seconds with async inference**"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Benchmark command: `benchmark_app -m model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.xml -d CPU -t 15 -api async -b 1`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "command ended\n",
      "Traceback (most recent call last):\n",
      "  File \"/home/ea/work/my_optimum_intel/optimum_env/lib/python3.8/site-packages/openvino/tools/benchmark/main.py\", line 171, in main\n",
      "    set_performance_hint(device)\n",
      "  File \"/home/ea/work/my_optimum_intel/optimum_env/lib/python3.8/site-packages/openvino/tools/benchmark/main.py\", line 109, in set_performance_hint\n",
      "    perf_hint = properties.hint.PerformanceMode.UNDEFINED\n",
      "AttributeError: type object 'openvino._pyopenvino.properties.hint.PerformanceMo' has no attribute 'UNDEFINED'\n"
     ]
    }
   ],
   "source": [
    "benchmark_model(model_path, device=device.value, seconds=15, api=\"async\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "openvino_notebooks": {
   "imageUrl": "",
   "tags": {
    "categories": [
     "API Overview",
     "Convert"
    ],
    "libraries": [],
    "other": [],
    "tasks": [
     "Image Classification"
    ]
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}