hardware-heterogeneity/Python/hardware-heterogeneity.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "# Intel® Distribution of OpenVINO™ toolkit hetero plugin\n",
    "\n",
    "\n",
    "    \n",
    "This example shows how to use hetero plugin to define preferences to run different network layers on different hardware types. Here, we will use the command line option to define hetero plugin usage where the layer distribution is already defined. However, hetero plugin also allows developers to customize distribution of layers execution on different hardware by specifying it in the application code.\n",
    "\n",
    "## Car detection tutorial example\n",
    "\n",
    "### 1. Setting the Environment variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!/opt/intel/openvino/bin/setupvars.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "### 2. Run the car detection tutorial with hetero plugin\n",
    "\n",
    "#### a) Prioritizing running on GPU first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 ../../object-detection/Python/tutorial1.py -i /opt/intel/workshop/smart-video-workshop/object-detection/Python/Cars\\ -\\ 1900.mp4 -m /opt/intel/workshop/smart-video-workshop/object-detection/mobilenet-ssd/FP32/mobilenet-ssd.xml -l $HOME/inference_engine_samples_build/intel64/Release/lib/libcpu_extension.so -d HETERO:GPU,CPU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "    \n",
    "#### b) Prioritizing running on CPU first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 ../../object-detection/Python/tutorial1.py -i /opt/intel/workshop/smart-video-workshop/object-detection/Python/Cars\\ -\\ 1900.mp4 -m /opt/intel/workshop/smart-video-workshop/object-detection/mobilenet-ssd/FP32/mobilenet-ssd.xml -l $HOME/inference_engine_samples_build/intel64/Release/lib/libcpu_extension.so -d HETERO:CPU,GPU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Observe the performance time required to process each frame by Inference Engine. For this particular example, inferance ran faster when prioritized for CPU as oppose to when GPU was the first priority.\n",
    "\n",
    " \n",
    "## Inference Engine classification sample\n",
    "\n",
    "\n",
    "Intel® Distribution of OpenVINO™ toolkit install folder (/opt/intel/computer_vision_sdk/) includes various samples for developers to understand how Inference Engine APIs can be used. These samples have -pc flag implmented which shows per topology layer performance report. This will allow to see which layers are running on which hardware. We will run a very basic classification sample as an example in this section. We will provide car image as input to the classification sample. The output will be object labels with confidence numbers.\n",
    "\n",
    "### 1. First, get the classification model and convert that to IR using Model Optimizer\n",
    "\n",
    "For this example, we will use squeezenet model downloaded with the model downlaoder script while setting up the OS for the workshop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! /opt/intel/openvino/deployment_tools/tools/model_downloader/downloader.py --name squeezenet1.1 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! /opt/intel/openvino/deployment_tools/model_optimizer/mo_caffe.py --input_model /opt/intel/workshop/smart-video-workshop/hardware-heterogeneity/Python/classification/squeezenet/1.1/caffe/squeezenet1.1.caffemodel -o /opt/intel/workshop/smart-video-workshop/object-detection/squeezenet/FP32/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "To display labels after classifictaion, you will need a labels file for the SqueezeNet* model. Get the available labels file from demo directory to your working directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%cp /opt/intel/openvino/deployment_tools/demo/squeezenet1.1.labels /opt/intel/workshop/smart-video-workshop/object-detection/squeezenet/FP32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "    \n",
    "### 2. Run classification sample with hetero plugin, prioritizing running on GPU first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! python3 classification_sample.py -i car_1.bmp -m /opt/intel/workshop/smart-video-workshop/object-detection/squeezenet/FP32/squeezenet1.1.xml -d HETERO:GPU,CPU -pc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "perfromance counts:\n",
    "\n",
    "<img src='gpu.png'>\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "    \n",
    "### 3. Now, run with CPU first"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! python3 classification_sample.py -i car_1.bmp -m /opt/intel/workshop/smart-video-workshop/object-detection/squeezenet/FP32/squeezenet1.1.xml -d HETERO:CPU,GPU -pc\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "performance counts:\n",
    "\n",
    "\n",
    "<img src='cpu.png'>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}