kubeflow_seldon_e2e_pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# End-to-end Reusable ML Pipeline with Seldon and Kubeflow\n",
    "\n",
    "In this example we showcase how to build re-usable components to build an ML pipeline that can be trained and deployed at scale.\n",
    "\n",
    "We will automate content moderation on the Reddit comments in /r/science building a machine learning NLP model with the following components:\n",
    "\n",
    "![completed-pipeline-deploy](img/completed-pipeline-deploy.jpg)\n",
    "\n",
    "This tutorial will break down in the following sections:\n",
    "\n",
    "1) Test and build all our reusable pipeline steps\n",
    "\n",
    "2) Use Kubeflow to Train the Pipeline and Deploy to Seldon\n",
    "\n",
    "3) Test Seldon Deployed ML REST Endpoints\n",
    "\n",
    "4) Visualise Seldon's Production ML Pipelines\n",
    "\n",
    "## Before you start\n",
    "\n",
    "Make sure you have the following components set-up and running in your Kubernetes cluster:\n",
    "\n",
    "* [Seldon Core installed](https://docs.seldon.io/projects/seldon-core/en/latest/workflow/install.html#install-seldon-core-with-helm) with an [ingress (Ambassador / Istio) set up](https://docs.seldon.io/projects/seldon-core/en/latest/workflow/install.html#ingress-support)\n",
    "* Kubeflow Pipelines [version 1.0.0 Standalone](https://www.kubeflow.org/docs/pipelines/installation/standalone-deployment/#deploying-kubeflow-pipelines) set up\n",
    "\n",
    "Let's get started! 🚀🔥 We will be building the end-to-end pipeline below:\n",
    "\n",
    "![kubeflow-seldon-nlp-full](img/kubeflow-seldon-nlp-full.jpg)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting requirements-dev.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile requirements-dev.txt\n",
    "python-dateutil==2.8.1\n",
    "kfp==1.0.0\n",
    "kubernetes==11.0.0\n",
    "click==7.1.2\n",
    "seldon_core==1.2.3\n",
    "numpy==1.19.1\n",
    "pandas==1.1.1\n",
    "spacy==2.3.2\n",
    "scikit-learn==0.23.2\n",
    "en-core-web-sm==2.3.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install -r requirements-dev.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1) Test and build all our reusable pipeline steps\n",
    "\n",
    "We will start by building each of the components in our ML pipeline. \n",
    "\n",
    "![kubeflow-seldon-nlp-reusable-components](img/kubeflow-seldon-nlp-reusable-components.jpg)\n",
    "\n",
    "### Let's first have a look at our clean_text step:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clean_text\t lr_text_classifier  tfidf_vectorizer\n",
      "data_downloader  spacy_tokenize\n"
     ]
    }
   ],
   "source": [
    "!ls pipeline/pipeline_steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like in this step, all of the other steps can be found in the `pipeline/pipeline_steps/` folder, and all have the following structure:\n",
    "* `pipeline_step.py` which exposes the functionality through a CLI \n",
    "* `Transformer.py` which transforms the data accordingly\n",
    "* `requirements.txt` which states the python dependencies to run\n",
    "* `build_image.sh` which uses `s2i` to build the image with one line\n",
    "\n",
    "### Let's check out the CLI for clean_text\n",
    "The pipeline_step CLI is the entry point for the kubeflow image as it will be able to pass any relevant parameters\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Usage: pipeline_step.py [OPTIONS]\n",
      "\n",
      "Options:\n",
      "  --in-path TEXT\n",
      "  --out-path TEXT\n",
      "  --help           Show this message and exit.\n"
     ]
    }
   ],
   "source": [
    "!python pipeline/pipeline_steps/clean_text/pipeline_step.py --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is actually a very simple file, as we are using the click library to define the commands:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "import dill\n",
      "import click\n",
      "import dill\n",
      "try:\n",
      "    # Running for tests\n",
      "    from .Transformer import Transformer\n",
      "except:\n",
      "    # Running from CLI\n",
      "    from Transformer import Transformer\n",
      "\n",
      "@click.command()\n",
      "@click.option('--in-path', default=\"/mnt/raw_text.data\")\n",
      "@click.option('--out-path', default=\"/mnt/clean_text.data\")\n",
      "def run_pipeline(in_path, out_path):\n",
      "    clean_text_transformer = Transformer()\n",
      "    with open(in_path, 'rb') as in_f:\n",
      "        x = dill.load(in_f)\n",
      "    y = clean_text_transformer.predict(x)\n",
      "    with open(out_path, \"wb\") as out_f:\n",
      "        dill.dump(y, out_f)\n",
      "\n",
      "if __name__ == \"__main__\":\n",
      "    run_pipeline()\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!cat pipeline/pipeline_steps/clean_text/pipeline_step.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Transformer is where the data munging and transformation stage comes in, which will be wrapped by the container and exposed through the Seldon Engine to ensure our pipeline can be used in production.\n",
    "\n",
    "Seldon provides multiple different features, such as abilities to send custom metrics, pre-process / post-process data and more. In this example we will only be exposing the `predict` step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "import re \n",
      "from html.parser import HTMLParser\n",
      "import numpy as np\n",
      "import logging\n",
      "\n",
      "class Transformer():\n",
      "    __html_parser = HTMLParser()\n",
      "    __uplus_pattern = \\\n",
      "        re.compile(\"\\<[uU]\\+(?P<digit>[a-zA-Z0-9]+)\\>\")\n",
      "    __markup_link_pattern = \\\n",
      "        re.compile(\"\\[(.*)\\]\\((.*)\\)\")\n",
      "\n",
      "    def predict(self, X, feature_names=[]):\n",
      "        logging.warning(X)\n",
      "        f = np.vectorize(Transformer.transform_clean_text)\n",
      "        X_clean = f(X)\n",
      "        logging.warning(X_clean)\n",
      "        return X_clean\n",
      "\n",
      "    def fit(self, X, y=None, **fit_params):\n",
      "        return self\n",
      "    \n",
      "    @staticmethod\n",
      "    def transform_clean_text(raw_text):\n",
      "        try:\n",
      "            decoded = raw_text.encode(\"ISO-8859-1\").decode(\"utf-8\")\n",
      "        except:\n",
      "            decoded = raw_text.encode(\"ISO-8859-1\").decode(\"cp1252\")\n",
      "        html_unescaped =Transformer.\\\n",
      "            __html_parser.unescape(decoded) \n",
      "        html_unescaped = re.sub(r\"\\r\\n\", \" \", html_unescaped)\n",
      "        html_unescaped = re.sub(r\"\\r\\r\\n\", \" \", html_unescaped)\n",
      "        html_unescaped = re.sub(r\"\\r\", \" \", html_unescaped)\n",
      "        html_unescaped = html_unescaped.replace(\"&gt;\", \" > \")\n",
      "        html_unescaped = html_unescaped.replace(\"&lt;\", \" < \")\n",
      "        html_unescaped = html_unescaped.replace(\"--\", \" - \")\n",
      "        html_unescaped = Transformer.__uplus_pattern.sub(\n",
      "            \" U\\g<digit> \", html_unescaped)\n",
      "        html_unescaped = Transformer.__markup_link_pattern.sub(\n",
      "            \" \\1 \\2 \", html_unescaped)\n",
      "        html_unescaped = html_unescaped.replace(\"\\\\\", \"\")\n",
      "        return html_unescaped\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!cat pipeline/pipeline_steps/clean_text/Transformer.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to understand how the CLI pipeline talks to each other, have a look at the end to end test in `pipeline/pipeline_tests/`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mTest session starts (platform: linux, Python 3.7.4, pytest 6.0.1, pytest-sugar 0.9.4)\u001b[0m\n",
      "rootdir: /home/alejandro/Programming/kubernetes/seldon/seldon-core/examples/kubeflow\n",
      "plugins: celery-4.4.0, flaky-3.6.1, cov-2.10.0, django-3.8.0, forked-1.1.3, sugar-0.9.4, xdist-1.30.0\n",
      "\u001b[1mcollecting ... \u001b[0m\n",
      " \u001b[36mpipeline/pipeline_tests/\u001b[0mtest_pipeline.py\u001b[0m \u001b[32m✓\u001b[0m                      \u001b[32m100% \u001b[0m\u001b[40m\u001b[32m█\u001b[0m\u001b[40m\u001b[32m█████████\u001b[0m\n",
      "\n",
      "Results (2.12s):\n",
      "\u001b[32m       1 passed\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pytest ./pipeline/pipeline_tests/. --disable-pytest-warnings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To build the image we provide a build script in each of the steps that contains the instructions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#!/bin/bash\n",
      "\n",
      "s2i build . seldonio/seldon-core-s2i-python3:1.15.0-dev clean_text_transformer:0.1\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!cat pipeline/pipeline_steps/clean_text/build_image.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The only thing you need to make sure is that Seldon knows how to wrap the right model and file.\n",
    "\n",
    "This can be achieved with the s2i/environment file. \n",
    "\n",
    "As you can see, here we just tell it we want it to use our `Transformer.py` file:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MODEL_NAME=Transformer\n",
      "API_TYPE=REST\n",
      "SERVICE_TYPE=MODEL\n",
      "PERSISTENCE=0\n"
     ]
    }
   ],
   "source": [
    "!cat pipeline/pipeline_steps/clean_text/.s2i/environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once this is defined, the only thing we need to do is to run the `build_image.sh` for all the reusable components.\n",
    "\n",
    "Here we show the manual way to do it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sending build context to Docker daemon  9.728kB\n",
      "Step 1/4 : FROM python:3.7-slim\n",
      " ---> d3fbf7fff365\n",
      "Step 2/4 : COPY . /microservice\n",
      " ---> Using cache\n",
      " ---> 6e9d7e162536\n",
      "Step 3/4 : WORKDIR /microservice\n",
      " ---> Using cache\n",
      " ---> b3d69a634dd1\n",
      "Step 4/4 : RUN pip install -r requirements.txt\n",
      " ---> Using cache\n",
      " ---> f1c421a68fe7\n",
      "Successfully built f1c421a68fe7\n",
      "Successfully tagged data_downloader:0.1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "---> Installing application source...\n",
      "---> Installing dependencies ...\n",
      "Looking in links: /whl\n",
      "Collecting dill==0.3.2 (from -r requirements.txt (line 1))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/e2/96/518a8ea959a734b70d2e95fef98bcbfdc7adad1c1e5f5dd9148c835205a5/dill-0.3.2.zip (177kB)\n",
      "Requirement already satisfied: click==7.1.2 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (7.1.2)\n",
      "Requirement already satisfied: numpy==1.19.1 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (1.19.1)\n",
      "Building wheels for collected packages: dill\n",
      "Building wheel for dill (setup.py): started\n",
      "Building wheel for dill (setup.py): finished with status 'done'\n",
      "Created wheel for dill: filename=dill-0.3.2-cp37-none-any.whl size=78913 sha256=9f9ec39fffe7a46bdc1a164bc0cd8d61201f04ff0c4bfc68e7dda78d6a4c29cf\n",
      "Stored in directory: /root/.cache/pip/wheels/27/4b/a2/34ccdcc2f158742cfe9650675560dea85f78c3f4628f7daad0\n",
      "Successfully built dill\n",
      "Installing collected packages: dill\n",
      "Successfully installed dill-0.3.2\n",
      "WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Collecting pip-licenses\n",
      "Downloading https://files.pythonhosted.org/packages/c5/50/6c4b4e69a0c43bd9f03a30579695093062ba72da4e3e4026cd2144dbcc71/pip_licenses-2.3.0-py3-none-any.whl\n",
      "Collecting PTable (from pip-licenses)\n",
      "Downloading https://files.pythonhosted.org/packages/ab/b3/b54301811173ca94119eb474634f120a49cd370f257d1aae5a4abaf12729/PTable-0.9.2.tar.gz\n",
      "Building wheels for collected packages: PTable\n",
      "Building wheel for PTable (setup.py): started\n",
      "Building wheel for PTable (setup.py): finished with status 'done'\n",
      "Created wheel for PTable: filename=PTable-0.9.2-cp37-none-any.whl size=22906 sha256=6c6c94080f12738603dbd55c01213f1c28f9922c915c92d72b547e85d229bfba\n",
      "Stored in directory: /root/.cache/pip/wheels/22/cc/2e/55980bfe86393df3e9896146a01f6802978d09d7ebcba5ea56\n",
      "Successfully built PTable\n",
      "Installing collected packages: PTable, pip-licenses\n",
      "Successfully installed PTable-0.9.2 pip-licenses-2.3.0\n",
      "created path: ./licenses/license_info.csv\n",
      "created path: ./licenses/license.txt\n",
      "Build completed successfully\n",
      "---> Installing application source...\n",
      "---> Installing dependencies ...\n",
      "Looking in links: /whl\n",
      "Collecting dill==0.3.2 (from -r requirements.txt (line 1))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/e2/96/518a8ea959a734b70d2e95fef98bcbfdc7adad1c1e5f5dd9148c835205a5/dill-0.3.2.zip (177kB)\n",
      "Requirement already satisfied: click==7.1.2 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (7.1.2)\n",
      "Requirement already satisfied: numpy==1.19.1 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (1.19.1)\n",
      "Collecting scikit-learn==0.23.2 (from -r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/f4/cb/64623369f348e9bfb29ff898a57ac7c91ed4921f228e9726546614d63ccb/scikit_learn-0.23.2-cp37-cp37m-manylinux1_x86_64.whl (6.8MB)\n",
      "Collecting scipy>=0.19.1 (from scikit-learn==0.23.2->-r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/65/f9/f7a7e5009711579c72da2725174825e5056741bf4001815d097eef1b2e17/scipy-1.5.2-cp37-cp37m-manylinux1_x86_64.whl (25.9MB)\n",
      "Collecting joblib>=0.11 (from scikit-learn==0.23.2->-r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/51/dd/0e015051b4a27ec5a58b02ab774059f3289a94b0906f880a3f9507e74f38/joblib-0.16.0-py3-none-any.whl (300kB)\n",
      "Collecting threadpoolctl>=2.0.0 (from scikit-learn==0.23.2->-r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/f7/12/ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133/threadpoolctl-2.1.0-py3-none-any.whl\n",
      "Building wheels for collected packages: dill\n",
      "Building wheel for dill (setup.py): started\n",
      "Building wheel for dill (setup.py): finished with status 'done'\n",
      "Created wheel for dill: filename=dill-0.3.2-cp37-none-any.whl size=78913 sha256=f4aa43d7acbc7953839cc8177c0e03bec249f9a7dffc10c7e9db79201ce519d3\n",
      "Stored in directory: /root/.cache/pip/wheels/27/4b/a2/34ccdcc2f158742cfe9650675560dea85f78c3f4628f7daad0\n",
      "Successfully built dill\n",
      "Installing collected packages: dill, scipy, joblib, threadpoolctl, scikit-learn\n",
      "Successfully installed dill-0.3.2 joblib-0.16.0 scikit-learn-0.23.2 scipy-1.5.2 threadpoolctl-2.1.0\n",
      "WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Collecting pip-licenses\n",
      "Downloading https://files.pythonhosted.org/packages/c5/50/6c4b4e69a0c43bd9f03a30579695093062ba72da4e3e4026cd2144dbcc71/pip_licenses-2.3.0-py3-none-any.whl\n",
      "Collecting PTable (from pip-licenses)\n",
      "Downloading https://files.pythonhosted.org/packages/ab/b3/b54301811173ca94119eb474634f120a49cd370f257d1aae5a4abaf12729/PTable-0.9.2.tar.gz\n",
      "Building wheels for collected packages: PTable\n",
      "Building wheel for PTable (setup.py): started\n",
      "Building wheel for PTable (setup.py): finished with status 'done'\n",
      "Created wheel for PTable: filename=PTable-0.9.2-cp37-none-any.whl size=22906 sha256=e296b386d6581fe3343b84d718f7e7624bdbabfff569c4220a1f2039dd0abbdf\n",
      "Stored in directory: /root/.cache/pip/wheels/22/cc/2e/55980bfe86393df3e9896146a01f6802978d09d7ebcba5ea56\n",
      "Successfully built PTable\n",
      "Installing collected packages: PTable, pip-licenses\n",
      "Successfully installed PTable-0.9.2 pip-licenses-2.3.0\n",
      "created path: ./licenses/license_info.csv\n",
      "created path: ./licenses/license.txt\n",
      "Build completed successfully\n",
      "---> Installing application source...\n",
      "---> Installing dependencies ...\n",
      "Looking in links: /whl\n",
      "Collecting dill==0.3.2 (from -r requirements.txt (line 1))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/e2/96/518a8ea959a734b70d2e95fef98bcbfdc7adad1c1e5f5dd9148c835205a5/dill-0.3.2.zip (177kB)\n",
      "Requirement already satisfied: click==7.1.2 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (7.1.2)\n",
      "Requirement already satisfied: numpy==1.19.1 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (1.19.1)\n",
      "Collecting spacy==2.3.2 (from -r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/55/24/70c615f5b22440c679a4132b81eee67d1dfd70d159505a28ff949c78a1ac/spacy-2.3.2-cp37-cp37m-manylinux1_x86_64.whl (9.9MB)\n",
      "Collecting en-core-web-sm==2.3.1 (from -r requirements.txt (line 5))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "  ERROR: Could not find a version that satisfies the requirement en-core-web-sm==2.3.1 (from -r requirements.txt (line 5)) (from versions: none)\n",
      "ERROR: No matching distribution found for en-core-web-sm==2.3.1 (from -r requirements.txt (line 5))\n",
      "WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Build failed\n",
      "ERROR: An error occurred: non-zero (13) exit code from seldonio/seldon-core-s2i-python37:1.15.0-dev\n",
      "---> Installing application source...\n",
      "---> Installing dependencies ...\n",
      "Looking in links: /whl\n",
      "Collecting dill==0.3.2 (from -r requirements.txt (line 1))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/e2/96/518a8ea959a734b70d2e95fef98bcbfdc7adad1c1e5f5dd9148c835205a5/dill-0.3.2.zip (177kB)\n",
      "Requirement already satisfied: click==7.1.2 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (7.1.2)\n",
      "Requirement already satisfied: numpy==1.19.1 in /opt/conda/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (1.19.1)\n",
      "Collecting scikit-learn==0.23.2 (from -r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/f4/cb/64623369f348e9bfb29ff898a57ac7c91ed4921f228e9726546614d63ccb/scikit_learn-0.23.2-cp37-cp37m-manylinux1_x86_64.whl (6.8MB)\n",
      "Collecting joblib>=0.11 (from scikit-learn==0.23.2->-r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/51/dd/0e015051b4a27ec5a58b02ab774059f3289a94b0906f880a3f9507e74f38/joblib-0.16.0-py3-none-any.whl (300kB)\n",
      "Collecting scipy>=0.19.1 (from scikit-learn==0.23.2->-r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/65/f9/f7a7e5009711579c72da2725174825e5056741bf4001815d097eef1b2e17/scipy-1.5.2-cp37-cp37m-manylinux1_x86_64.whl (25.9MB)\n",
      "Collecting threadpoolctl>=2.0.0 (from scikit-learn==0.23.2->-r requirements.txt (line 4))\n",
      "  WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Downloading https://files.pythonhosted.org/packages/f7/12/ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133/threadpoolctl-2.1.0-py3-none-any.whl\n",
      "Building wheels for collected packages: dill\n",
      "Building wheel for dill (setup.py): started\n",
      "Building wheel for dill (setup.py): finished with status 'done'\n",
      "Created wheel for dill: filename=dill-0.3.2-cp37-none-any.whl size=78913 sha256=2bdafea9bf8ead275cf6123cd330c7362915943a397e00ca7b203b9b8759f2a7\n",
      "Stored in directory: /root/.cache/pip/wheels/27/4b/a2/34ccdcc2f158742cfe9650675560dea85f78c3f4628f7daad0\n",
      "Successfully built dill\n",
      "Installing collected packages: dill, joblib, scipy, threadpoolctl, scikit-learn\n",
      "Successfully installed dill-0.3.2 joblib-0.16.0 scikit-learn-0.23.2 scipy-1.5.2 threadpoolctl-2.1.0\n",
      "WARNING: Url '/whl' is ignored. It is either a non-existing path or lacks a specific scheme.\n",
      "Collecting pip-licenses\n",
      "Downloading https://files.pythonhosted.org/packages/c5/50/6c4b4e69a0c43bd9f03a30579695093062ba72da4e3e4026cd2144dbcc71/pip_licenses-2.3.0-py3-none-any.whl\n",
      "Collecting PTable (from pip-licenses)\n",
      "Downloading https://files.pythonhosted.org/packages/ab/b3/b54301811173ca94119eb474634f120a49cd370f257d1aae5a4abaf12729/PTable-0.9.2.tar.gz\n",
      "Building wheels for collected packages: PTable\n",
      "Building wheel for PTable (setup.py): started\n",
      "Building wheel for PTable (setup.py): finished with status 'done'\n",
      "Created wheel for PTable: filename=PTable-0.9.2-cp37-none-any.whl size=22906 sha256=46a9ca3a63fe171ad06a2e9bbd7f00fba00ca0bb4bbda6177f3571077d79228e\n",
      "Stored in directory: /root/.cache/pip/wheels/22/cc/2e/55980bfe86393df3e9896146a01f6802978d09d7ebcba5ea56\n",
      "Successfully built PTable\n",
      "Installing collected packages: PTable, pip-licenses\n",
      "Successfully installed PTable-0.9.2 pip-licenses-2.3.0\n",
      "created path: ./licenses/license_info.csv\n",
      "created path: ./licenses/license.txt\n",
      "Build completed successfully\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "# we must be in the same directory\n",
    "cd pipeline/pipeline_steps/clean_text/ && ./build_image.sh\n",
    "cd ../data_downloader && ./build_image.sh\n",
    "cd ../lr_text_classifier && ./build_image.sh\n",
    "cd ../spacy_tokenize && ./build_image.sh\n",
    "cd ../tfidf_vectorizer && ./build_image.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3) Train our NLP Pipeline through the Kubeflow UI\n",
    "We can access the Kubeflow dashboard to train our ML pipeline via http://localhost/_/pipeline-dashboard\n",
    "\n",
    "If you can't edit this, you need to make sure that the ambassador gateway service is accessible:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME         TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE\n",
      "ambassador   NodePort   10.97.236.196   <none>        80:30209/TCP   8m58s\n"
     ]
    }
   ],
   "source": [
    "!kubectl get svc ambassador -n kubeflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In my case, I need to change the kind from `NodePort` into `LoadBalancer` which can be done with the following command:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "service/ambassador patched\n"
     ]
    }
   ],
   "source": [
    "!kubectl patch svc ambassador --type='json' -p '[{\"op\":\"replace\",\"path\":\"/spec/type\",\"value\":\"LoadBalancer\"}]' -n kubeflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that I've changed it to a loadbalancer, it has allocated the external IP as my localhost so I can access it at http://localhost/_/pipeline-dashboard\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME         TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE\n",
      "ambassador   LoadBalancer   10.97.236.196   localhost     80:30209/TCP   9m20s\n"
     ]
    }
   ],
   "source": [
    "!kubectl get svc ambassador -n kubeflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If this was successful, you should be able to access the dashboard\n",
    "![kf-pipeline-dashboard](img/k-pipeline-dashboard.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the pipeline\n",
    "Now we want to generate the pipeline. For this we can use the DSL provided by kubeflow to define the actual steps required. \n",
    "\n",
    "The pipeline will look as follows:\n",
    "\n",
    "![kf-seldon-nlp-ml-pipelines](img/kubeflow-seldon-nlp-ml-pipelines.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "import kfp.dsl as dsl\n",
      "import yaml\n",
      "from kubernetes import client as k8s\n",
      "\n",
      "\n",
      "@dsl.pipeline(\n",
      "  name='NLP',\n",
      "  description='A pipeline demonstrating reproducible steps for NLP'\n",
      ")\n",
      "def nlp_pipeline(\n",
      "        csv_url=\"https://raw.githubusercontent.com/axsauze/reddit-classification-exploration/master/data/reddit_train.csv\",\n",
      "        csv_encoding=\"ISO-8859-1\",\n",
      "        features_column=\"BODY\",\n",
      "        labels_column=\"REMOVED\",\n",
      "        raw_text_path='/mnt/text.data',\n",
      "        labels_path='/mnt/labels.data',\n",
      "        clean_text_path='/mnt/clean.data',\n",
      "        spacy_tokens_path='/mnt/tokens.data',\n",
      "        tfidf_vectors_path='/mnt/tfidf.data',\n",
      "        lr_prediction_path='/mnt/prediction.data',\n",
      "        tfidf_model_path='/mnt/tfidf.model',\n",
      "        lr_model_path='/mnt/lr.model',\n",
      "        lr_c_param=0.1,\n",
      "        tfidf_max_features=10000,\n",
      "        tfidf_ngram_range=3,\n",
      "        batch_size='100'):\n",
      "    \"\"\"\n",
      "    Pipeline \n",
      "    \"\"\"\n",
      "    vop = dsl.VolumeOp(\n",
      "      name='my-pvc',\n",
      "      resource_name=\"my-pvc\",\n",
      "      modes=[\"ReadWriteMany\"],\n",
      "      size=\"1Gi\"\n",
      "    )\n",
      "\n",
      "    download_step = dsl.ContainerOp(\n",
      "        name='data_downloader',\n",
      "        image='data_downloader:0.1',\n",
      "        command=\"python\",\n",
      "        arguments=[\n",
      "            \"/microservice/pipeline_step.py\",\n",
      "            \"--labels-path\", labels_path,\n",
      "            \"--features-path\", raw_text_path,\n",
      "            \"--csv-url\", csv_url,\n",
      "            \"--csv-encoding\", csv_encoding,\n",
      "            \"--features-column\", features_column,\n",
      "            \"--labels-column\", labels_column\n",
      "        ],\n",
      "        pvolumes={\"/mnt\": vop.volume}\n",
      "    )\n",
      "\n",
      "    clean_step = dsl.ContainerOp(\n",
      "        name='clean_text',\n",
      "        image='clean_text_transformer:0.1',\n",
      "        command=\"python\",\n",
      "        arguments=[\n",
      "            \"/microservice/pipeline_step.py\",\n",
      "            \"--in-path\", raw_text_path,\n",
      "            \"--out-path\", clean_text_path,\n",
      "        ],\n",
      "        pvolumes={\"/mnt\": download_step.pvolume}\n",
      "    )\n",
      "\n",
      "    tokenize_step = dsl.ContainerOp(\n",
      "        name='tokenize',\n",
      "        image='spacy_tokenizer:0.1',\n",
      "        command=\"python\",\n",
      "        arguments=[\n",
      "            \"/microservice/pipeline_step.py\",\n",
      "            \"--in-path\", clean_text_path,\n",
      "            \"--out-path\", spacy_tokens_path,\n",
      "        ],\n",
      "        pvolumes={\"/mnt\": clean_step.pvolume}\n",
      "    )\n",
      "\n",
      "    vectorize_step = dsl.ContainerOp(\n",
      "        name='vectorize',\n",
      "        image='tfidf_vectorizer:0.1',\n",
      "        command=\"python\",\n",
      "        arguments=[\n",
      "            \"/microservice/pipeline_step.py\",\n",
      "            \"--in-path\", spacy_tokens_path,\n",
      "            \"--out-path\", tfidf_vectors_path,\n",
      "            \"--max-features\", tfidf_max_features,\n",
      "            \"--ngram-range\", tfidf_ngram_range,\n",
      "            \"--action\", \"train\",\n",
      "            \"--model-path\", tfidf_model_path,\n",
      "        ],\n",
      "        pvolumes={\"/mnt\": tokenize_step.pvolume}\n",
      "    )\n",
      "\n",
      "    predict_step = dsl.ContainerOp(\n",
      "        name='predictor',\n",
      "        image='lr_text_classifier:0.1',\n",
      "        command=\"python\",\n",
      "        arguments=[\n",
      "            \"/microservice/pipeline_step.py\",\n",
      "            \"--in-path\", tfidf_vectors_path,\n",
      "            \"--labels-path\", labels_path,\n",
      "            \"--out-path\", lr_prediction_path,\n",
      "            \"--c-param\", lr_c_param,\n",
      "            \"--action\", \"train\",\n",
      "            \"--model-path\", lr_model_path,\n",
      "        ],\n",
      "        pvolumes={\"/mnt\": vectorize_step.pvolume}\n",
      "    )\n",
      "\n",
      "    try:\n",
      "        seldon_config = yaml.load(open(\"../deploy_pipeline/seldon_production_pipeline.yaml\"))\n",
      "    except:\n",
      "        # If this file is run from the project core directory \n",
      "        seldon_config = yaml.load(open(\"deploy_pipeline/seldon_production_pipeline.yaml\"))\n",
      "\n",
      "    deploy_step = dsl.ResourceOp(\n",
      "        name=\"seldondeploy\",\n",
      "        k8s_resource=seldon_config,\n",
      "        attribute_outputs={\"name\": \"{.metadata.name}\"})\n",
      "\n",
      "    deploy_step.after(predict_step)\n",
      "\n",
      "if __name__ == '__main__':\n",
      "  import kfp.compiler as compiler\n",
      "  compiler.Compiler().compile(nlp_pipeline, __file__ + '.tar.gz')\n"
     ]
    }
   ],
   "source": [
    "!cat train_pipeline/nlp_pipeline.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Breaking down the  code\n",
    "As you can see in the DSL, we have the ContainerOp - each of those is a step in the Kubeflow pipeline.\n",
    "\n",
    "At the end we can see the `seldondeploy` step which basically deploys the trained pipeline\n",
    "\n",
    "The definition of the SeldonDeployment graph is provided in the `deploy_pipeline/seldon_production_pipeline.yaml` file.\n",
    "\n",
    "The seldondeployment file defines our production execution graph using the same reusable components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---\n",
      "apiVersion: machinelearning.seldon.io/v1alpha2\n",
      "kind: SeldonDeployment\n",
      "metadata:\n",
      "  labels:\n",
      "    app: seldon\n",
      "  name: \"seldon-deployment-{{workflow.name}}\"\n",
      "  namespace: kubeflow\n",
      "spec:\n",
      "  annotations:\n",
      "    project_name: NLP Pipeline\n",
      "    deployment_version: v1\n",
      "  name: \"seldon-deployment-{{workflow.name}}\"\n",
      "  predictors:\n",
      "  - componentSpecs:\n",
      "    - spec:\n",
      "        containers:\n",
      "        - image: clean_text_transformer:0.1\n",
      "          imagePullPolicy: IfNotPresent\n",
      "          name: cleantext\n",
      "          resources:\n",
      "            requests:\n",
      "              memory: 1Mi\n",
      "        - image: spacy_tokenizer:0.1\n",
      "          imagePullPolicy: IfNotPresent\n",
      "          name: spacytokenizer\n",
      "        - image: tfidf_vectorizer:0.1\n",
      "          imagePullPolicy: IfNotPresent\n",
      "          name: tfidfvectorizer\n",
      "          volumeMounts:\n",
      "          - name: mypvc\n",
      "            mountPath: /mnt\n",
      "        - image: lr_text_classifier:0.1\n",
      "          imagePullPolicy: IfNotPresent\n",
      "          name: lrclassifier\n",
      "          volumeMounts:\n",
      "          - name: mypvc\n",
      "            mountPath: /mnt\n",
      "        terminationGracePeriodSeconds: 20\n",
      "        volumes:\n",
      "        - name: mypvc\n",
      "          persistentVolumeClaim:\n",
      "            claimName: \"{{workflow.name}}-my-pvc\"\n",
      "    graph:\n",
      "      children:\n",
      "      - name: spacytokenizer\n",
      "        endpoint:\n",
      "          type: REST\n",
      "        type: MODEL\n",
      "        children:\n",
      "        - name: tfidfvectorizer\n",
      "          endpoint:\n",
      "            type: REST\n",
      "          type: MODEL\n",
      "          children:\n",
      "          - name: lrclassifier\n",
      "            endpoint:\n",
      "              type: REST\n",
      "            type: MODEL\n",
      "            children: []\n",
      "      name: cleantext\n",
      "      endpoint:\n",
      "        type: REST\n",
      "      type: MODEL\n",
      "    name: single-model\n",
      "    replicas: 1\n",
      "    annotations:\n",
      "      predictor_version: v1\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!cat deploy_pipeline/seldon_production_pipeline.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seldon Production pipeline contents\n",
    "If we look at the file we'll be using to deploy our pipeline, we can see that it has the following key points:\n",
    "\n",
    "1) Reusable components definitions as containerSpecs: cleantext, spacytokenizer, tfidfvectorizer & lrclassifier\n",
    "\n",
    "2) DAG (directed acyclic graph) definition for REST pipeline: cleantext -> spacytokenizer -> tfidfvectorizer -> lrclassifier\n",
    "\n",
    "This graph in our production deployment looks as follows:\n",
    "\n",
    "![kf-seldon-npl-pipelines-deploy](img/kubeflow-seldon-nlp-ml-pipelines-deploy.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate the pipeline files to upload to Kubeflow\n",
    "To generate the pipeline we just have to run the pipeline file, which will output the `tar.gz` file that will be uploaded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nlp_pipeline.py\n",
      "nlp_pipeline.py.tar.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/alejandro/miniconda3/lib/python3.7/site-packages/kfp/components/_data_passing.py:168: UserWarning: Missing type name was inferred as \"Float\" based on the value \"0.1\".\n",
      "  warnings.warn('Missing type name was inferred as \"{}\" based on the value \"{}\".'.format(type_name, str(value)))\n",
      "/home/alejandro/miniconda3/lib/python3.7/site-packages/kfp/components/_data_passing.py:168: UserWarning: Missing type name was inferred as \"Integer\" based on the value \"10000\".\n",
      "  warnings.warn('Missing type name was inferred as \"{}\" based on the value \"{}\".'.format(type_name, str(value)))\n",
      "/home/alejandro/miniconda3/lib/python3.7/site-packages/kfp/components/_data_passing.py:168: UserWarning: Missing type name was inferred as \"Integer\" based on the value \"3\".\n",
      "  warnings.warn('Missing type name was inferred as \"{}\" based on the value \"{}\".'.format(type_name, str(value)))\n",
      "train_pipeline/nlp_pipeline.py:114: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.\n",
      "  seldon_config = yaml.load(open(\"deploy_pipeline/seldon_production_pipeline.yaml\"))\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "# Generating graph definition\n",
    "python train_pipeline/nlp_pipeline.py\n",
    "ls train_pipeline/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Run the pipeline\n",
    "\n",
    "You can access the Kubeflow Pipelines UI by forwarding the port with the following command:\n",
    "\n",
    "```\n",
    " kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8000:80\n",
    "```\n",
    "\n",
    "The UI should now be accessible via [http://localhost:8000](http://localhost:8000).\n",
    "\n",
    "We now need to upload the resulting `nlp_pipeline.py.tar.gz` file generated.\n",
    "\n",
    "This can be done through the \"Upload PIpeline\" button in the UI.\n",
    "\n",
    "Once it's uploaded, we want to create and trigger a run! You should now be able to see how each step is executed:\n",
    "\n",
    "![running-pipeline](img/running-pipeline.jpg)\n",
    "\n",
    "### Inspecting the data created in the Persistent Volume\n",
    "The pipeline saves the output of the pipeline together with the trained model in the persistent volume claim.\n",
    "\n",
    "The persistent volume claim is the same name as the argo workflow:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME        AGE\n",
      "nlp-bddff   2m\n"
     ]
    }
   ],
   "source": [
    "!kubectl get workflow -n kubeflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our workflow is there! So we can actually access it by running"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nlp-bddff"
     ]
    }
   ],
   "source": [
    "!kubectl get workflow -n kubeflow -o jsonpath='{.items[0].metadata.name}'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can use good old `sed` to insert this workflow name in our PVC-Access controler which we can use to inspect the contents of the volume:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: v1\n",
      "kind: Pod\n",
      "metadata:\n",
      "  name: pvc-access-container\n",
      "spec:\n",
      "  containers:\n",
      "  - name: pvc-access-container\n",
      "    image: busybox\n",
      "    command: [\"/bin/sh\", \"-ec\", \"sleep 1000\"]\n",
      "    volumeMounts:\n",
      "    - name: mypvc\n",
      "      mountPath: /mnt\n",
      "  volumes:\n",
      "  - name: mypvc\n",
      "    persistentVolumeClaim:\n",
      "      claimName: nlp-b7qt8-my-pvc\n"
     ]
    }
   ],
   "source": [
    "!sed \"s/PVC_NAME/\"$(kubectl get workflow -n kubeflow -o jsonpath='{.items[0].metadata.name}')\"-my-pvc/g\" deploy_pipeline/pvc-access.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We just need to apply this container with our kubectl command, and we can use it to inspect the mounted folder:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pod/pvc-access-container created\n"
     ]
    }
   ],
   "source": [
    "!sed \"s/PVC_NAME/\"$(kubectl get workflow -n kubeflow -o jsonpath='{.items[0].metadata.name}')\"-my-pvc/g\" deploy_pipeline/pvc-access.yaml | kubectl -n kubeflow apply -f -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME                   READY   STATUS    RESTARTS   AGE\n",
      "pvc-access-container   1/1     Running   0          6s\n"
     ]
    }
   ],
   "source": [
    "!kubectl get pods -n kubeflow pvc-access-container"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can run an `ls` command to see what's inside:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;32mclean.data\u001b[m       \u001b[1;32mlr.model\u001b[m         \u001b[1;32mtext.data\u001b[m        \u001b[1;32mtfidf.model\u001b[m\n",
      "\u001b[1;32mlabels.data\u001b[m      \u001b[1;32mprediction.data\u001b[m  \u001b[1;32mtfidf.data\u001b[m       \u001b[1;32mtokens.data\u001b[m\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kubeflow exec -it pvc-access-container ls /mnt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pod \"pvc-access-container\" deleted\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f deploy_pipeline/pvc-access.yaml -n kubeflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5) Test Deployed ML REST Endpoints\n",
    "Now that it's running we have a production ML text pipeline that we can Query using REST and GRPC\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we can check if our Seldon deployment is running with"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME                          AGE\n",
      "seldon-deployment-nlp-b7qt8   57m\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kubeflow get seldondeployment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will need the Seldon Pipeline Deployment name to reach the API, so we can get it using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldon-deployment-nlp-b7qt8"
     ]
    }
   ],
   "source": [
    "!kubectl -n kubeflow get seldondeployment -o jsonpath='{.items[0].metadata.name}'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can interact with our API in two ways: \n",
    "\n",
    "1) Using CURL or any client like PostMan\n",
    "\n",
    "2) Using the Python SeldonClient\n",
    "\n",
    "### Using CURL from the terminal\n",
    "When using CURL, the only thing we need to provide is the data in JSON format, as well as the url, which is of the format:\n",
    "\n",
    "```\n",
    "http://<ENDPOINT>/seldon/kubeflow/<PIPELINE_NAME>/api/v0.1/predictions\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"k89krp6t7tfgb386nt6vc3iftk\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"cleantext\": -1,\n",
      "      \"tfidfvectorizer\": -1,\n",
      "      \"spacytokenizer\": -1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"cleantext\": \"clean_text_transformer:0.1\",\n",
      "      \"tfidfvectorizer\": \"tfidf_vectorizer:0.1\",\n",
      "      \"lrclassifier\": \"lr_text_classifier:0.1\",\n",
      "      \"spacytokenizer\": \"spacy_tokenizer:0.1\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"t:0\", \"t:1\"],\n",
      "    \"ndarray\": [[0.6729318752883149, 0.3270681247116851]]\n",
      "  }\n",
      "}"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100   599  100   527  100    72    516     70  0:00:01  0:00:01 --:--:--   588\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "curl -X POST -H 'Content-Type: application/json' \\\n",
    "    -d \"{'data': {'names': ['text'], 'ndarray': ['Hello world this is a test']}}\" \\\n",
    "    http://127.0.0.1/seldon/kubeflow/$(kubectl -n kubeflow get seldondeployment -o jsonpath='{.items[0].metadata.name}')/api/v0.1/predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the SeldonClient\n",
    "We can also use the Python SeldonClient to interact with the pipeline we just deployed "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "data {\n",
      "  names: \"text\"\n",
      "  ndarray {\n",
      "    values {\n",
      "      string_value: \"Hello world this is a test\"\n",
      "    }\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "meta {\n",
      "  puid: \"qtdca40d3s0463nn4ginhkvc6t\"\n",
      "  routing {\n",
      "    key: \"cleantext\"\n",
      "    value: -1\n",
      "  }\n",
      "  routing {\n",
      "    key: \"spacytokenizer\"\n",
      "    value: -1\n",
      "  }\n",
      "  routing {\n",
      "    key: \"tfidfvectorizer\"\n",
      "    value: -1\n",
      "  }\n",
      "  requestPath {\n",
      "    key: \"cleantext\"\n",
      "    value: \"clean_text_transformer:0.1\"\n",
      "  }\n",
      "  requestPath {\n",
      "    key: \"lrclassifier\"\n",
      "    value: \"lr_text_classifier:0.1\"\n",
      "  }\n",
      "  requestPath {\n",
      "    key: \"spacytokenizer\"\n",
      "    value: \"spacy_tokenizer:0.1\"\n",
      "  }\n",
      "  requestPath {\n",
      "    key: \"tfidfvectorizer\"\n",
      "    value: \"tfidf_vectorizer:0.1\"\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"t:0\"\n",
      "  names: \"t:1\"\n",
      "  ndarray {\n",
      "    values {\n",
      "      list_value {\n",
      "        values {\n",
      "          number_value: 0.6729318752883149\n",
      "        }\n",
      "        values {\n",
      "          number_value: 0.3270681247116851\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import subprocess\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "from seldon_core.seldon_client import SeldonClient\n",
    "\n",
    "host = \"localhost\"\n",
    "port = \"80\"  # Make sure you use the port above\n",
    "batch = np.array([\"Hello world this is a test\"])\n",
    "payload_type = \"ndarray\"\n",
    "# Get the deployment name\n",
    "deployment_name = subprocess.getoutput(\n",
    "    \"kubectl -n kubeflow get seldondeployment -o jsonpath='{.items[0].metadata.name}'\"\n",
    ")\n",
    "transport = \"rest\"\n",
    "namespace = \"kubeflow\"\n",
    "\n",
    "sc = SeldonClient(\n",
    "    gateway=\"ambassador\", ambassador_endpoint=host + \":\" + port, namespace=namespace\n",
    ")\n",
    "\n",
    "client_prediction = sc.predict(\n",
    "    data=batch,\n",
    "    deployment_name=deployment_name,\n",
    "    names=[\"text\"],\n",
    "    payload_type=payload_type,\n",
    "    transport=\"rest\",\n",
    ")\n",
    "\n",
    "print(client_prediction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6) Visualise Seldon's Production ML Pipelines\n",
    "We can visualise the performance using the SeldonAnalytics package, which we can deploy using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!helm install seldon-core-analytics --repo https://storage.googleapis.com/seldon-charts --namespace kubeflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In my case, similar to what I did with Ambassador, I need to make sure the service is a LoadBalancer instead of a NodePort"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "service/grafana-prom patched\n"
     ]
    }
   ],
   "source": [
    "!kubectl patch svc grafana-prom --type='json' -p '[{\"op\":\"replace\",\"path\":\"/spec/type\",\"value\":\"LoadBalancer\"}]' -n kubeflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME           TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE\n",
      "grafana-prom   LoadBalancer   10.98.248.223   localhost     80:32445/TCP   64m\n"
     ]
    }
   ],
   "source": [
    "!kubectl get svc grafana-prom -n kubeflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can access it at the port provided, in my case it is http://localhost:32445/d/3swM2iGWz/prediction-analytics?refresh=5s&orgId=1\n",
    "\n",
    "(initial username is admin and password is password, which will be requested to be changed on the first login)\n",
    "\n",
    "Generate a bunch of requests and visualise:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "while True:\n",
    "    client_prediction = sc.predict(\n",
    "        data=batch,\n",
    "        deployment_name=deployment_name,\n",
    "        names=[\"text\"],\n",
    "        payload_type=payload_type,\n",
    "        transport=\"rest\",\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## You now have a full end-to-end training and production NLP pipeline\n",
    "![seldon-analytics](img/seldon-analytics.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}