prednasky/10/10_rozhodovacie_stromy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Rozhodovacie stromy\n",
    "\n",
    "### Jakub Ševcech, Róbert Móro\n",
    "\n",
    "IAU, 28.11.2019"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Spätnú väzbu nám môžete nechať tu: https://tinyurl.com/iau2019-w10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Pýtať sa môžete na http://slido.com#iau2019-w10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Spätná väzba z minulého týždňa\n",
    "\n",
    "Co mam robiť ak ničomu nechápem a mám z tohto predmetu depresie ? :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Opakovacia otázka"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 1. Rozhodovacie stromy\n",
    "\n",
    "# 2. Preučenie\n",
    "\n",
    "# 3. Výber atribútov"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Rozhodovacie stromy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"img/binary_tree.png\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"img/flowchart.jpg\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"img/one_does_not.jpg\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"img/guess-ill-just-explain-it.jpg\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Príklad a časti vysvetlenia prebraté z:\n",
    "* [IAML: Decision Trees - slides](http://www.inf.ed.ac.uk/teaching/courses/iaml/2011/slides/dt.pdf)\n",
    "* [IAML: Decision Trees - video](https://www.youtube.com/playlist?list=PLBv09BD7ez_4temBw7vLA19p3tdQH6FYO)\n",
    "* [Decision tree pruning](https://www.ismll.uni-hildesheim.de/lehre/ml-08w/skript/decision_trees2.pdf)\n",
    "\n",
    "Odporúčam pozrieť si vo voľnom čase"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "Predstavte si, že máte takéto dáta o tom ako niekto (volajme ho Jano) chodí hrať tenis a ku každému riadku máte informácie o počasí. Chcete vytvoriť model, ktorý bude predpovedať, či Jano pôjde hrať na ďalší deň.\n",
    "<img src=\"img/data1.png\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Ako by sa na takýchto dátach snažil predikciu spraviť rozhodovací strom?\n",
    "\n",
    "Tento algoritmus by sa snažil vytvoriť pravidlá, kedy Jano chodí a kedy nechodí hrať. Snažil by sa \"pochopiť\" pravidlám v dátach. Ako?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Rozhodovací strom je greedy, divide & conquer algoritmus.\n",
    "\n",
    "Snaží sa rozdeliť dáta na základe hodnôt zvoleného atribútu tak, aby boli v podmnožinách pozorovania jednotné (ideálne len z jednej triedy).\n",
    "\n",
    "Divide & conquer kvôli rozdeľovaniu na podmnožiny.\n",
    "\n",
    "Greedy kvôli spôsobu výberu atribútu, ktorý použije ako pravidlo.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<img src=\"img/test.jpg\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#  Kostra algoritmu:\n",
    "\n",
    "1. Rozdeľ na podmnožiny\n",
    "2. Sú podmnožiny čisté (obsahujú len pozorovania z jednej triedy)?\n",
    "3. Ak áno: stop\n",
    "4. Ak nie: rekurzívne opakuj\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Skúsime si to vyskúšať na týchto dátach.\n",
    "\n",
    "<img src=\"img/data2.png\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "\n",
    "Vyberme si atribút *Outlook* na základe ktorého to rozdelíme.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "<img src=\"img/split-outlook.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "Rozdelilo sa nám to na 3 skupiny. Jedna je už čista, takže na nej sme už skončili a tie ďalšie dve budeme musieť ešte rekurzívne riešiť.\n",
    "\n",
    "Keď je jasno, tak sa zdá, že keď si vyberieme vlhkosť, tak nám vzniknú čisté podmnožiny.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "<img src=\"img/split-humidity.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "Teraz nám už vznikli čisté podmnožiny, takže tu sme skončili.\n",
    "\n",
    "Teraz ešte pravá vetva. Tu sa dá rozdeliť podľa vetra.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "<img src=\"img/split-wind.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "\n",
    "Všetky podmnožiny sú čisté, takže sme skončili a vznikol nám rozhodovací strom.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Ako teraz budeme klasifikovať nove pozorovanie?\n",
    "<img src=\"img/dt-classif.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Teraz sme si vlastne ukázali priebeh algoritmu ID3 na vytvorenie rozhodovacieho stromu\n",
    "\n",
    "<img src=\"img/id3.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "\n",
    "Tento algoritmus viac menej naraz vymysleli dvaja ľudia. Boli ale z rôznej domény, takže o sebe nevedeli. Stále platí rozdelenie medzi komunitami. ID3 máte implementovaný v Pythne a CART v R.\n",
    "\n",
    "Otázne je, ako vybrať ten správny atribút na rozdelenie\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"img/which_attribute.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Chceme metriku, ktorá nám dá vysokú hodnotu pre čisté podmnožiny a nízku pre nečisté (alebo naopak)\n",
    "\n",
    "Podmienená pravdepodobnosť nie je dobrá, keďže tá nie je symetrická. Má dve extrémne hodnoty, ktoré by nás zaujímali, ale medzi nimi je presne to, čo nechceme. Ak by sme porovnávali dve pravdepodobnosti, tak nevieme jednoducho povedať, ktorá je lepšia.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"img/entropy.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Interpretácia Entropie\n",
    "\n",
    "Množstvo bitov informácie (per pozorovanie), ktoré potrebujeme na to, aby sme zaradili každé pozorovanie do správnej triedy.\n",
    "\n",
    "Ak je to 50/50, tak pre každé pozorovanie potrebujeme 1 bit na to aby sme označili či patrí do triedy alebo nie. Ak to je iný pomer, tak potrebujeme menej, pretože nám stačí označiť pozorovania z menej početnej triedy. Ak sú všetky pozorovania z jednej triedy, tak ich nepotrebujeme označovať vôbec.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Pre Multiclass klasifikáciu je vzorec entropie\n",
    "\n",
    "<img src=\"img/multiclass.PNG\" alt=\"Drawing\" style=\"width: 300px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"img/information_gain.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### information gain = mutual information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Interpretácia Information gain\n",
    "\n",
    "Koľko bitov istoty získame ak použijeme zvolený atribút na rozdelenie?\n",
    "\n",
    "Ide vlastne o zmenu entropie pri použití rozdelenia pomocou zvoleného atribútu. Koľko informácie vlastne dané rozdelenie do dát pridáva.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Veľká výhoda rozhodovacích stromov je priamočiara interpretovateľnosť natrénovaného modelu. Dajú sa z neho priamo vytvoriť pravidlá.\n",
    "\n",
    "<img src=\"img/interpretable.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Čo s dátami, ktoré nie sú kategorické?\n",
    "\n",
    "## Threshold split\n",
    "\n",
    "Usporiadajte hodnoty atribútu a vyskúšajte hodnoty medzi hodnotami atribútu ako potenciálne hraničné hodnoty. \n",
    "\n",
    "<img src=\"img/threshold_split.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "\n",
    "Vyhodnotenie vhodnosti rôznych hraničných bodov pomocou zvolenej metriky (Entropy, Information gain, ...)\n",
    "\n",
    "Stačí pozerať na hraničné body medzi pozorovaniami, kde sa mení trieda.\n",
    "\n",
    "Väčšina knižníc to už má v sebe zabudované a nemusíte sa o to starať.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Preučenie\n",
    "\n",
    "Zastavovacia podmienka vytvárania stromu sú čisté listy. Čo ak nám v liste zostane len veľmi malo pozorovaní. Dajme tomu 1. \n",
    "\n",
    "* Vtedy je list určite čistý, ale bude sa dať taký strom použiť aj na testovacej sade? \n",
    "* Chyba na trénovanej sade bude 0, ale na testovacej bude asi pomerne vysoká. \n",
    "* Pravidlá, ktoré vytvorí budú veľmi zložité a budú veľmi zle zovšeobecniteľné. V podstate sa len naučí celú trénovaciu sadu naspamäť. \n",
    "* Bude pretrénovaný. \n",
    "\n",
    "<img src=\"img/overfitting.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Bias / Variance\n",
    "\n",
    "S pretrénovaním úzko súvisia termíny bias (skreslenie) a viarance (variancia)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "* **Bias** je chyba spôsobená aproximáciou zložitého problému jednoduchším modelom \n",
    "\n",
    "> Niečo nám v tom modely chyba\n",
    "\n",
    "> Robíme nejakú systematickú chybu. \n",
    "\n",
    "* **Variancia** nám hovorí, ako veľmi by sa zmenil model, ak by sme použili inú trénovaciu sadu\n",
    "\n",
    "> Hovorí nám to, aký nestály bude model na dátach, ktoré predtým nevidel\n",
    "\n",
    "![bias-variance](img/bias-variance2.jpg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Pekne sa to vizualizuje na regresii\n",
    "\n",
    "![overfit](img/overfit.png)\n",
    "\n",
    "* Ak máme veľmi veľký bias, tak to znamená, že máme nedotrénovaný model (underfitting) a niečo nám v ňom chýba\n",
    "* Ak máme veľkú varianciu, tak to indikuje, že sme model pretrénovali (overfitting)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Celková chyba je zlozená z variancie a biasu\n",
    "\n",
    "A z náhodného šumu v dátach\n",
    "\n",
    "Tieto dve chyby nevieme od seba oddeliť, ale snažíme sa nájsť bod, kde je ich súčet minimálny\n",
    "\n",
    "![bias-variance](img/bias-variance.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Tie dve chyby nevieme od seba oddeliť, ale ich súčet vieme odhadnúť pomocou chyby na validačnej vzorke\n",
    "\n",
    "Snažíme sa nájsť takú zložitosť modelu, aby sme mali čo najmenšiu chybu na dátach, ktoré model nikdy nevidel\n",
    "\n",
    "![validation error](img/validation-error.png)\n",
    "\n",
    "Toto je vlastne prevrátený obrázok oproti tomu, ktorý som ukazoval pri preučení stromu pretože sme použili metriku merajúcu chybu a nie úspešnosť.\n",
    "\n",
    "\n",
    "zdroj obrázkov:\n",
    "* [http://puremonkey2010.blogspot.sk/2016/09/ml-bias-variance-and-overfitting.html](http://puremonkey2010.blogspot.sk/2016/09/ml-bias-variance-and-overfitting.html),\n",
    "* [https://datascience.stackexchange.com/questions/361/when-is-a-model-underfitted](https://datascience.stackexchange.com/questions/361/when-is-a-model-underfitted) a \n",
    "* [http://scott.fortmann-roe.com/docs/BiasVariance.html](http://scott.fortmann-roe.com/docs/BiasVariance.html)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Čo sa da spraviť na zabránenie pretrénovaniu\n",
    "\n",
    "* Obmedziť rast stromu (maximálna hĺbka, minimálna veľkosť listu, minimálna veľkosť vnútorného uzlu, minimálny prírastok entropie pri rozdelení ...)\n",
    "* Dodatočne orezávanie stromu (odstránenie podstromov a nahradenie predikciou majoritnej triedy v podstrome)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Orezávanie (Pruning)\n",
    "Veľa rôznych algoritmov. Väčšinou vyžadujú validačnú vzorku na overenie či orezanie zlepšilo strom. Pri pretrénovanom strome je chyba na trénovanej sade veľmi malá a každé orezanie by ju zhoršilo. Preto potrebujeme validačnú sadu aby sme zistili či sa orezanie nejak prejaví aj na nej.\n",
    "\n",
    "## Reduced-Error Pruning\n",
    "Najpopulárnejší algoritmus orezávania\n",
    "\n",
    "1. Klasifikuj pozorovania z validačnej vzorky\n",
    "2. Pre každý vnútorný uzol:\n",
    "> a. Spočítaj chyby na pozorovaniach z validačnej vzorky, ktoré spadajú do podstromu. \n",
    "> b. Spočítaj chybu ak by sa podstrom nahradil predikciou majoritnej triedy trenovacích pozorovaní, ktoré spadajú do podstromu.\n",
    "> Rozdiel týchto dvoch chýb je redukcia chyby na podstrome\n",
    "3. Odstráň podstrom s najväčšou redukciou chyby na validačnej vzorke\n",
    "4. Opakuj pokiaľ nastáva nejaká redukcia\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Rule Post-Pruning\n",
    "\n",
    "1) Transformuj strom na pravidlá (jedno pravidlo pre každý prechod od koreňa k listom)\n",
    "\n",
    "> Outlook=sunny ^ humidity=high -> No\n",
    "\n",
    "> Outlook=sunny ^ humidity=normal -> Yes\n",
    "\n",
    "> Outlook=overcast -> Yes\n",
    "\n",
    "> Outlook=rain ^ wind=strong -> No\n",
    "\n",
    "> Outlook=rain ^ wind=weak -> Yes\n",
    "\n",
    "2) Pre každého predchodcu v pravidle: odstráň ho ak jeho odstránením klesne chyba na validačnej vzorke (v rámci jedného pravidla sa zastav ak už chyba neklesá)\n",
    "\n",
    "3) Usporiadaj zostávajúce pravidlá podľa úspešnosti.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Ukážka v Pythone"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = 9, 6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Načítam si všetky dáta. Trénovacie a aj jedno testovacie pozorovanie"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Humidity</th>\n",
       "      <th>Outlook</th>\n",
       "      <th>Play</th>\n",
       "      <th>Wind</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>High</td>\n",
       "      <td>Sunny</td>\n",
       "      <td>No</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>High</td>\n",
       "      <td>Sunny</td>\n",
       "      <td>No</td>\n",
       "      <td>Strong</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>High</td>\n",
       "      <td>Overcast</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>High</td>\n",
       "      <td>Rain</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Normal</td>\n",
       "      <td>Rain</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Normal</td>\n",
       "      <td>Rain</td>\n",
       "      <td>No</td>\n",
       "      <td>Strong</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Normal</td>\n",
       "      <td>Overcast</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Strong</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>High</td>\n",
       "      <td>Sunny</td>\n",
       "      <td>No</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Normal</td>\n",
       "      <td>Sunny</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Normal</td>\n",
       "      <td>Rain</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Normal</td>\n",
       "      <td>Sunny</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Strong</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>High</td>\n",
       "      <td>Overcast</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Strong</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Normal</td>\n",
       "      <td>Overcast</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>High</td>\n",
       "      <td>Rain</td>\n",
       "      <td>No</td>\n",
       "      <td>Strong</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>High</td>\n",
       "      <td>Rain</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Weak</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Humidity   Outlook Play    Wind\n",
       "0      High     Sunny   No    Weak\n",
       "1      High     Sunny   No  Strong\n",
       "2      High  Overcast  Yes    Weak\n",
       "3      High      Rain  Yes    Weak\n",
       "4    Normal      Rain  Yes    Weak\n",
       "5    Normal      Rain   No  Strong\n",
       "6    Normal  Overcast  Yes  Strong\n",
       "7      High     Sunny   No    Weak\n",
       "8    Normal     Sunny  Yes    Weak\n",
       "9    Normal      Rain  Yes    Weak\n",
       "10   Normal     Sunny  Yes  Strong\n",
       "11     High  Overcast  Yes  Strong\n",
       "12   Normal  Overcast  Yes    Weak\n",
       "13     High      Rain   No  Strong\n",
       "14     High      Rain  Yes    Weak"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv('tenis.csv')\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "could not convert string to float: 'High'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-4-12fddc32c13c>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[0mX\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mloc\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mindex\u001b[0m \u001b[1;33m<\u001b[0m \u001b[1;36m14\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;34m'Humidity'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'Outlook'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'Wind'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mloc\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mindex\u001b[0m \u001b[1;33m<\u001b[0m \u001b[1;36m14\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'Play'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 6\u001b[1;33m \u001b[0mcls\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python37\\site-packages\\sklearn\\tree\\tree.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, sample_weight, check_input, X_idx_sorted)\u001b[0m\n\u001b[0;32m    814\u001b[0m             \u001b[0msample_weight\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0msample_weight\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    815\u001b[0m             \u001b[0mcheck_input\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcheck_input\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 816\u001b[1;33m             X_idx_sorted=X_idx_sorted)\n\u001b[0m\u001b[0;32m    817\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    818\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python37\\site-packages\\sklearn\\tree\\tree.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, sample_weight, check_input, X_idx_sorted)\u001b[0m\n\u001b[0;32m    128\u001b[0m         \u001b[0mrandom_state\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcheck_random_state\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrandom_state\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    129\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcheck_input\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 130\u001b[1;33m             \u001b[0mX\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcheck_array\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mDTYPE\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maccept_sparse\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m\"csc\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    131\u001b[0m             \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcheck_array\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mensure_2d\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    132\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0missparse\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python37\\site-packages\\sklearn\\utils\\validation.py\u001b[0m in \u001b[0;36mcheck_array\u001b[1;34m(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)\u001b[0m\n\u001b[0;32m    494\u001b[0m             \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    495\u001b[0m                 \u001b[0mwarnings\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msimplefilter\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'error'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mComplexWarning\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 496\u001b[1;33m                 \u001b[0marray\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0morder\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0morder\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    497\u001b[0m             \u001b[1;32mexcept\u001b[0m \u001b[0mComplexWarning\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    498\u001b[0m                 raise ValueError(\"Complex data not supported\\n\"\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\numpy\\core\\_asarray.py\u001b[0m in \u001b[0;36masarray\u001b[1;34m(a, dtype, order)\u001b[0m\n\u001b[0;32m     83\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     84\u001b[0m     \"\"\"\n\u001b[1;32m---> 85\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0ma\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0morder\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0morder\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     86\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     87\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: could not convert string to float: 'High'"
     ]
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "cls = DecisionTreeClassifier(criterion='entropy')\n",
    "\n",
    "X = data.loc[data.index < 14, ['Humidity', 'Outlook', 'Wind']]\n",
    "y = data.loc[data.index < 14, 'Play']\n",
    "cls.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Scikit-learn vie pracovať len s numerickými hodnotami\n",
    "\n",
    "Potrebujeme teda spraviť kódovanie"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Humidity_High</th>\n",
       "      <th>Humidity_Normal</th>\n",
       "      <th>Wind_Strong</th>\n",
       "      <th>Wind_Weak</th>\n",
       "      <th>Outlook_Overcast</th>\n",
       "      <th>Outlook_Rain</th>\n",
       "      <th>Outlook_Sunny</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Humidity_High  Humidity_Normal  Wind_Strong  Wind_Weak  Outlook_Overcast  \\\n",
       "0               1                0            0          1                 0   \n",
       "1               1                0            1          0                 0   \n",
       "2               1                0            0          1                 1   \n",
       "3               1                0            0          1                 0   \n",
       "4               0                1            0          1                 0   \n",
       "5               0                1            1          0                 0   \n",
       "6               0                1            1          0                 1   \n",
       "7               1                0            0          1                 0   \n",
       "8               0                1            0          1                 0   \n",
       "9               0                1            0          1                 0   \n",
       "10              0                1            1          0                 0   \n",
       "11              1                0            1          0                 1   \n",
       "12              0                1            0          1                 1   \n",
       "13              1                0            1          0                 0   \n",
       "14              1                0            0          1                 0   \n",
       "\n",
       "    Outlook_Rain  Outlook_Sunny  \n",
       "0              0              1  \n",
       "1              0              1  \n",
       "2              0              0  \n",
       "3              1              0  \n",
       "4              1              0  \n",
       "5              1              0  \n",
       "6              0              0  \n",
       "7              0              1  \n",
       "8              0              1  \n",
       "9              1              0  \n",
       "10             0              1  \n",
       "11             0              0  \n",
       "12             0              0  \n",
       "13             1              0  \n",
       "14             1              0  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# One-hot kodovanie vsetkych dat okrem predikovaneho stlpcu\n",
    "encoded = pd.concat([pd.get_dummies(data[column], prefix=column) for column in set(data.columns) - {'Play'}], axis=1)\n",
    "encoded"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Pozor, v predchádzajúcom kóde som trochu podvádzal. Kto vie čo som spravil zle?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "# Použil som testovacie dáta na natrénovanie transformácie. \n",
    "\n",
    "Posledné pozorovanie patrí do testovacích dát. One hot encoder, ktorý som použil sa pozrel na všetky unikátne hodnoty v dátach vrátane testovacích a vytvoril pre každú unikátnu hodnotu nový stĺpec. \n",
    "\n",
    "Čo by sa stalo ak by som v testovacích dátach mal hodnotu, ktorá sa v trenovacích nenachádza?\n",
    "\n",
    "Čo by sa malo stať po správnosti?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Vyberiem si len trénovacie dáta a natrénujem klasifikátor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=None,\n",
       "                       max_features=None, max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, presort=False,\n",
       "                       random_state=None, splitter='best')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = encoded[encoded.index < 14]\n",
    "y = data.loc[data.index < 14, 'Play']\n",
    "\n",
    "cls.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Yes'], dtype=object)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test = encoded[encoded.index == 14]\n",
    "cls.predict(test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Natrénovaný strom si môžem vizualizovať"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"552pt\" viewBox=\"0.00 0.00 565.00 552.00\" width=\"565pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g class=\"graph\" id=\"graph0\" transform=\"scale(1 1) rotate(0) translate(4 548)\">\n",
       "<title>Tree</title>\n",
       "<polygon fill=\"white\" points=\"-4,4 -4,-548 561,-548 561,4 -4,4\" stroke=\"none\"/>\n",
       "<!-- 0 -->\n",
       "<g class=\"node\" id=\"node1\"><title>0</title>\n",
       "<polygon fill=\"#a7d3f3\" points=\"399.5,-544 238.5,-544 238.5,-461 399.5,-461 399.5,-544\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-528.8\">Outlook_Overcast &lt;= 0.5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-513.8\">entropy = 0.94</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-498.8\">samples = 14</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-483.8\">value = [5, 9]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-468.8\">class = yes</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g class=\"node\" id=\"node2\"><title>1</title>\n",
       "<polygon fill=\"#ffffff\" points=\"327,-425 169,-425 169,-342 327,-342 327,-425\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"248\" y=\"-409.8\">Humidity_Normal &lt;= 0.5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"248\" y=\"-394.8\">entropy = 1.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"248\" y=\"-379.8\">samples = 10</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"248\" y=\"-364.8\">value = [5, 5]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"248\" y=\"-349.8\">class = no</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g class=\"edge\" id=\"edge1\"><title>0-&gt;1</title>\n",
       "<path d=\"M294.367,-460.907C289.025,-452.105 283.32,-442.703 277.803,-433.612\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"280.77,-431.754 272.59,-425.021 274.785,-435.386 280.77,-431.754\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"266.648\" y=\"-445.604\">True</text>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g class=\"node\" id=\"node13\"><title>12</title>\n",
       "<polygon fill=\"#399de5\" points=\"437,-417.5 345,-417.5 345,-349.5 437,-349.5 437,-417.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"391\" y=\"-402.3\">entropy = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"391\" y=\"-387.3\">samples = 4</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"391\" y=\"-372.3\">value = [0, 4]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"391\" y=\"-357.3\">class = yes</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;12 -->\n",
       "<g class=\"edge\" id=\"edge12\"><title>0-&gt;12</title>\n",
       "<path d=\"M343.98,-460.907C350.838,-449.763 358.287,-437.658 365.191,-426.439\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"368.329,-428.018 370.59,-417.667 362.368,-424.349 368.329,-428.018\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"376.38\" y=\"-438.287\">False</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g class=\"node\" id=\"node3\"><title>2</title>\n",
       "<polygon fill=\"#eca06a\" points=\"239.5,-306 102.5,-306 102.5,-223 239.5,-223 239.5,-306\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-290.8\">Outlook_Rain &lt;= 0.5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-275.8\">entropy = 0.722</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-260.8\">samples = 5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-245.8\">value = [4, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-230.8\">class = no</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g class=\"edge\" id=\"edge2\"><title>1-&gt;2</title>\n",
       "<path d=\"M221.285,-341.907C215.492,-333.105 209.305,-323.703 203.322,-314.612\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"206.089,-312.45 197.668,-306.021 200.241,-316.298 206.089,-312.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g class=\"node\" id=\"node8\"><title>7</title>\n",
       "<polygon fill=\"#6ab6ec\" points=\"392,-306 258,-306 258,-223 392,-223 392,-306\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"325\" y=\"-290.8\">Wind_Strong &lt;= 0.5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"325\" y=\"-275.8\">entropy = 0.722</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"325\" y=\"-260.8\">samples = 5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"325\" y=\"-245.8\">value = [1, 4]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"325\" y=\"-230.8\">class = yes</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;7 -->\n",
       "<g class=\"edge\" id=\"edge7\"><title>1-&gt;7</title>\n",
       "<path d=\"M274.715,-341.907C280.508,-333.105 286.695,-323.703 292.678,-314.612\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"295.759,-316.298 298.332,-306.021 289.911,-312.45 295.759,-316.298\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g class=\"node\" id=\"node4\"><title>3</title>\n",
       "<polygon fill=\"#e58139\" points=\"92,-179.5 0,-179.5 0,-111.5 92,-111.5 92,-179.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"46\" y=\"-164.3\">entropy = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"46\" y=\"-149.3\">samples = 3</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"46\" y=\"-134.3\">value = [3, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"46\" y=\"-119.3\">class = no</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g class=\"edge\" id=\"edge3\"><title>2-&gt;3</title>\n",
       "<path d=\"M127.632,-222.907C115.136,-211.211 101.51,-198.457 89.0339,-186.78\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"91.1274,-183.945 81.4348,-179.667 86.3439,-189.056 91.1274,-183.945\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g class=\"node\" id=\"node5\"><title>4</title>\n",
       "<polygon fill=\"#ffffff\" points=\"244,-187 110,-187 110,-104 244,-104 244,-187\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"177\" y=\"-171.8\">Wind_Strong &lt;= 0.5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"177\" y=\"-156.8\">entropy = 1.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"177\" y=\"-141.8\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"177\" y=\"-126.8\">value = [1, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"177\" y=\"-111.8\">class = no</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;4 -->\n",
       "<g class=\"edge\" id=\"edge4\"><title>2-&gt;4</title>\n",
       "<path d=\"M173.082,-222.907C173.51,-214.558 173.966,-205.671 174.409,-197.02\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"177.905,-197.187 174.922,-187.021 170.914,-196.828 177.905,-197.187\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g class=\"node\" id=\"node6\"><title>5</title>\n",
       "<polygon fill=\"#399de5\" points=\"164,-68 72,-68 72,-0 164,-0 164,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"118\" y=\"-52.8\">entropy = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"118\" y=\"-37.8\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"118\" y=\"-22.8\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"118\" y=\"-7.8\">class = yes</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;5 -->\n",
       "<g class=\"edge\" id=\"edge5\"><title>4-&gt;5</title>\n",
       "<path d=\"M155.031,-103.726C150.362,-95.0615 145.424,-85.8962 140.727,-77.1802\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"143.767,-75.4428 135.942,-68.2996 137.604,-78.7632 143.767,-75.4428\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g class=\"node\" id=\"node7\"><title>6</title>\n",
       "<polygon fill=\"#e58139\" points=\"274,-68 182,-68 182,-0 274,-0 274,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"228\" y=\"-52.8\">entropy = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"228\" y=\"-37.8\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"228\" y=\"-22.8\">value = [1, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"228\" y=\"-7.8\">class = no</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;6 -->\n",
       "<g class=\"edge\" id=\"edge6\"><title>4-&gt;6</title>\n",
       "<path d=\"M195.991,-103.726C199.984,-95.1527 204.205,-86.0891 208.226,-77.4555\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"211.441,-78.8423 212.491,-68.2996 205.096,-75.8869 211.441,-78.8423\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g class=\"node\" id=\"node9\"><title>8</title>\n",
       "<polygon fill=\"#399de5\" points=\"365,-179.5 273,-179.5 273,-111.5 365,-111.5 365,-179.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-164.3\">entropy = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-149.3\">samples = 3</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-134.3\">value = [0, 3]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"319\" y=\"-119.3\">class = yes</text>\n",
       "</g>\n",
       "<!-- 7&#45;&gt;8 -->\n",
       "<g class=\"edge\" id=\"edge8\"><title>7-&gt;8</title>\n",
       "<path d=\"M322.918,-222.907C322.369,-212.204 321.775,-200.615 321.219,-189.776\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"324.709,-189.475 320.701,-179.667 317.718,-189.833 324.709,-189.475\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g class=\"node\" id=\"node10\"><title>9</title>\n",
       "<polygon fill=\"#ffffff\" points=\"529,-187 383,-187 383,-104 529,-104 529,-187\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"456\" y=\"-171.8\">Outlook_Sunny &lt;= 0.5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"456\" y=\"-156.8\">entropy = 1.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"456\" y=\"-141.8\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"456\" y=\"-126.8\">value = [1, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"456\" y=\"-111.8\">class = no</text>\n",
       "</g>\n",
       "<!-- 7&#45;&gt;9 -->\n",
       "<g class=\"edge\" id=\"edge9\"><title>7-&gt;9</title>\n",
       "<path d=\"M370.45,-222.907C380.916,-213.56 392.137,-203.538 402.896,-193.929\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"405.504,-196.293 410.631,-187.021 400.841,-191.072 405.504,-196.293\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g class=\"node\" id=\"node11\"><title>10</title>\n",
       "<polygon fill=\"#e58139\" points=\"447,-68 355,-68 355,-0 447,-0 447,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"401\" y=\"-52.8\">entropy = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"401\" y=\"-37.8\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"401\" y=\"-22.8\">value = [1, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"401\" y=\"-7.8\">class = no</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;10 -->\n",
       "<g class=\"edge\" id=\"edge10\"><title>9-&gt;10</title>\n",
       "<path d=\"M435.52,-103.726C431.214,-95.1527 426.661,-86.0891 422.325,-77.4555\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"425.342,-75.6647 417.726,-68.2996 419.087,-78.8067 425.342,-75.6647\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g class=\"node\" id=\"node12\"><title>11</title>\n",
       "<polygon fill=\"#399de5\" points=\"557,-68 465,-68 465,-0 557,-0 557,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"511\" y=\"-52.8\">entropy = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"511\" y=\"-37.8\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"511\" y=\"-22.8\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"511\" y=\"-7.8\">class = yes</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;11 -->\n",
       "<g class=\"edge\" id=\"edge11\"><title>9-&gt;11</title>\n",
       "<path d=\"M476.48,-103.726C480.786,-95.1527 485.339,-86.0891 489.675,-77.4555\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"492.913,-78.8067 494.274,-68.2996 486.658,-75.6647 492.913,-78.8067\" stroke=\"black\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>svg{width:70% !important;height:70% !important;}</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.tree import export_graphviz\n",
    "from graphviz import Source\n",
    "from IPython.display import SVG\n",
    "\n",
    "graph = Source(export_graphviz(cls, \n",
    "                               out_file=None,\n",
    "                               feature_names=encoded.columns,\n",
    "                               class_names=['no', 'yes'],\n",
    "                               filled = True))\n",
    "\n",
    "display(SVG(graph.pipe(format='svg')))\n",
    "\n",
    "from IPython.display import HTML # toto je tu len pre to aby sa mi obrazok zmestil na obrazovku\n",
    "style = \"<style>svg{width:70% !important;height:70% !important;}</style>\"\n",
    "HTML(style)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Stromček nevyzerá úplne rovnako ako ten, ktorý sme ukazovali na začiatku. Je to spôsobené one-hot kódovaním. Vieme robiť len binárne rozhodnutia."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Ukážka - Pretrénovanie"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_breast_cancer\n",
    "\n",
    "data = load_breast_cancer()\n",
    "\n",
    "label_names = data['target_names']\n",
    "labels = data['target']\n",
    "feature_names = data['feature_names']\n",
    "features = data['data']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# train data:  455\n",
      "# test data:  114\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Rozdelíme údaje\n",
    "X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size=0.2, random_state=4)\n",
    "\n",
    "print('# train data: ', len(X_train))\n",
    "print('# test data: ', len(X_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Natrénujem si viacero stromov, kde každému obmedzím jeho maximálnu hĺbku aby som vytvoril stromy s rôznou zložitosťou"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "results = []\n",
    "estimators = []\n",
    "for i in range(1, X_train.shape[1] + 1):  \n",
    "    row = {'model_complexity': i}\n",
    "    # Vytvoríme rozhodovací strom\n",
    "    clf = DecisionTreeClassifier(max_depth = i) # strom s maximalnou hlbkou 1-pocet atributov,  simulujeme tak zlozitost modelu\n",
    "    \n",
    "    pred = clf.fit(X_train, y_train).predict(X_train) # natrenovanie modelu a predikovanie na trenovacej sade\n",
    "    row['train'] = 1-accuracy_score(y_train, pred) # chyba na trenovacej sade\n",
    "    \n",
    "    pred = clf.predict(X_test)\n",
    "    row['test'] = 1-accuracy_score(y_test, pred) # chyba na testovacej sade\n",
    "    results.append(row)\n",
    "    estimators.append(clf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>model_complexity</th>\n",
       "      <th>train</th>\n",
       "      <th>test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.065934</td>\n",
       "      <td>0.140351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.052747</td>\n",
       "      <td>0.131579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.021978</td>\n",
       "      <td>0.114035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.017582</td>\n",
       "      <td>0.105263</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.008791</td>\n",
       "      <td>0.096491</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   model_complexity     train      test\n",
       "0                 1  0.065934  0.140351\n",
       "1                 2  0.052747  0.131579\n",
       "2                 3  0.021978  0.114035\n",
       "3                 4  0.017582  0.105263\n",
       "4                 5  0.008791  0.096491"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "complexity_df = pd.DataFrame(results)\n",
    "complexity_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x270617b04a8>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAF3CAYAAACPPbA6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xl8lOW9///XRQKENUASIBAg7BK2IIgoq1oVawVUVNztZntau7vg+XX1nH6P1ra2PdWeamvFXasVcWm1toKoCAQMS9h3QgIEAmHNfv3+uGYgxITMZJZ7ZvJ+Ph4+Jpm5574/tMnkc3+uz3VdxlqLiIiISCxp5XUAIiIiIvUpQREREZGYowRFREREYo4SFBEREYk5SlBEREQk5ihBERERkZijBEVERERijhIUERERiTlKUERERCTmJHsdQH3p6ek2Ozvb6zBEREQkTFasWHHAWpsRzHtiLkHJzs4mLy/P6zBEREQkTIwxO4N9j4Z4REREJOYoQREREZGYowRFREREYo4SFBEREYk5SlBEREQk5ihBERERkZijBEVERERijhIUERERiTlKUERERCTmKEERERGRmKMERURERGKOEhQRERH5rNpaOH7As8srQREREZHPWvUCPDICjh/05PJKUEREROSzdn4E1SehaKUnl1eCIiIiIp9VlH/mY5QpQREREZEzVZ2Ekg3u62IlKCIiIhIL9q4FWwMpqaqgiIiISIzwV01GzYEjhZ7M5lGCIiIiImcqzof2aXDOlae/jzIlKCIiInKmolWQmQuZo33fK0ERERERL1WVQ8l66JUL7bpA1/6qoIiIiIjH9hVAbbWroIBLVIpWRT0MJSgiIiJyWvGn7rGXL0HJzIWyXXCiNKphKEERERGR04ryoV03SO3jvvf3oUR5mEcJioiIiJxWvMolJca47z1qlFWCIiIiIk51Bexff3p4B6B9N+jSTxUUERER8ci+AqitOt0g69crVxUUERER8Yi/StKrXoKSmQuHd0a1UTagBMUYM90Ys9EYs8UYM7eB16cYY1YaY6qNMbMbeL2zMWaPMeb34QhaREREIqAoH1K6uCGduvwJS3H0phs3maAYY5KAR4ErgBzgRmNMTr3DdgF3AM83cpr/AhYFFFHJRji0I6BDRUREJIyK810y4m+Q9cuMwQQFGA9ssdZus9ZWAi8CM+seYK3dYa1dDdTWf7MxZizQA3g3oIhqKuGpL8ChnQEdLiIiImFQXQn71p2etVNX+26Q2jeqjbKBJCi9gd11vi/0PdckY0wr4FfAPU0cd6cxJs8Yk3eoVRpUHHVJyuFdgVxGREREQrV/XcMNsn69Rke1UTaQBMU08JwN8PzfAN621u4+20HW2setteOsteO6du8Ft82HijJ46ko4fNa3ioiISDg01iDrl5kLh7bDycNRCSeQBKUQ6FPn+yygKMDzXwDcZYzZAfwSuM0Y82CT7+o1Bm6dDyd9SUpZYYCXExERkWYpyoeUVLc5YEOi3CgbSIKyHBhsjOlvjGkDzAEWBHJya+3N1tq+1tps4G7gaWvtZ2YBNaj3uXDba3DykJIUERGRSCvOP3MF2fpONcpGZ5inyQTFWlsN3AW8A6wHXrbWFhhjHjDGzAAwxpxnjCkErgP+aIwpCEt0vce6SsqJUteTUrYnLKcVERGROqor3SJtjfWfAHRIh85ZMVVBwVr7trV2iLV2oLX2577nfmytXeD7erm1Nsta28Fam2atHd7AOZ6y1t4VdIRZY+HW1+D4AZj3BTgS6OiSiIiIBKRkvZtF29AMnrqiuKJsfKwkmzUObv0bHCtxlZQjxV5HJCIikjj8VZFeY85+XGYulG6F8rKIhxQfCQpAn/Fwy6twbJ+vkqIkRUREJCyK8qFt58YbZP1ONcqujnhI8ZOgAPQ93yUpR4ph3lVwdK/XEYmIiMQ/f4NsqybSgig2ysZXggLQdwLc8orrRZl3FRzd53VEIiIi8aumCvaubbr/BKBjBnTuHZU+lPhLUAD6XQg3/9VNPZ53FRzb73VEIiIi8alkA9RUNN1/4pc5OiozeeIzQQHInuhLUnb7kpQSryMSERGJP/5qSCAVFHDDPAe3uG1pIih+ExSA7Elw08tuY0ElKSIiIsErXgVtOkG3gYEd3ysXsBFvlI3vBAWg/2S46SU4tAOenuHWSxEREZHAFOdD5qimG2T9otQoG/8JCsCAqXDTi1C6DebNgOMHvY5IREQk9tVU+xpkz7KCbH2dekCnzIg3yiZGggIwYBrc+KJbQObpGW55fBEREWncgY1QfbLxHYwbk5kb8UbZxElQAAZeBHOehwOb4B/3ex2NiIhIbDvVIBtsgjLa/a2tOBb+mHwSK0EBGHQJjL8T1rwMB7d6HY2IiEjsKs6H1h0gLcAGWT9/o+zeNREJCxIxQQG48NuQ1AY++KXXkYiIiMSu4lW+Btmk4N4XhUbZxExQOvWAcV+C1S+5xlkRERE5U22Nq4AEO7wD0DkTOvaIaKNsYiYoABO/A0mt4YNfeR2JiIhI7DmwCapOBN8g65eZqwpKs3TqCWPvgFUvQOl2r6MRERGJLc1tkPXrleuSnMrj4YupjsRNUAAmfhdaJcNiVVFERETOUJwPrdtD+uDmvT9zNNhat45KBCR2gtI583QV5dBOr6MRERGJHUX50HNk8A2yfhFulE3sBAVg0nfBtFIVRURExK+2Bvaubv7wDkDnXtAhI2KNsomfoHTuBefeDvnPqYoiIiICbjfiUBpkAYyJaKNs4icoAJO+56ooH/7a60hERES8F2qDrF+vXCjZAJUnQo+pnpaRoKT2hjG3wqfPweHdXkcjIiLireJ8SG4H6UNCO09mrmuU3VcQnrjqaBkJCrgqCqiKIiIi4m+QTUoO7TyZo91jBIZ5Wk6C0qUPjLkFVj4DZYVeRyMiIuKN2lpfg+zo0M+VmgXt0yLSKNtyEhSAyd93jx8+4m0cIiIiXjm4BSqPhdYg6xfBRtmWlaB06QtjboaVT0PZHq+jERERib7iVe4x1AZZv165sH89VJ0Mz/l8WlaCAjDp+66h56PfeB2JiIhI9BXnQ3IKZJwTnvNl5oKtCXujbMtLULr2g9ybYMU8OFLkdTTSXP/+b3jze1BT7W0cJZvg6ZkRW+pZYoC18MqXIO8vXkciLUHlcZg3Aza/F7lrFOVDjxGhN8j69YrMirItL0EBmPwDl+19qCpK3Fr1IuQ9CfO/7lZE9MKBzTDvC7BtodtOQRLTxr/D2lfhvZ9AeZnX0Uiiy3sSti+Cd/4zMp9ttbVuiCcc/Sd+qX2gXdewN8q2zASlazaMngMrnoIjxV5HI8GqOApluyFjGKz5K7zmQZJyYAs89QV33fShsOPD6F5fosNaWPQgtE93ycnSx72OSBJZ5Qn46Ldu+fgDG2Hd/PBfo3QbVB4Nzwwevwg1ygaUoBhjphtjNhpjthhj5jbw+hRjzEpjTLUxZnad53ONMUuMMQXGmNXGmBvCGXxIJt8NtdXuh0HiS8km93jxD+HiH8Gal2H+N6KXpBzc6iontdVwx5swfJabsqe768Sz6R13t/m5n8KQ6bDk91B+xOuoJFGt+AscL4HrnnI3PosedhWPcCoO0wqy9Z1qlC0P2ymbTFCMMUnAo8AVQA5wozEmp95hu4A7gOfrPX8CuM1aOxyYDvzGGNMl1KDDolt/XxXlL3B0r9fRSDBKNrjHjHNgyt0uUVn9Irz+zcgnKQe3uspJTSXc/gZ0Hwb9JrrG612fRPbaEl3+6kmXfu6zYup9UH4YlqmKIhFQddLdMGdPhuxJMPVeKFkP618P73WK8yGprfvsCqfMXHfTtj98jbKBVFDGA1ustdustZXAi8DMugdYa3dYa1cDtfWe32St3ez7ugjYD2SEJfJwmPwDqKmCj37ndSQSjJINkNTGDdUBTLkHLvr/XB/Igm+F/47Dr3QbzLsKqstdctLDl6dnnefi2bE4MtcVb2z+JxR96j4nklpD73Nh8GWuilJx1OvoJNGseAqO7YNpvkGK4Ve7ZegX/SK8n2lF+dBjuPuZDid/T0sY+1ACSVB6A3U3sCn0PRcUY8x4oA2wtYHX7jTG5Blj8kpKSoI9dfOlDYRR17umpGP7o3ddCU3JBveLW7cDfeq9MO1+t2v1GxFIUkq3w1NXubuc2xe4X3C/Nu2h91jY8VF4ryne8VdPUvvC6BtPPz91Lpw8BMue8C42STxV5W7SRr9JrnoC0CrJ3XztXwcb3gjPdayF4tXhbZD169IPUrqcXmMlDAJJUEwDz9lgLmKMyQSeAb5orf3MXw5r7ePW2nHW2nEZGVEusEy5B2oq1IsST0o2QMbQzz4/ba4rw3/6LLz5nfAlKYd2uMpJ1XG47XW3f0V92ZNc6VT9CYlhy79gzwq3+nRym9PPZ42FQZ+Dj/8XKo55F58klpXz4NhemHbfmc+PuBbSBoWvilK6DSrKwt9/Ar5G2dFhbZQNJEEpBPrU+T4LCHgBEWNMZ+At4IfW2tgbpE8bCCOvh+V/hmNRrN5I81Qeh8O7Gl9gaNr9Lulc+TS8+d3Qf6kP7XSVk4qjLjnJHNXwcf4+lN1LQ7ueeO9U9aQP5N782denzoWTpbD8T9GPTRJPVbnbfqXfRNd/UlerJJhyL+xbCxvfCv1apxpkwziDp65eubBvHVRXhOV0gSQoy4HBxpj+xpg2wBxgQSAn9x3/GvC0tfavzQ8zwvxVlI/VixLzSja6x8YSFGNcP8rku91dyVvfa36ScniXa4itOOJLTs7yS91nPLRqrT6URLD131C4/LPVE78+58HAS1wVpfJ49OOTxPLpM3C02FV/TQMDFiOuhW4DYdFDLnkORVG+65frXn+eS5hk5kJtlRuWCoMmExRrbTVwF/AOsB542VpbYIx5wBgzA8AYc54xphC4DvijMcbfxns9MAW4wxiT7/svArWlEKUPghGz3R2RqiixrakEBdwv+cU/dNsarHgK3v5B8L/Yh3fDU1e6cuht85ses23TQX0oicBa94egcxbk3tL4cdPmwokDrvIq0lzVFa560vcC6D+l4WOSkt1N9N41sPHt0K5XnO+Sk4YS73AIc6NsQOugWGvfttYOsdYOtNb+3Pfcj621C3xfL7fWZllrO1hr03zTirHWPmutbW2tza3zX/i3PAyHKfe4Bsgl/+t1JHI2JetdpaLbgLMfZwxc8mOY+F3XBP323YEnKWWFLjk5WQa3zodeYwJ7X/ZEN+tDMzzi17aFbphu8vfO/iHeZzwMuMhVXStPRC08STCfPgNH9jRePfEbeZ37zFv4YPOrKNaGfwXZ+rr2h5TUsPWhtMyVZBuSMcSV0pb9CY4f9DoaaUzJRkgfHNgeEsa4BbYu/Larjv393qZ/ucv2+JKTQ3Dba25qaaCyJ7ktFNSHEp9OVU96w5hbmz5+2ly3qFbek5GPTRJPdQUsfgT6nA8Dpp392KRkN2y9dzVs+kfzrndoh1tMMhINsn6nGmXDM5NHCUpdU++FqhOqosSyxmbwNMYYuPQBuOAut8DWP+Y2nqQcKXLJyYlSuPU1N2QTjD7nQ6tkLXsfr7Yvgl1LYNL3ILlt08f3nQD9p7oZgKqiSLDyn4MjhU1XT/xG3eDWfmpuFcVf1YhkBQVcgrKvAKorQz6VEpS6Moa6xXGWPeH+SElsqTzhZtVkBLkCojFw2X/DhG/C0v+Df9z/2V9wf3Jy/ADc8jfIGhd8fG06QK9z1YcSj6yFhQ9Bp8zAqid+0+bC8f1uRWqRQFVXwuJfu0UeB14c2Hv8VZTifLcFQ7CK8t3weKQaZP0yc91K2yXrQz6VEpT6pt7rOvOX/N7rSKS+A5sAG1wFxc8YuPznMOEbsPQPbqdQf5JypNjN1jlWArf+zc3SaK7siVC0UrM74s2OxbDrY1c9aZ0S+Pv6Xeimhn70W9fDJhKI/OfchqdT5wZWPfEbPcctiLaoGVWU4ny3vH0g1cFQ+Hv2wtAoqwSlvu7DIGem27VUVZTYEsgMnrMxBi7/f3D+1+GTx+DdH7p9mOZ9wS0xfcurrvkxFNmT3H4U6kOJLwsfgo494dzbg3/vtLnu52fFU2EPSxKQv3rSeywMuiS49ya1dlsvFH3qtmIIlLUuYYj08A64Rtm2ncPSKKsEpSFT73PbUS951OtIpK6S9a7HI21g889hDEx/EMbf6apkj453FZRbXoW+54ceY5/zwSSpDyWe7PgQdn4Ik74bXPXEL3uSW6L8w9+EdSdXSVCrXoCyXcFXT/xG3+i2YAiminJ4l9voMpINsn6tWrk+FFVQIqRHjpvR8+EjUPCa19GIX8lGt+xzqJtcGQNX/MJVUixwyyuu4TEc2nZyJU71ocSPhQ9Cxx4w9o7mn2PafW6p8pXzwhaWJKCaKlj8S/cZMfjS5p0juY1bRHDPCrclQyCi1SDr52+UrakK6TRKUBpz1W9dA9MrX4aC+V5HIxD8DJ6zMQaueAju3eb6CMIpe6L78FAfSuzb+bHrP5n4HWjdrvnnyZ4MfS90NzWqokhjVr3oqhnNrZ745d7stmIItIpSlO+qz92HN31sOGTmutXZSzaEdBolKI1p28ndWWeNg1e+BOte9zqilq3qpNtRONgZPE0JZD2VYGVPdss9714W/nNLeC18EDp0h7FfDO08xrgqytFit/iWSH3+6klmLgy5PLRzJbdxDd2Fy93WDE0pznefnc0ZwmyOMK0oqwTlbNp2gptfcc1Mr3wJ1odpy2sJ3oHNNHsGT7T5+1B2apgnpu36xK19MvE70KZ96OfrP9UtWf7hI2HbLE0SyOqX3WJp00KsnviNucVtydDUHj2nGmQjtEFgQ7oNhDadQm6UVYLSlJTOroGy1xj46x2wIQw7Skrw/KXC7mGuoERCSmc3BqtG2di28EHokAHjvhSe8xnjGuyP7FEVRc5UUw0fPOw+F4ZMD885k9u6LRl2L3VbNDSmbLfbfTsaDbJ+rVq5nd9VQYkCf5KSORpevh02hLhhkwSvZIOrSnQLYQZPNGVP8vWhaIXRmLR7GWx7322DEI7qid+Aaa6CtlhVFKljzV/h0PbAV40N1Jhb3dYMZ6ui+JedD3RPsXDJzIV9a11y1kxKUAKVkupWGO05El6+DTY2cz8EaZ6SjW56caR24Qy37EluNcXC5V5HIg1Z+CC0T4fzvhze856qohS6xbhE/NWTniNh6OfDe+7ktq4XZdcS2P5Bw8cU5bubux5RapD165UL1eVwYGOzT6EEJRjturg9WnqOgJdvbd5yw9I8+9c3f4E2L/SdAKaV+lBi0e7lsPVfcOG33PYE4TbwYjcDcPGvw7IficS5ta9C6dbwV0/8xtzqtmhY9FDDr/tXkA1lllpzZPp6XkIY5lGCEix/ktI9B166BTa963VEia+q3JVH4ylBSUmFnqPUhxKLFj0I7dPgvK9E5vzGuGmkZbth1fORuYbEh9oaVz3pMQKGXhmZa7ROcVWUnR/B9sVnvuZvkI1m/4lf2iBo3SGkRlklKM3RrivcNt9lpS/dDJvf8zqixHZwC9ja+JjBU1f2JCjM07oYsaRwBWx5z+1u3bZj5K4z6BI3+2/xr1RFacnW/g0ObnZ7vLWK4J/bc293WzXUr6Ic2QMnDpyuZkRTq6SQG2WVoDRXu65w63z3R/PFm9yHnkRGPM3gqSt7klusSH0osWPRg+53d/xXI3sdfxXl8C63tLm0PLU1LmHongPnXBXZa7VOcVs17Fh8ZtW2KMoryNaXmQt71zS7UVYJSijad4PbFkDGEHjhpsCXHZbglGxw/Rxpg7yOJDh9LwCM+lBixZ4VsPldX/WkU+SvN/hSN3Ni8S9DXvJb4lDBa9GpnviNvcNt2bDwwdPPFee7z84eIyJ//Yb0yoXqk+5/h2ZQghIqf5KSPsRVUra+73VEiadkA3QbEPltwsOtXRfXua8+lNiw6BeQ0sVtFBkNZ1RRXozONSU21Na4n7eMYTBsZnSu2bqdW3Rwx2K3hQO4KcYZ54R3Kn0wMkNbUVYJSji07wa3ve7W6HhhztkXzZHg7d8QXw2ydWVPdkM86kPxVtGnsOkfrnqS0jl61x1yufuQVhWlZVk3302vnXpPdKonfmO/6LZuWPigtw2yfumDoXX7ZjfKKkEJlw5pcPsCd6f//BzYtsjriBJDdQWUbovjBGWSWwtgzwqvI2nZFv3Czaw6P0rVEz//uiiHdrilziXx1dbCoochfSjkzIrutdu0h4nfdls4FLwGx/d7138CrlG250hVUGJCh3Q33NM1G56/4bNTviR4B7eArYm/Blm/fr4+FA3zeKd4FWx8GyZ80yUp0Tb0Cjfl/IOHQ1pVU+LE+tehZL2v9yQp+tcf9yW3COGb33PfezGDp67MXNi7ullvVYISbh0z4PY3oGs/eP56/WEKlX8GT7xNMfZr19Ut7LdTPweeWfQLaJsK53/Nm+sb4zaIO7TdLXkuiau21v28pQ+B4Vd7E0ObDq4Xpfywa5DtOdKbOPx65UJV87b8iMBe83IqSXnqC/DcdW5H5OyJXkcVn0o2+mbwDPY6kubrNwlW/MUNV8Vbo29zHD8AL93qVlSdeo+3sezfABvehGn3u6Zlrwz9vPtDMf/rsOCu0M7VugN86e/RX7o8EvasgHkz3DBoIrDWVXyv+ZM31RO/874MH/3WVfUjsVpyMELogVGCEikdu7skZZ4vSbnlVV+5X4Kyfz107e/m+cer7Emw9A+wZ2Xi/wwcP+j+4OwvcFMLJ3/f2w/qta+4BDdcOxY3lzEw6/+g4G+hn2vZn2Dh/8ANz4Z+Lq9t+TdUHnMroZoEKei36wYjrvE2hjYdfD8fjWwgGE3pQ9zwKv8T9FuVoERSpx51KimzXZLSd4LXUcWXko3x2yDr1+9C97jjw8ROUE6UwtMz3L4j4++EZY+7NWD6T/EmHmuhYD70m+huGLzWc4T7L1Stkt0CYHvXhud8XirOd7MfP/dTryNJPLHyWZOUDNP/H81JUBIkZY1hnXrCHW+6x2evhV1LvY4oflRXuj928dp/4te+m1soKZH7UE6UusrJgc1w4wvuD05yO1j3uncx7V/vqjg5UVqHIlom/Ae07dz45nDxpHiVt7NMJKYpQYmGTj3h9jfdKn/PXgu7l3kdUXwo3Qq11fE7g6eufhNdcpqI+7L4KycHNsGNz7vekzYdYMhlsG6BW7TKC+vmAwaGzfDm+pHSrqtr+F2/APYVeB1N8x0/6DZU9HKdDolpSlCipXOmq6R0zIBnrnFbvsvZxfsMnrqyJ7kln4s+9TqS8DpRCk/PhJJNMOd5GPS506/lzHLrMOxa4k1s/uGdTj28uX4kTfgGtOnkZozEq2Lf74LX02AlZilBiabOvVwlpUM6PHuN21lVGleyETDxPYPHr59vFteOBFob5+QheOZql0jOeQ4Gf+7M1wdf5oZ5CuZHP7b9691KnsOjvFBWtLTv5hadW/e6+7fGI//iXUpQpBFKUKIttberpLTv5j7ctcJo4/avd4veebWPRDh1SHO7mibKxoEnD7uf3/3r4Ibn3MZ49bXt6JKW9Qvc+hDRtO513PBOhHeR9dIFd7mhtHitohTnuxl6Xk7/lpgWUIJijJlujNlojNlijJnbwOtTjDErjTHVxpjZ9V673Riz2fff7eEKPK6lZrlKSrsu8PTVbvqpfFYizOCpy9+HEu97spSXueRk71q4/hnXa9KYnFlwbB/s/iR68YGr2vS9wPV/Jar23WD8V92S5vs3eB1N8IrUICtn12SCYoxJAh4FrgBygBuNMTn1DtsF3AE8X++93YCfAOcD44GfGGO6hh52AujSx1VS2qXCM7OavVdBwqqpcsvcd0+gBCV7ElQdj+//r08lJ2vghmdg6PSzHz/kckhOie4wT8lGt9R4og7v1HXBt9xmbB887HUkwTlRCmW71CArZxVIBWU8sMVau81aWwm8CJwxb89au8NauxqoX8e9HPintbbUWnsI+CfQxCdaC9KlL9zxltsf5OmZ8f2HK9xKt0FtVeJVUCB++1DKj7gG7+JVcP08t8dMU9p2co2z0Rzm8U9tTuThHb8OaTD+K7D2VdeoHC/8u9uqgiJnEUiC0hvYXef7Qt9zgQjovcaYO40xecaYvJKSkgBPnSC69HXDPW07uSSleJXXEcWGRJrB49cxwyVc8bg/U/kRN0W+OB+umwfnXBn4e3NmwdFiKIzS9PqC+dBngmtKbwku/Da0bhdfVRQ1yEoAAklQTAPPBbp+bkDvtdY+bq0dZ60dl5GREeCpE0jXfm64p01Hl6TsXeN1RN7bvwEwbsvyRJI9CXbHWR9KxVG3EnLRSrjuKRj2heDeP3Q6JLWNzjDPgc1umf2WMLzj1yHd7b2y9hU4sMXraAJTnA9d+rk1XUQaEUiCUgj0qfN9FlAU4PlDeW/L0jXbJSmt27sVOfeu9Toib5VscNWlRJjBU1e/iW7vkXiplFUchWdnQ2EezH6yecMm/mGeda9HfpjHnwQl2uJsTbnwOy4JjJcqSlG+hnekSYEkKMuBwcaY/saYNsAcYEGA538HuMwY09XXHHuZ7zlpSLf+LklJTnErc8bzKpGhSrQZPH7Zk9xjPAzzVByD566HwuUw+8+hLRmfMxOOFsGevPDF15B1r0PWeDedvyXpmOGqKGtejv0qyolSOLxTDbLSpCYTFGttNXAXLrFYD7xsrS0wxjxgjJkBYIw5zxhTCFwH/NEYU+B7bynwX7gkZznwgO85aUy3AS5JSWoL866Cfeu8jij6aqrdHiqJNIPHr2N3t7tnrCcoFcfg+evdcNS1f4LhV4d2vqHTIalNZId5Dm6FfWta1vBOXRN9VZTFv/Q6krPbu9o9qoIiTQhoHRRr7dvW2iHW2oHW2p/7nvuxtXaB7+vl1tosa20Ha22atXZ4nfc+aa0d5PvvL5H5ZySYtIG+JKWNS1LidaXI5jq0HWoqE7OCAq6KsusTl4jFosrj8PwNbon6a58Iz9bxKakw8JLIDvMUvOYeE21zwEB17A7jvgSrX3bJWqw61SCrBEXOLtnrAKQRaQPd7J6nrnRJyug5NNxzHGUdMtwKlq0iuAhxIs7gqavfRMh7Evaugt5jvY576iP3AAAgAElEQVTmTNWVvuTkY7jmCRhxbfjOPXwWbPq7a7bNGhe+8/qtex2yznMLIbZUE78DeX+Gxb+CWY95HU3DivNdf1n7bl5HIjFOCUosSx/kKikv3AjL/uR1NGBroabC/VHNnhi56/hXxUy0GTx+p/pQPoq9BOXTp906LbP+ACNnN318MIZMh1atXaUj3AlK6TY3dHDZf4f3vPGmUw8Y+0VY9jhMudsNGceaonxNL5aAKEGJdemD4VsRbiwMVMUxeHigu1ONZIJSsgFS+7q9XBJRp56QNsj1oUz8ttfRnFZdAYsfgT7nw+gbw3/+dl1g4MWwboFLJEwYK4L+3paWOrxT18TvuArd4l/BzEe9juZMJw+7Idwxt3gdicQBbRYogWvbMTqrgpZsTMwG2bqyJ7kej9oaryM57dNn4UghTL0vvMlDXcNnuSXOw73/1Lr5rhrVpW94zxuPOmfC2Dtg1YtwaIfX0ZzJP71eDbISACUoEhz/qqC7l0bm/LU1cGBT4vaf+GVPhoojp2c0eK26Ej58xPVwDLw4ctcZeoUb5lkXxtk8pdvdHz5VT06b9F0wrVwVJZb4E5TMMd7GIXFBCYoEx78qqH+/k3A7tMP1uSTqDB6/U/vyfORtHH75z0HZbpg6N3LVE3Arhw6Y5hIUG+iC1E3w/ywqQTmtcy8493bIfx4O7fQ6mtOK8yG1j9tDSKQJSlAkOJFeFdQ/pTpjWPjPHUs6Z0K3gbGxHkp1JSz+tRsiGXRJ5K83fBYc3gVFn4bnfOvmQ68xbjVmOW3S91wV5cNfex3JaWqQlSAoQZHg+VcFLVwe/nOfmmI8JPznjjXZE910Xq/7UFa94PpCIl098Rv6eWiVHJ5hnkM7XKKT00IXZzub1N5w7m3w6XNweHfTx0daeRmUbtX6JxIwJSgSPP+qoOHsI/Ar2Qids1ylJtFlT3Yf2vs83HeppsqtPNprDAy+NDrXbN8N+k91VbhQh3nW+Xbd0PBOwyZ9zz3GQhWlWCvISnCUoEjwIrkqaMn6xJ/B43eqD8XDYZ5VL7rhlmhVT/yGz3LVj1A3TVw33w0ZdOsflrASTmoWnHsrrHzG+ypKsVaQleAoQZHmGT4LjuyBPSvCd87aGjiwOfEbZP1Se0PX/t41yvqrJ5m5MOTy6F77nC+ASQqtCnd4l/v50/DO2U36vnv88BFv4yjKh8693caGIgFQgiLN418VNJzDPId3QnV54k8xrit7Iuz8KLLryjRm9cuuihHJdU8a074bDAhxmEfDO4Hp0gfG3AyfPgNle7yLo3iVqicSFCUo0jynVgUNQx+Bn3+J+0SfwVNX9mQoPwz7C6J73Zpq+OBh6DnKrU3ihZyZviXq1zTv/evmQ8+Rbt8qObtJ33dbVXhVRak4Cge3qP9EgqIERZpv+Cy3dka4VgVtSTN4/LzqQ1nzV7fk+LQo957Udc5VzR/mKSt0s8g0vBOYrv0g9yZYOQ+OFEX/+sWrAasKigRFCYo036lVQV8Lz/lKNrox6pTU8JwvHnTpA136RTdBqamGD37hqg9DPx+969bXIQ36T3b76ARbhTu1OJsSlIBN/oGvivKb6F/7VIOs1kCRwClBkebzrwpaEKZhnpL1Lav/xC97cnT7UNa+4oZWvOg9qS9nplsbY1+QQ1zrXoceI9yO3xKYrtkweg6seAqOFEf32kX50CnT7bYsEiAlKBIa/+Zvoa4KWlsLJZtazgyeurInwslDLkGLtNoa13vSYwQMvTLy12vKOVe51U6DGeYp2+P2glL1JHiT74baavjot9G9bnG+hnckaEpQJDThWhX08E6oPtkyE5Ro9qGsfdU1K069F1rFwK9/xwy3s3MwwzzrfbN3hitBCVq3/r4qyl/g6N7oXLPimFs+QA2yEqQY+ISSuOZfFbQ5fQR1lWx0jy0xQenaD7r0hZVPu5VlI6W2Bhb9ArrnuMpFrMiZBQc3n96HqSnrXofuwyF9cGTjSlSTf+DWwFn6x+hcb+8a1CArzaEERUI3fJargISyKuipGTwtsAcF4IqHXZL2zDWRS1IKXnOJQKxUT/yGBTHMc6QYdn2itU9CkTbQ9Y6tfTV8SwScjb9BVhUUCVIMfUpJ3ArHqqAlG1wTXbsu4YsrngydDtfPcx/mz14L5UfCe35/9SRjGAyLsT/uHbu7Ya6CAH5+1i8ArIZ3QnXqpiI/8tcqyoeOPaBTz8hfSxKKEhQJXftu0H9KaMM8JRtabvXE75wr4bqnXMNxuJOUdfPhwEaYek9sVU/8cma6+Joa5imY75Kslv6zEqqhV/puKl6P/LXUICvNFIOfVBKXhs9yC3/tXR38e1vyDJ76hl0Fs/8CRSvhudluBc5Q1dbCoochfWjsznwZNgMwZ/+DeXQv7Fqi6kk4dEgL/aYiEJXH4cAmDe9IsyhBkfA4tSpoM+7IynZD1XElKH45M2D2k1CYB8+GIUlZ/7qbwjz1XmiVFJ4Yw61TD+h34dmHeda/AVj1n4RLKDcVgdq71i0OpwqKNIMSFAmPUFYFbckzeBqTMxNm/9kt5/7c9W6qZnPU1rrek/QhMPzq8MYYbjmzXCLl/3mor2C+qwJ1b0F7NUXSqd6xCA7zqEFWQqAERcLn1Kqga4N7X0ufwdOY4VfDtX9yi5I9f70rlwdrwxuwfx1MieHqid+wqwDTcBXl6D632q6Gd8KnQ3rwa9AEqygfOnR3DfAiQVKCIuFzalXQIO/ISja4Lv/23SITVzwbcQ1c+4TrvXguyCSlthYWPgRpg915Yl3nTOg7oeGfnw0a3omI4bOad1MRqOJ8t/+O11sqSFxSgiLh05xVQUEzeJoy4lq45gnY9TE8fwNUngjsfRvehP0FMOWe2K+e+OXMcjEf2Hzm8wXzXaLVPcebuBKV/6YikCnewao84X63NbwjzZTsdQCSYHJmwls/cMMKPYY3fby1rucg9+bIxxbPRs52/1u9die8cAPc+BK0ad/48f7ek24DXYITL3JmwD/uc38wp97jnjtW4oZ3Jv9Ad+Lh1jHDrUGzbj5c/MPw/u+7Tw2yflVVVRQWFlJeXu51KBGXkpJCVlYWrVu3DvlcSlAkvIbNgLfv8e02G0CCUlYIlcdUQQnEqOvcB/5rX4MX5sBNL0Hrdg0fu/Ft2LcGZv0fJMXRr3nnXtDHN8zjT1A2vOH+3bE6RTreDZ8V3E1FoIrUIOtXWFhIp06dyM7OxiRwkm2t5eDBgxQWFtK/f/+Qz6chHgmvYFYFBc3gCdboG2DWH2D7B/DCjVB18rPHWAuLHoJuA2DkddGPMVQ5M11ydXCr+75gvqsEhfOPp5w2bEZkhnmKV0H7dOjcO7znjUPl5eWkpaUldHICYIwhLS0tbJWigBIUY8x0Y8xGY8wWY8zcBl5va4x5yff6UmNMtu/51saYecaYNcaY9caY+8MStcS2QFcFBTetFDR1NBi5N8Ksx2DbQnjxps8mKRv/7ta2mHx3fFVP/PyNsAWvwfEDsGOxu8tP8A93z/hvKsI93bg431VP9P8bQMInJ37h/Hc2maAYY5KAR4ErgBzgRmNM/U61LwOHrLWDgEeAh3zPXwe0tdaOBMYCX/MnL5LA/KuCBnJHVrIBOmRoBk+wcm+CmY/C1vfhxZuhynfHYi0sehC6ZsOoGzwNsdlSe0PWeNcXsV7DO1ERzE1FIKpOunOp/0RCEEgFZTywxVq7zVpbCbwI1J/rNxOY5/v6FeAS49IoC3QwxiQD7YBKIMy7oEnM8a8KGsgdWclGDe8015ibYcb/wtZ/wUu+JGXTO660Hq/VE7/hs2DvGvjkMTdU1XOk1xEltmBuKgKxrwBsjZtiLJ47fPgwjz32WNDv+/znP8/hw4cjEFFgAklQegO763xf6HuuwWOstdVAGZCGS1aOA8XALuCX1trS+hcwxtxpjMkzxuSVlJQE/Y+QGNTUqqBwegaPEpTmO/dWuOp3sOU9ePlWWPg/0KUfjJ7jdWShGTbDPR7Y5O7uW0h53DOnbirClKAUfeoe1SAbExpLUGpqas76vrfffpsuXbzbYT6QBKWhT4b6i1w0dsx4oAboBfQHfmCMGfCZA6193Fo7zlo7LiMjI4CQJOadbVVQvyNFUHFEM3hCNfZ2uOq3sPldN+4/+QeQFPoUP0916QO9x7mvNbwTHTkz3ZDr/g2hn6s4H9p1g9Q+oZ9LQjZ37ly2bt1Kbm4u5513HhdddBE33XQTI0e6yuSsWbMYO3Ysw4cP5/HHHz/1vuzsbA4cOMCOHTsYNmwYX/3qVxk+fDiXXXYZJ0820KAfZoHUgAuBuj9lWUBRI8cU+oZzUoFS4CbgH9baKmC/MeYjYBywLdTAJcbVXRV02n0NH+Nf4l4NsqEbewe0au2SlNE3eh1NeEz8jutB0TBBdAybAX+/z/3Odg+xqlm8Sg2yjfjZGwWsKwpvp0NOr8785KrGZ7k9+OCDrF27lvz8fBYuXMiVV17J2rVrT00FfvLJJ+nWrRsnT57kvPPO49prryUtLe2Mc2zevJkXXniBJ554guuvv55XX32VW265Jaz/jvoCqaAsBwYbY/obY9oAc4AF9Y5ZANzu+3o28G9rrcUN61xsnA7ABCAM6bnEhcZWBfU7tQePhnjCYszNcP08SG7jdSThkTPDLfOvP3LRceqmIsRhnqpyNcjGuPHjx5+xTsnvfvc7Ro8ezYQJE9i9ezebN3/2M7t///7k5rr/T8eOHcuOHTsiHmeTFRRrbbUx5i7gHSAJeNJaW2CMeQDIs9YuAP4MPGOM2YKrnPgHwB8F/gKsxQ0D/cVaG8G9vSWmNLQqaF0lG6B9mtu0TES8lzPL/c6WbIKMIc07x/4CqK1W/0kjzlbpiJYOHTqc+nrhwoW89957LFmyhPbt2zNt2rQG1zFp27btqa+TkpJiZogHa+3bwNv1nvtxna/LcVOK67/vWEPPSwvRuRf0Od/dkTWUoOzfABka3hGJGcOucglK3ZV8g+VfQVZDczGjU6dOHD16tMHXysrK6Nq1K+3bt2fDhg188sknUY6ucVpJViIrZ5bbk+PAljOfPzWDRw2yIjEjtffpm4rmKs6HlC5uNpnEhLS0NCZOnMiIESO4554zE8/p06dTXV3NqFGj+NGPfsSECRM8ivKz4nihBIkLOTPgnfvdB96Uu08/f3QvVJSp/0Qk1uTMcr+zB7ZA+qDg31+kFWRj0fPPP9/g823btuXvf/97g6/5+0zS09NZu3btqefvvvvuBo8PN1VQJLJSsyDrvM/ekZ1a4l4JikhMyfGtQdOcKkp1hRpkJWyUoEjk5fhWBfVv/gbaJFAkVjV2UxGI/eugtkoNshIWSlAk8vybv9Vd+r5kA7Tr6vbhEZHY0tBNRSBONcgqQZHQKUGRyPOvClr3jsw/g0fj1CKxp6GbikAU50NKqtusUiRESlAkOnJmutUlS7f7ZvBs0AwekVjVpQ/0Hhv8ME9RvpterBsPCQMlKBIdde/Iju2H8sNa4l4kluXMOn1TEYjqSteDouEdCRMlKBIdXftBr3PdHZl/Bo8qKCKx69RNRYBVlP3roKZSDbIxqLHdjAPxm9/8hhMnToQ5osAoQZHoyZnptmHf9K77XjN4RGLXqZuKAPtQile5R1VQYk68JihaqE2iZ/gseO8nkPekW2myYw+vIxKRs8mZ6X5nD+1ouvG1OB/apkK3AdGITIIwd+5ctm7dSm5uLpdeeindu3fn5ZdfpqKigquvvpqf/exnHD9+nOuvv57CwkJqamr40Y9+xL59+ygqKuKiiy4iPT2d999/P6pxK0GR6Oma7e6uitVIJxIX/DcV616Hid85+7FF+ZA5Sr/XTfn7XDeFO5x6joQrHmz05QcffJC1a9eSn5/Pu+++yyuvvMKyZcuw1jJjxgw++OADSkpK6NWrF2+99Rbg9uhJTU3l17/+Ne+//z7p6dHf1FVDPBJdw2e5R/WfiMQ+/01FU8M8NVWwr0AbBMaBd999l3fffZcxY8Zw7rnnsmHDBjZv3szIkSN57733uO+++1i8eDGpqaleh6oKikRZzkz413/pg0wkXgyfBe/9FA7vgi59Gz5m/3qoqYBeY6IaWlw6S6UjGqy13H///Xzta1/7zGsrVqzg7bff5v777+eyyy7jxz/+sQcRnqYKikRXtwHwzaVw7m1eRyIigQhk0bZirSAbyzp16sTRo0cBuPzyy3nyySc5duwYAHv27GH//v0UFRXRvn17brnlFu6++25Wrlz5mfdGmyooEn3pg72OQEQC1W0A9BwFBfPhwm81fEzxKmjTSQ2yMSotLY2JEycyYsQIrrjiCm666SYuuOACADp27Mizzz7Lli1buOeee2jVqhWtW7fmD3/4AwB33nknV1xxBZmZmVFvkjXW2qhesCnjxo2zeXl5XochIiJ+i38F/3oAvrvWrTJb3xOXQHIKfPGt6McWB9avX8+wYS1nYcqG/r3GmBXW2nHBnEdDPCIicnY5vub29Qs++1pNNexbqwXaJOyUoIiIyNmlDYQeI90wT30lG6C6XI3vEnZKUEREpGnDZ0LhMijbc+bzapANSKy1U0RKOP+dSlBERKRpOVe7x/rDPEX50KYjpA2KfkxxIiUlhYMHDyZ8kmKt5eDBg6SkpITlfDE3i2fTvqPsLSunZ2p4/oEiIhIG6YOg+3A3zDPhP04/X7zKzfJppfvdxmRlZVFYWEhJSYnXoURcSkoKWVlZYTlXzCUolTW1/Odra/jz7eMwWjJZRCR2DJ8F7/8cjhRB516uQXbvGhj3Ra8ji2mtW7emf//+XocRd2Iu5e3ZOYV/b9jPa5/uafpgERGJnlOzed5wjwc2QfVJ9Z9IRMRcgpLesS3n9u3Cz95Yx/4j5V6HIyIifhlDoHvO6dk8pxpkNYNHwi/mEhSAX8wezcmqGn44f23CNxWJiMSVnJmwawkc3esaZFt30OrQEhExmaAM6t6R7186hHfX7eON1cVehyMiIn45swDrhnmK86HnSGiV5HVUkoBiMkEB+Mqk/ozOSuUnr6/lwLEKr8MRERGA7udAxjmw9lXXIKsVZCVCYjZBSU5qxcPXjeZ4RQ0/eb3A63BERMTPP8xTdUINshIxMZugAAzp0YnvfG4wb60p5u9rNNQjIhIT/LN5QBUUiZiYTlAA7pwygBG9O/Oj19dSerzS63BERKT7MEgfAsnt3KNIBASUoBhjphtjNhpjthhj5jbweltjzEu+15caY7LrvDbKGLPEGFNgjFljjAlqidjWSa14ePZoyk5W8bM3NNQjIuI5Y+CSn8BF/6kGWYmYJhMUY0wS8ChwBZAD3GiMyal32JeBQ9baQcAjwEO+9yYDzwJft9YOB6YBVcEGOSyzM9+8aBCv5xfxz3X7gn27iIiE27AvwMRvex2FJLBAKijjgS3W2m3W2krgRWBmvWNmAvN8X78CXGLcOvWXAauttasArLUHrbU1zQn0G9MGcU7PTvzna2s4fEJDPSIiIokskASlN7C7zveFvucaPMZaWw2UAWnAEMAaY94xxqw0xtzb0AWMMXcaY/KMMXmNbabUJrkVv7xuNKXHK3ngzXUBhC0iIiLxKpAEpaEd++ov79rYMcnAJOBm3+PVxphLPnOgtY9ba8dZa8dlZGQ0GsiI3qn8x9SB/G3lHv69QUM9IiIiiSqQBKUQ6FPn+yygqLFjfH0nqUCp7/lF1toD1toTwNvAuaEE/K1LBjGkR0f+829rOVIedDuLiIiIxIFAEpTlwGBjTH9jTBtgDrCg3jELgNt9X88G/m3dJjrvAKOMMe19ictUIKTxmbbJSTw8ezT7j5bz8zfXh3IqERERiVFNJii+npK7cMnGeuBla22BMeYBY8wM32F/BtKMMVuA7wNzfe89BPwal+TkAyuttW+FGvToPl346pQBvJS3mw82NdyzIiIiIvHLxNpuwePGjbN5eXlNHldeVcOVv1tMeVUt73xvCh3bJkchOhEREQmWMWaFtXZcMO+J+ZVkG5PSOolfzB5NUdlJ/udtDfWIiIgkkrhNUADG9uvKlyf257mlu/h4ywGvwxEREZEwiesEBeAHlw0lO6099/1tNccrqr0OR0RERMIg7hOUdm3cUE/hoZM8/M5Gr8MRERGRMIj7BAVgfP9u3H5BNk99vIOl2w56HY6IiIiEKCESFIB7pw+lT7d23Pvqak5WNmu7HxEREYkRCZOgtG+TzEPXjmLnwRP88l0N9YiIiMSzhElQAC4cmM7N5/flyY+2s3LXIa/DERERkWZKqAQF4P7PDyO1XWueXbLT61BERESkmRIuQenYNpmpQzJYtKmE2trYWiVXREREApNwCQrAtKEZHDxeSUHREa9DERERkWZIyARl8uAMABZt2u9xJCIiItIcCZmgpHdsy6isVBZu1E7HIiIi8SghExSAqUMyWLnrEGUnqrwORURERIKUsAnKtKEZ1Fr4UJsIioiIxJ2ETVBGZ3Whc0qy+lBERETiUMImKMlJrZjsm25sraYbi4iIxJOETVDA9aHsO1LBhr1HvQ5FREREgpDQCcq0IW66sWbziIiIxJeETlC6d05hWGZn9aGIiIjEmYROUMAN8+TtOMTRck03FhERiRcJn6BMG5pBda3l460HvQ5FREREApTwCcrYfl3p2DaZRZvUhyIiIhIvEj5BaZ3UiomD0li0UdONRURE4kXCJygAU4d0Z8/hk2wtOeZ1KCIiIhKAlpGgDNV0YxERkXjSIhKU3l3aMbh7R/WhiIiIxIkWkaCAm82zdFspJyqrvQ5FREREmtBiEpSpQ7pTWVPLJ9s03VhERCTWtZgE5bz+XWnXOkl9KCIiInGgxSQobZOTuHBgmvpQRERE4kBACYoxZroxZqMxZosxZm4Dr7c1xrzke32pMSa73ut9jTHHjDF3hyfs5pk2NIOdB0+w/cBxL8MQERGRJjSZoBhjkoBHgSuAHOBGY0xOvcO+DByy1g4CHgEeqvf6I8DfQw83NFOHdAdg0UZtHigiIhLLAqmgjAe2WGu3WWsrgReBmfWOmQnM8339CnCJMcYAGGNmAduAgvCE3Hx909rTP70DCzXMIyIiEtMCSVB6A7vrfF/oe67BY6y11UAZkGaM6QDcB/zsbBcwxtxpjMkzxuSVlEQ2eZg6JINPth2kvKomotcRERGR5gskQTENPFd/U5vGjvkZ8Ii19qxrzFtrH7fWjrPWjsvIyAggpOabOjSD8qpalm0vjeh1REREpPmSAzimEOhT5/ssoKiRYwqNMclAKlAKnA/MNsb8AugC1Bpjyq21vw858ma6YEAabZNbsXBjCVOGRDYZEhERkeYJJEFZDgw2xvQH9gBzgJvqHbMAuB1YAswG/m3d1sGT/QcYY34KHPMyOQFIaZ3E+QPSWLRpP67nV0RERGJNk0M8vp6Su4B3gPXAy9baAmPMA8aYGb7D/ozrOdkCfB/4zFTkWDJtSAZbS46zu/SE16GIiIhIAwKpoGCtfRt4u95zP67zdTlwXRPn+Gkz4ouIqUMz4E1YtKmEWyb08zocERERqafFrCRb14D0DvTp1k7L3ouIiMSoFpmgGGOYOiSDj7ceoLK61utwREREpJ4WmaAATBvSnROVNeTt0HRjERGRWNNiE5QLBqbROslo80AREZEY1GITlA5tkzkvu5v6UERERGJQi01QwO1uvHHfUYrLTnodioiIiNTRohMU/+7GH2iYR0REJKa06ARlSI+OZKamaJhHREQkxrToBMU/3fjDzQeoqtF0YxERkVjRohMUcH0oRyuq+XTXYa9DEREREZ8Wn6BcOCid5FbGt3mgiIiIxIIWn6B0TmnNuf26qg9FREQkhrT4BAVg6pAMCoqOsP9oudehiIiICEpQANeHAvDBpgMeRyIiIiKgBAWAnMzOZHRqq2XvRUREYoQSFNx04ymDM1i8uYSaWut1OCIiIi2eEhSfaUMzOHyiilWFmm4sIiLiNSUoPpMHp9PKoNk8IiIiMUAJik+X9m3I7dNFfSgiIiIxQAlKHVOHdGd14WFKj1d6HYqIiEiLpgSljmlDM7AWFm9WFUVERMRLSlDqGNk7lW4d2rBIfSgiIiKeUoJSR6tWhimD01m0qYRaTTcWERHxjBKUeqYOzeDg8UoKio54HYqIiEiLpQSlnimDMzAGFm7U7sYiIiJeUYJST1rHtozsnarpxiIiIh5SgtKAaUMyWLnrEGUnqrwORUREpEVSgtKAqUMzqLXw4RbtbiwiIuIFJSgNGJ3Vhc4pyby3fh/WajaPiIhItClBaUByUiumj+jJa5/u4YY/fsLHW1VJERERiaaAEhRjzHRjzEZjzBZjzNwGXm9rjHnJ9/pSY0y27/lLjTErjDFrfI8Xhzf8yPmvWSN4YOZwdpYe56YnlnLDH5fwybaDXoclIiLSIpimhjCMMUnAJuBSoBBYDtxorV1X55hvAKOstV83xswBrrbW3mCMGQPss9YWGWNGAO9Ya3uf7Xrjxo2zeXl5of2rwqi8qoYXlu3isYVbKTlawYUD0/jepUM4L7ub16GJiIjEBWPMCmvtuGDeE0gFZTywxVq7zVpbCbwIzKx3zExgnu/rV4BLjDHGWvuptbbI93wBkGKMaRtMgF5LaZ3EFyf2Z/G9F/GjL+Swad8xrvu/Jdzyp6Ws2FnqdXgiIiIJKZAEpTewu873hb7nGjzGWlsNlAFp9Y65FvjUWltR/wLGmDuNMXnGmLySkthcfySldRJfnuQSlR9eOYz1xUe49g9LuPXPS1m565DX4YmIiCSUQBIU08Bz9ceFznqMMWY48BDwtYYuYK193Fo7zlo7LiMjI4CQvNOuTRJfmTyAxfddxP1XnENB0RGueexjbn9yGfm7D3sdnoiISEIIJEEpBPrU+T4LKGrsGGNMMpAKlPq+zwJeA26z1m4NNeBY0b5NMl+bOpDF917EvdOHsqrwMLMe/YgvPbWcNYVlXocnIoCd2JoAAA7tSURBVCIS1wJJUJYDg40x/Y0xbYA5wIJ6xywAbvd9PRv4t7XWGmO6AG8B91trPwpX0LGkQ9tkvjFtEB/edzH3XD6UFTsPcdXvP+Qr85azdo8SFRERkeZochYPgDHm88BvgCTgSWvtz40xDwB51toFxpgU4BlgDK5yMsdau80Y80PgfmBzndNdZq1tdCe+WJvFE6yj5VU89dEOnli8jSPl1Uwf3pOHZo8itV1rr0MTERHxRHNm8QSUoERTvCcofkfKq3jyw+08+v4WcjI788xXzqdzipIUERFpeSI1zViaoXNKa777uSE8dvNY1hUf4bY/L+NouTYfFBERCYQSlAi7NKcHv7/pXNbuKeP2J5dxrKLa65BERERinhKUKLh8eE9+f9O5rC4s4w4lKSIiIk1SghIl00f05H9vHMOnuw/zxb8s47iSFBERkUYpQYmiK0Zm8rs5Y1i56zBffGo5JyqVpIiIiDRECUqUXTkqk9/ckEvejlK+pCRFRESkQUpQPHDV6F48ckMuy7aX8uWn8jhZWeN1SCIiIjFFCYpHZub25tfX57J0+0G+8vRyyquUpIiIiPgpQfHQrDG9+eV1o/l460G++nSekhQREREfJSgeu+bcLB6ePZoPtxxQkiIiIuKjBCUGzB6bxUPXjuLDLQf42jMrlKSIiEiLpwQlRlw/rg8PXjOSRZtK+I9nV1BRrSRFRERaLiUoMeSG8/ryP9eM5P2NJXzj2ZVKUkREpMVSghJjbhzfl59fPYJ/bdjPN5/7lMrqWq9DEhERiTolKDHo5vP78V8zh/Pe+n188/mVSlJERKTFUYISo269IJufzRjOP9ft41svrFTjrIiItCjJXgcgjbv9wmxqreVnb6xj9M/eZWy/rlwwII0JA9MYndWFNsnKL0VEJDEpQYlxX5zYn6E9OvHe+v0s2XaQX/1zE/wT2rVOYlx2VyYMSGPCgDRGZaXSOkkJi4iIJAYlKHHgwkHpXDgoHYBDxytZur2UT7Yd5JNtB3n4nY0AtG+TxLjsbkwY0I0LBqQxsncqyUpYREQkThlrrdcxnGHcuHE2Ly/P6zDixsFjFSzbXsoSX8Kyad8xADq0SeK8/i5ZmTAgjeG9OithERERTxhjVlhrxwXzHlVQ4lxax7ZcMTKTK0ZmAnDgWMWp6sqSrQdZuLEEgE5tk7lkWHfuungwg7p39DJkERGRJqmCkuD2Hy3nk22lLNl6gNfziyivqmHG6F58+5LBDMhQoiIiIpHXnAqKEpQW5OCxCh5fvI2nP95JRXUNs8b05tsXDyY7vYPXoYmISAJTgiIBOXCsgj8u2sozn+ykqsZy9ZjefOviQfRLU6IiIiLhpwRFgrL/aDl/XLSNZz/ZSXWt5dpze/OtiwfTp1t7r0MTEZEEogRFmmX/kXIeW7iV55ftorbWct24LL550SCyuipRERGR0ClBkZDsLSvnDwu38MKy3Vgs143rwzcvGkTvLu28Dk1EROKYEhQJi+Kykzz2/lZeXL4LgDnn9eUbFw0kM1WJioiIBE8JioTVnsMnefT9Lfw1bzcGw43j+/CVyQPo2qFNSOdtZaB9Gy3BIyLSUihBkYgoPHTCl6gUUl0bnp+XPt3anVrl9oKBaarOiIgkMCUoElG7S0/w7rp91IaYpFTW1LK68DCfbCul7GQVAP3S2p+RsPTonBKOkEVEJAZELEExxkwHfgskAX+y1j5Y7/W2wNPAWOAgcIO1dofvtfuBLwM1wLette+c7VpKUFqO2lrL+r1HfCvdHmTp9oMcLa8GoH96B99OzW4/oe5KWERE4lZEEhRjTBKwCbgUKASWAzdaa9fVOeYbwChr7deNMXOAq621NxhjcoAXgPFAL+A9YIi1tqax6ylBablqai3ri4+wZKvbS2jZ9lKOVriEZWBGh1PVlfP7p5HRqa3H0YqISKAitVngeGCLtXab7yIvAjOBdXWOmQn81Pf1K8DvjTHG9/yL1toKYLsxZovvfEuCCVJahqRWhhG9UxnRO5WvThlAdU0tBUVH3MaH2w4y/9M9PLfUzSwa3L0j2ekdMB7HLCIikRFIgtIb2F3n+0Lg/MaOsdZWG2PKgDTf85/Ue2/v+hcwxtwJ3AnQt2/fQGOXBJec1IrRfbowuk8XvjZ1INU1tazZU8aSbQf5ZFspu0tPeB2iiIhESCAJSkM3qfXHhRo7JpD3Yq19HHgc3BBPADFJC5Sc1Ioxfbsypm9XvjHN62hERCRQ5nvBv6dVAMcUAn3qfJ8FFDV2jDEmGUgFSgN8r4iIiMgZAklQlgODjTH9jTFtgDnAgnrHLABu9309G/i3dd23C4A5xpi2xpj+wGBgWXhCFxERkUTV5BCPr6fkLuAd3DTjJ621BcaYB/j/27v3GCmvMo7j3x8XI6GNBUorsRa8JLXSIkhtJDaVQDXWf9RKq5ZGMDH6B6ao0VirSamGSBrrBZtga4tcxLakpbXEPwpZKRWNvbAst6LBy7ZqCZQi1lVjCzz+8T5jh3UvMzs7O+8uv09CZubM+77nmbOHmSfnnJkDT0fEI8A9wPpcBHuMIokhj9tIsaD2BLCkr2/wmJmZmYF/qM3MzMyabCBfM65lisfMzMxsSDlBMTMzs9JxgmJmZmal4wTFzMzMSscJipmZmZWOExQzMzMrHScoZmZmVjpOUMzMzKx0nKCYmZlZ6ZTul2QlvQA82+o4Suhc4GirgxiB3K7N47ZtDrdr87htm+eiiDi7nhP63YtnqEXE5FbHUEaSnq73Z4Ktf27X5nHbNofbtXncts0jqe49bDzFY2ZmZqXjBMXMzMxKxwnK8HFXqwMYodyuzeO2bQ63a/O4bZun7rYt3SJZMzMzM4+gmJmZWek4QTEzM7PScYJScpI6Je2V1DGQr2nZqyStlnRE0r6qsomStko6mLcTWhnjcNVL2y6T9Nfsux2SPtjKGIcjSW+UtE3SAUn7JS3NcvfbBvTRru6zDZL0WklPStqdbXtrlr9J0hPZZ++X9Jp+r+U1KOUmqRO4LCL840ENknQl0AWsi4hLsuw24FhErJB0EzAhIr7SyjiHo17adhnQFRHfbmVsw5mkKcCUiGiXdDawE/gwsBj32wHro12vw322IZIEjI+ILkljgR3AUuCLwKaIuE/SD4HdEbGqr2t5BMXOGBHxOHCsW/GHgLV5fy3Fm5TVqZe2tQZFxKGIaM/7/wAOAG/A/bYhfbSrNSgKXflwbP4LYB7wQJbX1GedoJRfAFsk7ZT0mVYHMwKdHxGHoHjTAs5rcTwjzeck7ckpIE9DNEDSNGAW8ATut4OmW7uC+2zDJI2W1AEcAbYCfwCOR8SJPOQv1JAQOkEpv/dExDuBq4ElOZRuNhysAt4CzAQOAbe3NpzhS9JZwIPA5yPipVbHM1L00K7us4MgIk5GxEzgAuBy4OKeDuvvOk5QSi4ins/bI8BDFH9sGzyHcz66Mi99pMXxjBgRcTjfqE4BP8J9d0ByHv9BYENEbMpi99sG9dSu7rODKyKOA48B7wbOkVTZ/+8C4Pn+zneCUmKSxucCLiSNB94P7Ov7LKvTI8CivL8I+FkLYxlRKh+g6SO479YtFxzeAxyIiO9UPeV+24De2tV9tnGSJks6J++PA66iWOOzDViQh9XUZ/0tnhKT9GaKURModp7+aUQsb2FIw5qke4G5FFuqHwZuAR4GNgIXAs8B10aEF3vWqZe2nUsxVB5AJ/DZyroJq42kK4BfAnuBU1l8M8V6CffbAeqjXT+B+2xDJM2gWAQ7mmIQZGNEfCM/z+4DJgK7gBsi4j99XssJipmZmZWNp3jMzMysdJygmJmZWek4QTEzM7PScYJiZmZmpeMExczMzErHCYqZmZmVjhMUM/s/kjolndvoMUNF0hpJC/o/ssdz75b09rx/8+BGZmYD5QTFzM5oEfHpiHgmHzpBMSsJJyhmI4SkaZJ+myMC+yRtkHSVpF9JOijpckkTJT2cu7X+Jn/1EUmTJG2RtEvSnYCqrnuDpCcldUi6U9LoGuP5ZNazW9L6LJsqqS3L2yRdmOVrJK2StE3SHyW9N3eTPSBpTdU1uyTdLqk9z5/cQ72zJW3PHcAflTRF0hhJT0mam8d8S9LyvP+YpMskrQDG5evcIOmbkpZWXXe5pBvr/8uY2UA4QTEbWd4KfB+YAbwNuB64AvgSxejArcCuiJiRj9flebcAOyJiFsU+L5XE4WLgYxS7as8ETgIL+wtC0nTga8C8iHgHUPmgvwNYl/VvAFZWnTYBmAd8AdgMfBeYDlwqaWYeMx5ozx2+t2fc1fWOBX4ALIiI2cBqYHlu874YWCXpfcAHsi3+JyJuAv4dETMjYiHFXi2L8rqjgI9nzGY2BMb0f4iZDSN/ioi9AJL2A20REZL2AtOAqcBHASLiFzly8jrgSuCaLP+5pL/l9eYDs4Gniv3VGEdtO+fOAx6IiKN5zco+MXMq9QDrgduqztlcFevhbq9jGtBBsW/K/Xn8T4BNnO4i4BJga8Y7GjiUMezPkZzNwJyIeLmvFxARnZJelDQLOJ8isXuxhtduZoPACYrZyFK9+dapqsenKP6/n+jhnOh2W03A2oj4ap1xqJfr9VY3nB5r99fR23tV9zoE7I+IOb0cfylwnCLhqMXdFCMvr6cYjTGzIeIpHrMzy+PkFE2uxzgaES91K7+aYroFoA1YIOm8fG6ipKk11NMGXCdpUuW8LP81xVQJWd+OOuMfxatbtl/fw/m/AyZLmpP1js3pJiRdA0yiGC1aWdkSvptXcpqo4iGK6aB3AY/WGauZNcAjKGZnlmXAjyXtAf5FrrGgWI9xr6R2irUdzwFExDOSvg5syXUYrwBLgGf7qiSnU5YD2yWdpNhefTFwI7Ba0peBF4BP1Rn/P4HpknYCf6dYH1Nd78v5deOVOXU1BviepMPACmB+RPxZ0h0Ua3UWnX557gL2SGqPiIV5vW3A8Yg4WWesZtYARdQyCmtm1nqSuiLirCGsbxTQDlwbEQeHql4z8xSPmVmP8sfbfk+x0NjJidkQ8wiKmQ1YrjFp6+Gp+f7Gi5k1wgmKmZmZlY6neMzMzKx0nKCYmZlZ6ThBMTMzs9JxgmJmZmal819WTQqX+/36OgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 648x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "complexity_df.plot(x='model_complexity')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "S rastúcou zložitosťou modelu sa mi nijak nezmenšuje chyba na trénovanej vzorke. Na testovacej tiež nie. Väčšinou sa dokonca zväčšuje. Toto je indikátor toho, že sme ten model preučili. Naučil sa dáta a nie vzťahy za nimi. Model zle zovšeobecňuje / generalizuje vzory v dátach. Ak skúšame predikciu na iných dátach, tak narazíme na veľkú chybu spôsobenú varianciou."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Môžeme si skúsiť vizualizovať rôzne natrénované modely"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"195pt\" viewBox=\"0.00 0.00 250.00 195.00\" width=\"250pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g class=\"graph\" id=\"graph0\" transform=\"scale(1 1) rotate(0) translate(4 191)\">\n",
       "<title>Tree</title>\n",
       "<polygon fill=\"white\" points=\"-4,4 -4,-191 246,-191 246,4 -4,4\" stroke=\"none\"/>\n",
       "<!-- 0 -->\n",
       "<g class=\"node\" id=\"node1\"><title>0</title>\n",
       "<polygon fill=\"#b8dcf6\" points=\"203,-187 38,-187 38,-104 203,-104 203,-187\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"120.5\" y=\"-171.8\">worst perimeter &lt;= 105.95</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"120.5\" y=\"-156.8\">gini = 0.476</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"120.5\" y=\"-141.8\">samples = 455</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"120.5\" y=\"-126.8\">value = [178, 277]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"120.5\" y=\"-111.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g class=\"node\" id=\"node2\"><title>1</title>\n",
       "<polygon fill=\"#44a2e6\" points=\"111,-68 0,-68 0,-0 111,-0 111,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"55.5\" y=\"-52.8\">gini = 0.097</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"55.5\" y=\"-37.8\">samples = 275</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"55.5\" y=\"-22.8\">value = [14, 261]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"55.5\" y=\"-7.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g class=\"edge\" id=\"edge1\"><title>0-&gt;1</title>\n",
       "<path d=\"M96.2964,-103.726C91.0988,-94.9703 85.5978,-85.7032 80.3752,-76.9051\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"83.3811,-75.1121 75.2669,-68.2996 77.3617,-78.6853 83.3811,-75.1121\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"69.0892\" y=\"-88.8243\">True</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g class=\"node\" id=\"node3\"><title>2</title>\n",
       "<polygon fill=\"#e88d4c\" points=\"242,-68 129,-68 129,-0 242,-0 242,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"185.5\" y=\"-52.8\">gini = 0.162</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"185.5\" y=\"-37.8\">samples = 180</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"185.5\" y=\"-22.8\">value = [164, 16]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"185.5\" y=\"-7.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;2 -->\n",
       "<g class=\"edge\" id=\"edge2\"><title>0-&gt;2</title>\n",
       "<path d=\"M144.704,-103.726C149.901,-94.9703 155.402,-85.7032 160.625,-76.9051\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"163.638,-78.6853 165.733,-68.2996 157.619,-75.1121 163.638,-78.6853\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171.911\" y=\"-88.8243\">False</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>svg{width:100% !important;height:100% !important;}</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph = Source(export_graphviz(estimators[0], # len jeden atribut pouzity na rozhodnutie\n",
    "                               out_file=None,\n",
    "                               feature_names=feature_names,\n",
    "                               class_names=label_names,\n",
    "                               filled = True))\n",
    "\n",
    "display(SVG(graph.pipe(format='svg')))\n",
    "\n",
    "from IPython.display import HTML # toto je tu len pre to aby sa mi obrazok zmestil na obrazovku\n",
    "style = \"<style>svg{width:100% !important;height:100% !important;}</style>\"\n",
    "HTML(style)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"671pt\" viewBox=\"0.00 0.00 1261.50 671.00\" width=\"1262pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g class=\"graph\" id=\"graph0\" transform=\"scale(1 1) rotate(0) translate(4 667)\">\n",
       "<title>Tree</title>\n",
       "<polygon fill=\"white\" points=\"-4,4 -4,-667 1257.5,-667 1257.5,4 -4,4\" stroke=\"none\"/>\n",
       "<!-- 0 -->\n",
       "<g class=\"node\" id=\"node1\"><title>0</title>\n",
       "<polygon fill=\"#b8dcf6\" points=\"638.5,-663 473.5,-663 473.5,-580 638.5,-580 638.5,-663\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"556\" y=\"-647.8\">worst perimeter &lt;= 105.95</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"556\" y=\"-632.8\">gini = 0.476</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"556\" y=\"-617.8\">samples = 455</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"556\" y=\"-602.8\">value = [178, 277]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"556\" y=\"-587.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g class=\"node\" id=\"node2\"><title>1</title>\n",
       "<polygon fill=\"#44a2e6\" points=\"542.5,-544 351.5,-544 351.5,-461 542.5,-461 542.5,-544\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-528.8\">worst concave points &lt;= 0.159</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-513.8\">gini = 0.097</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-498.8\">samples = 275</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-483.8\">value = [14, 261]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-468.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g class=\"edge\" id=\"edge1\"><title>0-&gt;1</title>\n",
       "<path d=\"M518.183,-579.907C509.644,-570.742 500.5,-560.927 491.708,-551.489\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"494.128,-548.952 484.75,-544.021 489.006,-553.723 494.128,-548.952\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"483.866\" y=\"-565.305\">True</text>\n",
       "</g>\n",
       "<!-- 14 -->\n",
       "<g class=\"node\" id=\"node15\"><title>14</title>\n",
       "<polygon fill=\"#e88d4c\" points=\"729.5,-544 600.5,-544 600.5,-461 729.5,-461 729.5,-544\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-528.8\">worst area &lt;= 868.2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-513.8\">gini = 0.162</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-498.8\">samples = 180</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-483.8\">value = [164, 16]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-468.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;14 -->\n",
       "<g class=\"edge\" id=\"edge14\"><title>0-&gt;14</title>\n",
       "<path d=\"M593.817,-579.907C602.356,-570.742 611.5,-560.927 620.292,-551.489\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"622.994,-553.723 627.25,-544.021 617.872,-548.952 622.994,-553.723\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"628.134\" y=\"-565.305\">False</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g class=\"node\" id=\"node3\"><title>2</title>\n",
       "<polygon fill=\"#3ea0e6\" points=\"316.5,-425 159.5,-425 159.5,-342 316.5,-342 316.5,-425\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-409.8\">symmetry error &lt;= 0.009</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-394.8\">gini = 0.051</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-379.8\">samples = 267</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-364.8\">value = [7, 260]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-349.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g class=\"edge\" id=\"edge2\"><title>1-&gt;2</title>\n",
       "<path d=\"M374.488,-460.907C356.656,-450.925 337.449,-440.172 319.235,-429.976\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"320.819,-426.852 310.383,-425.021 317.399,-432.96 320.819,-426.852\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g class=\"node\" id=\"node12\"><title>11</title>\n",
       "<polygon fill=\"#e99355\" points=\"519,-425 375,-425 375,-342 519,-342 519,-425\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-409.8\">worst texture &lt;= 23.47</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-394.8\">gini = 0.219</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-379.8\">samples = 8</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-364.8\">value = [7, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"447\" y=\"-349.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;11 -->\n",
       "<g class=\"edge\" id=\"edge11\"><title>1-&gt;11</title>\n",
       "<path d=\"M447,-460.907C447,-452.649 447,-443.864 447,-435.302\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"450.5,-435.021 447,-425.021 443.5,-435.021 450.5,-435.021\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g class=\"node\" id=\"node4\"><title>3</title>\n",
       "<polygon fill=\"#e58139\" points=\"133.5,-298.5 20.5,-298.5 20.5,-230.5 133.5,-230.5 133.5,-298.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"77\" y=\"-283.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"77\" y=\"-268.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"77\" y=\"-253.3\">value = [1, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"77\" y=\"-238.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g class=\"edge\" id=\"edge3\"><title>2-&gt;3</title>\n",
       "<path d=\"M182.142,-341.907C165.591,-329.88 147.502,-316.735 131.067,-304.791\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"132.787,-301.714 122.64,-298.667 128.672,-307.377 132.787,-301.714\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g class=\"node\" id=\"node5\"><title>4</title>\n",
       "<polygon fill=\"#3e9fe6\" points=\"324.5,-306 151.5,-306 151.5,-223 324.5,-223 324.5,-306\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-290.8\">worst smoothness &lt;= 0.178</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-275.8\">gini = 0.044</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-260.8\">samples = 266</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-245.8\">value = [6, 260]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"238\" y=\"-230.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;4 -->\n",
       "<g class=\"edge\" id=\"edge4\"><title>2-&gt;4</title>\n",
       "<path d=\"M238,-341.907C238,-333.649 238,-324.864 238,-316.302\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"241.5,-316.021 238,-306.021 234.5,-316.021 241.5,-316.021\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g class=\"node\" id=\"node6\"><title>5</title>\n",
       "<polygon fill=\"#3c9fe5\" points=\"226.5,-187 95.5,-187 95.5,-104 226.5,-104 226.5,-187\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"161\" y=\"-171.8\">area error &lt;= 48.975</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"161\" y=\"-156.8\">gini = 0.03</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"161\" y=\"-141.8\">samples = 262</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"161\" y=\"-126.8\">value = [4, 258]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"161\" y=\"-111.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;5 -->\n",
       "<g class=\"edge\" id=\"edge5\"><title>4-&gt;5</title>\n",
       "<path d=\"M211.285,-222.907C205.492,-214.105 199.305,-204.703 193.322,-195.612\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"196.089,-193.45 187.668,-187.021 190.241,-197.298 196.089,-193.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g class=\"node\" id=\"node9\"><title>8</title>\n",
       "<polygon fill=\"#ffffff\" points=\"385,-187 245,-187 245,-104 385,-104 385,-187\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"315\" y=\"-171.8\">texture error &lt;= 1.409</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"315\" y=\"-156.8\">gini = 0.5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"315\" y=\"-141.8\">samples = 4</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"315\" y=\"-126.8\">value = [2, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"315\" y=\"-111.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;8 -->\n",
       "<g class=\"edge\" id=\"edge8\"><title>4-&gt;8</title>\n",
       "<path d=\"M264.715,-222.907C270.508,-214.105 276.695,-204.703 282.678,-195.612\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"285.759,-197.298 288.332,-187.021 279.911,-193.45 285.759,-197.298\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g class=\"node\" id=\"node7\"><title>6</title>\n",
       "<polygon fill=\"#3b9ee5\" points=\"104,-68 0,-68 0,-0 104,-0 104,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"52\" y=\"-52.8\">gini = 0.023</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"52\" y=\"-37.8\">samples = 259</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"52\" y=\"-22.8\">value = [3, 256]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"52\" y=\"-7.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;6 -->\n",
       "<g class=\"edge\" id=\"edge6\"><title>5-&gt;6</title>\n",
       "<path d=\"M120.412,-103.726C111.242,-94.5142 101.509,-84.7364 92.3492,-75.5343\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"94.683,-72.9176 85.1475,-68.2996 89.7219,-77.8561 94.683,-72.9176\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g class=\"node\" id=\"node8\"><title>7</title>\n",
       "<polygon fill=\"#9ccef2\" points=\"219.5,-68 122.5,-68 122.5,-0 219.5,-0 219.5,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-52.8\">gini = 0.444</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-37.8\">samples = 3</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-22.8\">value = [1, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171\" y=\"-7.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;7 -->\n",
       "<g class=\"edge\" id=\"edge7\"><title>5-&gt;7</title>\n",
       "<path d=\"M164.724,-103.726C165.482,-95.4263 166.282,-86.6671 167.047,-78.2834\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"170.535,-78.5765 167.959,-68.2996 163.564,-77.9398 170.535,-78.5765\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g class=\"node\" id=\"node10\"><title>9</title>\n",
       "<polygon fill=\"#399de5\" points=\"353.5,-68 256.5,-68 256.5,-0 353.5,-0 353.5,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"305\" y=\"-52.8\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"305\" y=\"-37.8\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"305\" y=\"-22.8\">value = [0, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"305\" y=\"-7.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;9 -->\n",
       "<g class=\"edge\" id=\"edge9\"><title>8-&gt;9</title>\n",
       "<path d=\"M311.276,-103.726C310.518,-95.4263 309.718,-86.6671 308.953,-78.2834\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"312.436,-77.9398 308.041,-68.2996 305.465,-78.5765 312.436,-77.9398\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g class=\"node\" id=\"node11\"><title>10</title>\n",
       "<polygon fill=\"#e58139\" points=\"484.5,-68 371.5,-68 371.5,-0 484.5,-0 484.5,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"428\" y=\"-52.8\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"428\" y=\"-37.8\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"428\" y=\"-22.8\">value = [2, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"428\" y=\"-7.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;10 -->\n",
       "<g class=\"edge\" id=\"edge10\"><title>8-&gt;10</title>\n",
       "<path d=\"M357.077,-103.726C366.678,-94.423 376.874,-84.5428 386.452,-75.2612\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"388.89,-77.7721 393.636,-68.2996 384.019,-72.7451 388.89,-77.7721\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g class=\"node\" id=\"node13\"><title>12</title>\n",
       "<polygon fill=\"#399de5\" points=\"439.5,-298.5 342.5,-298.5 342.5,-230.5 439.5,-230.5 439.5,-298.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"391\" y=\"-283.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"391\" y=\"-268.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"391\" y=\"-253.3\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"391\" y=\"-238.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 11&#45;&gt;12 -->\n",
       "<g class=\"edge\" id=\"edge12\"><title>11-&gt;12</title>\n",
       "<path d=\"M427.571,-341.907C422.29,-330.873 416.558,-318.898 411.233,-307.773\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"414.349,-306.176 406.875,-298.667 408.035,-309.198 414.349,-306.176\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 13 -->\n",
       "<g class=\"node\" id=\"node14\"><title>13</title>\n",
       "<polygon fill=\"#e58139\" points=\"570.5,-298.5 457.5,-298.5 457.5,-230.5 570.5,-230.5 570.5,-298.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"514\" y=\"-283.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"514\" y=\"-268.3\">samples = 7</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"514\" y=\"-253.3\">value = [7, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"514\" y=\"-238.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 11&#45;&gt;13 -->\n",
       "<g class=\"edge\" id=\"edge13\"><title>11-&gt;13</title>\n",
       "<path d=\"M470.245,-341.907C476.627,-330.763 483.559,-318.658 489.984,-307.439\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"493.075,-309.084 495.007,-298.667 487,-305.606 493.075,-309.084\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 15 -->\n",
       "<g class=\"node\" id=\"node16\"><title>15</title>\n",
       "<polygon fill=\"#fdf7f2\" points=\"751.5,-425 578.5,-425 578.5,-342 751.5,-342 751.5,-425\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-409.8\">worst smoothness &lt;= 0.125</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-394.8\">gini = 0.499</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-379.8\">samples = 29</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-364.8\">value = [15, 14]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"665\" y=\"-349.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 14&#45;&gt;15 -->\n",
       "<g class=\"edge\" id=\"edge15\"><title>14-&gt;15</title>\n",
       "<path d=\"M665,-460.907C665,-452.649 665,-443.864 665,-435.302\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"668.5,-435.021 665,-425.021 661.5,-435.021 668.5,-435.021\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 22 -->\n",
       "<g class=\"node\" id=\"node23\"><title>22</title>\n",
       "<polygon fill=\"#e5833c\" points=\"998.5,-425 847.5,-425 847.5,-342 998.5,-342 998.5,-425\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-409.8\">worst texture &lt;= 16.295</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-394.8\">gini = 0.026</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-379.8\">samples = 151</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-364.8\">value = [149, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-349.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 14&#45;&gt;22 -->\n",
       "<g class=\"edge\" id=\"edge22\"><title>14-&gt;22</title>\n",
       "<path d=\"M729.771,-472.127C762.52,-457.276 802.682,-439.063 837.892,-423.096\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"839.542,-426.19 847.204,-418.873 836.651,-419.815 839.542,-426.19\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 16 -->\n",
       "<g class=\"node\" id=\"node17\"><title>16</title>\n",
       "<polygon fill=\"#399de5\" points=\"687,-298.5 589,-298.5 589,-230.5 687,-230.5 687,-298.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"638\" y=\"-283.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"638\" y=\"-268.3\">samples = 11</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"638\" y=\"-253.3\">value = [0, 11]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"638\" y=\"-238.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 15&#45;&gt;16 -->\n",
       "<g class=\"edge\" id=\"edge16\"><title>15-&gt;16</title>\n",
       "<path d=\"M655.632,-341.907C653.137,-331.094 650.433,-319.376 647.91,-308.441\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"651.313,-307.624 645.654,-298.667 644.492,-309.198 651.313,-307.624\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 17 -->\n",
       "<g class=\"node\" id=\"node18\"><title>17</title>\n",
       "<polygon fill=\"#ea9a61\" points=\"856.5,-306 705.5,-306 705.5,-223 856.5,-223 856.5,-306\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"781\" y=\"-290.8\">worst texture &lt;= 20.355</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"781\" y=\"-275.8\">gini = 0.278</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"781\" y=\"-260.8\">samples = 18</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"781\" y=\"-245.8\">value = [15, 3]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"781\" y=\"-230.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 15&#45;&gt;17 -->\n",
       "<g class=\"edge\" id=\"edge17\"><title>15-&gt;17</title>\n",
       "<path d=\"M705.246,-341.907C714.423,-332.651 724.257,-322.732 733.699,-313.209\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"736.27,-315.586 740.826,-306.021 731.299,-310.658 736.27,-315.586\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 18 -->\n",
       "<g class=\"node\" id=\"node19\"><title>18</title>\n",
       "<polygon fill=\"#399de5\" points=\"711.5,-179.5 614.5,-179.5 614.5,-111.5 711.5,-111.5 711.5,-179.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"663\" y=\"-164.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"663\" y=\"-149.3\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"663\" y=\"-134.3\">value = [0, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"663\" y=\"-119.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 17&#45;&gt;18 -->\n",
       "<g class=\"edge\" id=\"edge18\"><title>17-&gt;18</title>\n",
       "<path d=\"M740.06,-222.907C728.264,-211.211 715.401,-198.457 703.624,-186.78\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"706.016,-184.222 696.45,-179.667 701.087,-189.193 706.016,-184.222\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 19 -->\n",
       "<g class=\"node\" id=\"node20\"><title>19</title>\n",
       "<polygon fill=\"#e78946\" points=\"886,-187 730,-187 730,-104 886,-104 886,-187\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"808\" y=\"-171.8\">concavity error &lt;= 0.016</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"808\" y=\"-156.8\">gini = 0.117</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"808\" y=\"-141.8\">samples = 16</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"808\" y=\"-126.8\">value = [15, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"808\" y=\"-111.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 17&#45;&gt;19 -->\n",
       "<g class=\"edge\" id=\"edge19\"><title>17-&gt;19</title>\n",
       "<path d=\"M790.368,-222.907C792.294,-214.558 794.345,-205.671 796.341,-197.02\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"799.811,-197.552 798.649,-187.021 792.99,-195.978 799.811,-197.552\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 20 -->\n",
       "<g class=\"node\" id=\"node21\"><title>20</title>\n",
       "<polygon fill=\"#399de5\" points=\"757.5,-68 660.5,-68 660.5,-0 757.5,-0 757.5,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"709\" y=\"-52.8\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"709\" y=\"-37.8\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"709\" y=\"-22.8\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"709\" y=\"-7.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 19&#45;&gt;20 -->\n",
       "<g class=\"edge\" id=\"edge20\"><title>19-&gt;20</title>\n",
       "<path d=\"M771.136,-103.726C762.89,-94.6054 754.142,-84.93 745.895,-75.8078\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"748.409,-73.3701 739.106,-68.2996 743.217,-78.0647 748.409,-73.3701\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 21 -->\n",
       "<g class=\"node\" id=\"node22\"><title>21</title>\n",
       "<polygon fill=\"#e58139\" points=\"888.5,-68 775.5,-68 775.5,-0 888.5,-0 888.5,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"832\" y=\"-52.8\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"832\" y=\"-37.8\">samples = 15</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"832\" y=\"-22.8\">value = [15, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"832\" y=\"-7.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 19&#45;&gt;21 -->\n",
       "<g class=\"edge\" id=\"edge21\"><title>19-&gt;21</title>\n",
       "<path d=\"M816.937,-103.726C818.756,-95.4263 820.676,-86.6671 822.513,-78.2834\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"825.979,-78.8171 824.701,-68.2996 819.142,-77.3184 825.979,-78.8171\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 23 -->\n",
       "<g class=\"node\" id=\"node24\"><title>23</title>\n",
       "<polygon fill=\"#399de5\" points=\"971.5,-298.5 874.5,-298.5 874.5,-230.5 971.5,-230.5 971.5,-298.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-283.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-268.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-253.3\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"923\" y=\"-238.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 22&#45;&gt;23 -->\n",
       "<g class=\"edge\" id=\"edge23\"><title>22-&gt;23</title>\n",
       "<path d=\"M923,-341.907C923,-331.204 923,-319.615 923,-308.776\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"926.5,-308.667 923,-298.667 919.5,-308.667 926.5,-308.667\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 24 -->\n",
       "<g class=\"node\" id=\"node25\"><title>24</title>\n",
       "<polygon fill=\"#e5823a\" points=\"1120.5,-306 989.5,-306 989.5,-223 1120.5,-223 1120.5,-306\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1055\" y=\"-290.8\">area error &lt;= 22.005</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1055\" y=\"-275.8\">gini = 0.013</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1055\" y=\"-260.8\">samples = 150</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1055\" y=\"-245.8\">value = [149, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1055\" y=\"-230.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 22&#45;&gt;24 -->\n",
       "<g class=\"edge\" id=\"edge24\"><title>22-&gt;24</title>\n",
       "<path d=\"M968.797,-341.907C979.342,-332.56 990.65,-322.538 1001.49,-312.929\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1004.12,-315.273 1009.28,-306.021 999.479,-310.035 1004.12,-315.273\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 25 -->\n",
       "<g class=\"node\" id=\"node26\"><title>25</title>\n",
       "<polygon fill=\"#eca06a\" points=\"1122.5,-187 931.5,-187 931.5,-104 1122.5,-104 1122.5,-187\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1027\" y=\"-171.8\">worst concave points &lt;= 0.118</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1027\" y=\"-156.8\">gini = 0.32</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1027\" y=\"-141.8\">samples = 5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1027\" y=\"-126.8\">value = [4, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1027\" y=\"-111.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 24&#45;&gt;25 -->\n",
       "<g class=\"edge\" id=\"edge25\"><title>24-&gt;25</title>\n",
       "<path d=\"M1045.29,-222.907C1043.29,-214.558 1041.16,-205.671 1039.09,-197.02\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1042.43,-195.932 1036.7,-187.021 1035.62,-197.561 1042.43,-195.932\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 28 -->\n",
       "<g class=\"node\" id=\"node29\"><title>28</title>\n",
       "<polygon fill=\"#e58139\" points=\"1253.5,-179.5 1140.5,-179.5 1140.5,-111.5 1253.5,-111.5 1253.5,-179.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1197\" y=\"-164.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1197\" y=\"-149.3\">samples = 145</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1197\" y=\"-134.3\">value = [145, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1197\" y=\"-119.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 24&#45;&gt;28 -->\n",
       "<g class=\"edge\" id=\"edge28\"><title>24-&gt;28</title>\n",
       "<path d=\"M1104.27,-222.907C1118.73,-210.99 1134.53,-197.976 1148.91,-186.12\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1151.25,-188.727 1156.75,-179.667 1146.8,-183.325 1151.25,-188.727\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 26 -->\n",
       "<g class=\"node\" id=\"node27\"><title>26</title>\n",
       "<polygon fill=\"#399de5\" points=\"1051.5,-68 954.5,-68 954.5,-0 1051.5,-0 1051.5,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1003\" y=\"-52.8\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1003\" y=\"-37.8\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1003\" y=\"-22.8\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1003\" y=\"-7.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 25&#45;&gt;26 -->\n",
       "<g class=\"edge\" id=\"edge26\"><title>25-&gt;26</title>\n",
       "<path d=\"M1018.06,-103.726C1016.24,-95.4263 1014.32,-86.6671 1012.49,-78.2834\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1015.86,-77.3184 1010.3,-68.2996 1009.02,-78.8171 1015.86,-77.3184\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 27 -->\n",
       "<g class=\"node\" id=\"node28\"><title>27</title>\n",
       "<polygon fill=\"#e58139\" points=\"1182.5,-68 1069.5,-68 1069.5,-0 1182.5,-0 1182.5,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1126\" y=\"-52.8\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1126\" y=\"-37.8\">samples = 4</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1126\" y=\"-22.8\">value = [4, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1126\" y=\"-7.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 25&#45;&gt;27 -->\n",
       "<g class=\"edge\" id=\"edge27\"><title>25-&gt;27</title>\n",
       "<path d=\"M1063.86,-103.726C1072.11,-94.6054 1080.86,-84.93 1089.11,-75.8078\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1091.78,-78.0647 1095.89,-68.2996 1086.59,-73.3701 1091.78,-78.0647\" stroke=\"black\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>svg{width:70% !important;height:70% !important;}</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph = Source(export_graphviz(estimators[4], # model, za ktorym zacala rast chyba na validacnej vzorke\n",
    "                               out_file=None,\n",
    "                               feature_names=feature_names,\n",
    "                               class_names=label_names,\n",
    "                               filled = True))\n",
    "\n",
    "display(SVG(graph.pipe(format='svg')))\n",
    "\n",
    "from IPython.display import HTML # toto je tu len pre to aby sa mi obrazok zmestil na obrazovku\n",
    "style = \"<style>svg{width:70% !important;height:70% !important;}</style>\"\n",
    "HTML(style)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"1147pt\" viewBox=\"0.00 0.00 1651.00 1147.00\" width=\"1651pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g class=\"graph\" id=\"graph0\" transform=\"scale(1 1) rotate(0) translate(4 1143)\">\n",
       "<title>Tree</title>\n",
       "<polygon fill=\"white\" points=\"-4,4 -4,-1143 1647,-1143 1647,4 -4,4\" stroke=\"none\"/>\n",
       "<!-- 0 -->\n",
       "<g class=\"node\" id=\"node1\"><title>0</title>\n",
       "<polygon fill=\"#b8dcf6\" points=\"1036,-1139 871,-1139 871,-1056 1036,-1056 1036,-1139\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"953.5\" y=\"-1123.8\">worst perimeter &lt;= 105.95</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"953.5\" y=\"-1108.8\">gini = 0.476</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"953.5\" y=\"-1093.8\">samples = 455</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"953.5\" y=\"-1078.8\">value = [178, 277]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"953.5\" y=\"-1063.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g class=\"node\" id=\"node2\"><title>1</title>\n",
       "<polygon fill=\"#44a2e6\" points=\"944,-1020 753,-1020 753,-937 944,-937 944,-1020\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-1004.8\">worst concave points &lt;= 0.159</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-989.8\">gini = 0.097</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-974.8\">samples = 275</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-959.8\">value = [14, 261]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-944.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g class=\"edge\" id=\"edge1\"><title>0-&gt;1</title>\n",
       "<path d=\"M917.071,-1055.91C908.845,-1046.74 900.037,-1036.93 891.567,-1027.49\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"894.149,-1025.13 884.865,-1020.02 888.939,-1029.8 894.149,-1025.13\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"883.515\" y=\"-1041.28\">True</text>\n",
       "</g>\n",
       "<!-- 24 -->\n",
       "<g class=\"node\" id=\"node25\"><title>24</title>\n",
       "<polygon fill=\"#e88d4c\" points=\"1124,-1020 995,-1020 995,-937 1124,-937 1124,-1020\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-1004.8\">worst area &lt;= 868.2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-989.8\">gini = 0.162</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-974.8\">samples = 180</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-959.8\">value = [164, 16]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-944.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;24 -->\n",
       "<g class=\"edge\" id=\"edge24\"><title>0-&gt;24</title>\n",
       "<path d=\"M990.276,-1055.91C998.58,-1046.74 1007.47,-1036.93 1016.02,-1027.49\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1018.67,-1029.78 1022.79,-1020.02 1013.48,-1025.08 1018.67,-1029.78\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1024.02\" y=\"-1041.29\">False</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g class=\"node\" id=\"node3\"><title>2</title>\n",
       "<polygon fill=\"#3ea0e6\" points=\"730.5,-901 594.5,-901 594.5,-818 730.5,-818 730.5,-901\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-885.8\">radius error &lt;= 1.048</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-870.8\">gini = 0.051</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-855.8\">samples = 267</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-840.8\">value = [7, 260]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-825.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g class=\"edge\" id=\"edge2\"><title>1-&gt;2</title>\n",
       "<path d=\"M783.968,-936.907C768.387,-927.106 751.626,-916.563 735.68,-906.533\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"737.246,-903.383 726.918,-901.021 733.519,-909.308 737.246,-903.383\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 21 -->\n",
       "<g class=\"node\" id=\"node22\"><title>21</title>\n",
       "<polygon fill=\"#e99355\" points=\"914,-901 783,-901 783,-818 914,-818 914,-901\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-885.8\">area error &lt;= 13.725</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-870.8\">gini = 0.219</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-855.8\">samples = 8</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-840.8\">value = [7, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"848.5\" y=\"-825.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;21 -->\n",
       "<g class=\"edge\" id=\"edge21\"><title>1-&gt;21</title>\n",
       "<path d=\"M848.5,-936.907C848.5,-928.649 848.5,-919.864 848.5,-911.302\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"852,-911.021 848.5,-901.021 845,-911.021 852,-911.021\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g class=\"node\" id=\"node4\"><title>3</title>\n",
       "<polygon fill=\"#3e9fe6\" points=\"588,-782 415,-782 415,-699 588,-699 588,-782\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"501.5\" y=\"-766.8\">worst smoothness &lt;= 0.178</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"501.5\" y=\"-751.8\">gini = 0.044</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"501.5\" y=\"-736.8\">samples = 266</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"501.5\" y=\"-721.8\">value = [6, 260]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"501.5\" y=\"-706.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g class=\"edge\" id=\"edge3\"><title>2-&gt;3</title>\n",
       "<path d=\"M606.642,-817.907C593.405,-808.288 579.184,-797.953 565.611,-788.09\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"567.407,-785.068 557.259,-782.021 563.291,-790.731 567.407,-785.068\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 20 -->\n",
       "<g class=\"node\" id=\"node21\"><title>20</title>\n",
       "<polygon fill=\"#e58139\" points=\"719,-774.5 606,-774.5 606,-706.5 719,-706.5 719,-774.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-759.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-744.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-729.3\">value = [1, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"662.5\" y=\"-714.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;20 -->\n",
       "<g class=\"edge\" id=\"edge20\"><title>2-&gt;20</title>\n",
       "<path d=\"M662.5,-817.907C662.5,-807.204 662.5,-795.615 662.5,-784.776\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"666,-784.667 662.5,-774.667 659,-784.667 666,-784.667\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g class=\"node\" id=\"node5\"><title>4</title>\n",
       "<polygon fill=\"#3c9fe5\" points=\"483,-663 352,-663 352,-580 483,-580 483,-663\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"417.5\" y=\"-647.8\">area error &lt;= 48.975</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"417.5\" y=\"-632.8\">gini = 0.03</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"417.5\" y=\"-617.8\">samples = 262</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"417.5\" y=\"-602.8\">value = [4, 258]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"417.5\" y=\"-587.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;4 -->\n",
       "<g class=\"edge\" id=\"edge4\"><title>3-&gt;4</title>\n",
       "<path d=\"M472.357,-698.907C465.971,-690.014 459.148,-680.509 452.558,-671.331\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"455.267,-669.103 446.592,-663.021 449.581,-673.185 455.267,-669.103\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 17 -->\n",
       "<g class=\"node\" id=\"node18\"><title>17</title>\n",
       "<polygon fill=\"#ffffff\" points=\"670,-663 501,-663 501,-580 670,-580 670,-663\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"585.5\" y=\"-647.8\">smoothness error &lt;= 0.012</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"585.5\" y=\"-632.8\">gini = 0.5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"585.5\" y=\"-617.8\">samples = 4</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"585.5\" y=\"-602.8\">value = [2, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"585.5\" y=\"-587.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;17 -->\n",
       "<g class=\"edge\" id=\"edge17\"><title>3-&gt;17</title>\n",
       "<path d=\"M530.643,-698.907C537.029,-690.014 543.852,-680.509 550.442,-671.331\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"553.419,-673.185 556.408,-663.021 547.733,-669.103 553.419,-673.185\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g class=\"node\" id=\"node6\"><title>5</title>\n",
       "<polygon fill=\"#3b9ee5\" points=\"325,-544 174,-544 174,-461 325,-461 325,-544\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"249.5\" y=\"-528.8\">worst texture &lt;= 30.145</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"249.5\" y=\"-513.8\">gini = 0.023</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"249.5\" y=\"-498.8\">samples = 259</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"249.5\" y=\"-483.8\">value = [3, 256]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"249.5\" y=\"-468.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;5 -->\n",
       "<g class=\"edge\" id=\"edge5\"><title>4-&gt;5</title>\n",
       "<path d=\"M359.213,-579.907C345.27,-570.197 330.281,-559.758 315.998,-549.811\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"317.89,-546.864 307.684,-544.021 313.89,-552.608 317.89,-546.864\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 14 -->\n",
       "<g class=\"node\" id=\"node15\"><title>14</title>\n",
       "<polygon fill=\"#9ccef2\" points=\"500,-544 343,-544 343,-461 500,-461 500,-544\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"421.5\" y=\"-528.8\">symmetry error &lt;= 0.026</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"421.5\" y=\"-513.8\">gini = 0.444</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"421.5\" y=\"-498.8\">samples = 3</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"421.5\" y=\"-483.8\">value = [1, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"421.5\" y=\"-468.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;14 -->\n",
       "<g class=\"edge\" id=\"edge14\"><title>4-&gt;14</title>\n",
       "<path d=\"M418.888,-579.907C419.173,-571.558 419.477,-562.671 419.773,-554.02\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"423.271,-554.135 420.115,-544.021 416.275,-553.895 423.271,-554.135\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g class=\"node\" id=\"node7\"><title>6</title>\n",
       "<polygon fill=\"#399de5\" points=\"175.5,-417.5 71.5,-417.5 71.5,-349.5 175.5,-349.5 175.5,-417.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"123.5\" y=\"-402.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"123.5\" y=\"-387.3\">samples = 232</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"123.5\" y=\"-372.3\">value = [0, 232]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"123.5\" y=\"-357.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;6 -->\n",
       "<g class=\"edge\" id=\"edge6\"><title>5-&gt;6</title>\n",
       "<path d=\"M205.785,-460.907C193.189,-449.211 179.454,-436.457 166.878,-424.78\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"168.928,-421.907 159.218,-417.667 164.165,-427.036 168.928,-421.907\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g class=\"node\" id=\"node8\"><title>7</title>\n",
       "<polygon fill=\"#52a9e8\" points=\"323,-425 194,-425 194,-342 323,-342 323,-425\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"258.5\" y=\"-409.8\">worst area &lt;= 721.9</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"258.5\" y=\"-394.8\">gini = 0.198</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"258.5\" y=\"-379.8\">samples = 27</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"258.5\" y=\"-364.8\">value = [3, 24]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"258.5\" y=\"-349.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;7 -->\n",
       "<g class=\"edge\" id=\"edge7\"><title>5-&gt;7</title>\n",
       "<path d=\"M252.623,-460.907C253.265,-452.558 253.948,-443.671 254.614,-435.02\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"258.106,-435.26 255.383,-425.021 251.126,-434.723 258.106,-435.26\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g class=\"node\" id=\"node9\"><title>8</title>\n",
       "<polygon fill=\"#41a1e6\" points=\"256.5,-306 116.5,-306 116.5,-223 256.5,-223 256.5,-306\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"186.5\" y=\"-290.8\">texture error &lt;= 1.429</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"186.5\" y=\"-275.8\">gini = 0.077</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"186.5\" y=\"-260.8\">samples = 25</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"186.5\" y=\"-245.8\">value = [1, 24]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"186.5\" y=\"-230.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 7&#45;&gt;8 -->\n",
       "<g class=\"edge\" id=\"edge8\"><title>7-&gt;8</title>\n",
       "<path d=\"M233.52,-341.907C228.103,-333.105 222.317,-323.703 216.723,-314.612\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"219.658,-312.703 211.436,-306.021 213.696,-316.372 219.658,-312.703\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 13 -->\n",
       "<g class=\"node\" id=\"node14\"><title>13</title>\n",
       "<polygon fill=\"#e58139\" points=\"388,-298.5 275,-298.5 275,-230.5 388,-230.5 388,-298.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"331.5\" y=\"-283.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"331.5\" y=\"-268.3\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"331.5\" y=\"-253.3\">value = [2, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"331.5\" y=\"-238.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 7&#45;&gt;13 -->\n",
       "<g class=\"edge\" id=\"edge13\"><title>7-&gt;13</title>\n",
       "<path d=\"M283.827,-341.907C290.781,-330.763 298.333,-318.658 305.333,-307.439\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"308.482,-309.004 310.806,-298.667 302.543,-305.298 308.482,-309.004\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g class=\"node\" id=\"node10\"><title>9</title>\n",
       "<polygon fill=\"#6ab6ec\" points=\"196,-187 23,-187 23,-104 196,-104 196,-187\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"109.5\" y=\"-171.8\">worst smoothness &lt;= 0.137</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"109.5\" y=\"-156.8\">gini = 0.32</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"109.5\" y=\"-141.8\">samples = 5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"109.5\" y=\"-126.8\">value = [1, 4]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"109.5\" y=\"-111.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;9 -->\n",
       "<g class=\"edge\" id=\"edge9\"><title>8-&gt;9</title>\n",
       "<path d=\"M159.785,-222.907C153.992,-214.105 147.805,-204.703 141.822,-195.612\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"144.589,-193.45 136.168,-187.021 138.741,-197.298 144.589,-193.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g class=\"node\" id=\"node13\"><title>12</title>\n",
       "<polygon fill=\"#399de5\" points=\"312.5,-179.5 214.5,-179.5 214.5,-111.5 312.5,-111.5 312.5,-179.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"263.5\" y=\"-164.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"263.5\" y=\"-149.3\">samples = 20</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"263.5\" y=\"-134.3\">value = [0, 20]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"263.5\" y=\"-119.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;12 -->\n",
       "<g class=\"edge\" id=\"edge12\"><title>8-&gt;12</title>\n",
       "<path d=\"M213.215,-222.907C220.622,-211.652 228.673,-199.418 236.118,-188.106\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"239.098,-189.944 241.672,-179.667 233.251,-186.096 239.098,-189.944\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g class=\"node\" id=\"node11\"><title>10</title>\n",
       "<polygon fill=\"#399de5\" points=\"97,-68 0,-68 0,-0 97,-0 97,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"48.5\" y=\"-52.8\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"48.5\" y=\"-37.8\">samples = 4</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"48.5\" y=\"-22.8\">value = [0, 4]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"48.5\" y=\"-7.8\">class = benign</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;10 -->\n",
       "<g class=\"edge\" id=\"edge10\"><title>9-&gt;10</title>\n",
       "<path d=\"M86.7859,-103.726C81.9589,-95.0615 76.8531,-85.8962 71.9976,-77.1802\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"74.9747,-75.3322 67.0505,-68.2996 68.8595,-78.7389 74.9747,-75.3322\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g class=\"node\" id=\"node12\"><title>11</title>\n",
       "<polygon fill=\"#e58139\" points=\"228,-68 115,-68 115,-0 228,-0 228,-68\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171.5\" y=\"-52.8\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171.5\" y=\"-37.8\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171.5\" y=\"-22.8\">value = [1, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"171.5\" y=\"-7.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;11 -->\n",
       "<g class=\"edge\" id=\"edge11\"><title>9-&gt;11</title>\n",
       "<path d=\"M132.587,-103.726C137.493,-95.0615 142.682,-85.8962 147.617,-77.1802\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"150.764,-78.726 152.645,-68.2996 144.673,-75.277 150.764,-78.726\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 15 -->\n",
       "<g class=\"node\" id=\"node16\"><title>15</title>\n",
       "<polygon fill=\"#e58139\" points=\"462,-417.5 349,-417.5 349,-349.5 462,-349.5 462,-417.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"405.5\" y=\"-402.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"405.5\" y=\"-387.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"405.5\" y=\"-372.3\">value = [1, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"405.5\" y=\"-357.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 14&#45;&gt;15 -->\n",
       "<g class=\"edge\" id=\"edge15\"><title>14-&gt;15</title>\n",
       "<path d=\"M415.949,-460.907C414.485,-450.204 412.9,-438.615 411.418,-427.776\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"414.858,-427.1 410.036,-417.667 407.923,-428.049 414.858,-427.1\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 16 -->\n",
       "<g class=\"node\" id=\"node17\"><title>16</title>\n",
       "<polygon fill=\"#399de5\" points=\"577,-417.5 480,-417.5 480,-349.5 577,-349.5 577,-417.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"528.5\" y=\"-402.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"528.5\" y=\"-387.3\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"528.5\" y=\"-372.3\">value = [0, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"528.5\" y=\"-357.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 14&#45;&gt;16 -->\n",
       "<g class=\"edge\" id=\"edge16\"><title>14-&gt;16</title>\n",
       "<path d=\"M458.623,-460.907C469.219,-449.321 480.764,-436.698 491.36,-425.111\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"494.002,-427.408 498.168,-417.667 488.836,-422.684 494.002,-427.408\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 18 -->\n",
       "<g class=\"node\" id=\"node19\"><title>18</title>\n",
       "<polygon fill=\"#e58139\" points=\"638,-536.5 525,-536.5 525,-468.5 638,-468.5 638,-536.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"581.5\" y=\"-521.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"581.5\" y=\"-506.3\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"581.5\" y=\"-491.3\">value = [2, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"581.5\" y=\"-476.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 17&#45;&gt;18 -->\n",
       "<g class=\"edge\" id=\"edge18\"><title>17-&gt;18</title>\n",
       "<path d=\"M584.112,-579.907C583.746,-569.204 583.35,-557.615 582.98,-546.776\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"586.474,-546.541 582.634,-536.667 579.478,-546.781 586.474,-546.541\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 19 -->\n",
       "<g class=\"node\" id=\"node20\"><title>19</title>\n",
       "<polygon fill=\"#399de5\" points=\"753,-536.5 656,-536.5 656,-468.5 753,-468.5 753,-536.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"704.5\" y=\"-521.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"704.5\" y=\"-506.3\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"704.5\" y=\"-491.3\">value = [0, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"704.5\" y=\"-476.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 17&#45;&gt;19 -->\n",
       "<g class=\"edge\" id=\"edge19\"><title>17-&gt;19</title>\n",
       "<path d=\"M626.786,-579.907C638.683,-568.211 651.655,-555.457 663.532,-543.78\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"666.089,-546.174 670.766,-536.667 661.182,-541.182 666.089,-546.174\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 22 -->\n",
       "<g class=\"node\" id=\"node23\"><title>22</title>\n",
       "<polygon fill=\"#399de5\" points=\"834,-774.5 737,-774.5 737,-706.5 834,-706.5 834,-774.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"785.5\" y=\"-759.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"785.5\" y=\"-744.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"785.5\" y=\"-729.3\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"785.5\" y=\"-714.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 21&#45;&gt;22 -->\n",
       "<g class=\"edge\" id=\"edge22\"><title>21-&gt;22</title>\n",
       "<path d=\"M826.642,-817.907C820.701,-806.873 814.253,-794.898 808.262,-783.773\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"811.182,-781.812 803.359,-774.667 805.019,-785.131 811.182,-781.812\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 23 -->\n",
       "<g class=\"node\" id=\"node24\"><title>23</title>\n",
       "<polygon fill=\"#e58139\" points=\"965,-774.5 852,-774.5 852,-706.5 965,-706.5 965,-774.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"908.5\" y=\"-759.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"908.5\" y=\"-744.3\">samples = 7</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"908.5\" y=\"-729.3\">value = [7, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"908.5\" y=\"-714.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 21&#45;&gt;23 -->\n",
       "<g class=\"edge\" id=\"edge23\"><title>21-&gt;23</title>\n",
       "<path d=\"M869.317,-817.907C874.975,-806.873 881.117,-794.898 886.822,-783.773\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"890.042,-785.162 891.491,-774.667 883.814,-781.968 890.042,-785.162\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 25 -->\n",
       "<g class=\"node\" id=\"node26\"><title>25</title>\n",
       "<polygon fill=\"#fdf7f2\" points=\"1146,-901 973,-901 973,-818 1146,-818 1146,-901\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-885.8\">worst smoothness &lt;= 0.125</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-870.8\">gini = 0.499</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-855.8\">samples = 29</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-840.8\">value = [15, 14]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1059.5\" y=\"-825.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 24&#45;&gt;25 -->\n",
       "<g class=\"edge\" id=\"edge25\"><title>24-&gt;25</title>\n",
       "<path d=\"M1059.5,-936.907C1059.5,-928.649 1059.5,-919.864 1059.5,-911.302\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1063,-911.021 1059.5,-901.021 1056,-911.021 1063,-911.021\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 32 -->\n",
       "<g class=\"node\" id=\"node33\"><title>32</title>\n",
       "<polygon fill=\"#e5833c\" points=\"1383.5,-901 1233.5,-901 1233.5,-818 1383.5,-818 1383.5,-901\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-885.8\">symmetry error &lt;= 0.01</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-870.8\">gini = 0.026</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-855.8\">samples = 151</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-840.8\">value = [149, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-825.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 24&#45;&gt;32 -->\n",
       "<g class=\"edge\" id=\"edge32\"><title>24-&gt;32</title>\n",
       "<path d=\"M1124.27,-947.065C1154.83,-932.707 1191.68,-915.392 1224.36,-900.034\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1225.9,-903.178 1233.46,-895.758 1222.93,-896.843 1225.9,-903.178\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 26 -->\n",
       "<g class=\"node\" id=\"node27\"><title>26</title>\n",
       "<polygon fill=\"#399de5\" points=\"1081.5,-774.5 983.5,-774.5 983.5,-706.5 1081.5,-706.5 1081.5,-774.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1032.5\" y=\"-759.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1032.5\" y=\"-744.3\">samples = 11</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1032.5\" y=\"-729.3\">value = [0, 11]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1032.5\" y=\"-714.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 25&#45;&gt;26 -->\n",
       "<g class=\"edge\" id=\"edge26\"><title>25-&gt;26</title>\n",
       "<path d=\"M1050.13,-817.907C1047.64,-807.094 1044.93,-795.376 1042.41,-784.441\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1045.81,-783.624 1040.15,-774.667 1038.99,-785.198 1045.81,-783.624\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 27 -->\n",
       "<g class=\"node\" id=\"node28\"><title>27</title>\n",
       "<polygon fill=\"#ea9a61\" points=\"1241.5,-782 1099.5,-782 1099.5,-699 1241.5,-699 1241.5,-782\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1170.5\" y=\"-766.8\">mean texture &lt;= 15.37</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1170.5\" y=\"-751.8\">gini = 0.278</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1170.5\" y=\"-736.8\">samples = 18</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1170.5\" y=\"-721.8\">value = [15, 3]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1170.5\" y=\"-706.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 25&#45;&gt;27 -->\n",
       "<g class=\"edge\" id=\"edge27\"><title>25-&gt;27</title>\n",
       "<path d=\"M1098.01,-817.907C1106.71,-808.742 1116.02,-798.927 1124.97,-789.489\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1127.71,-791.684 1132.06,-782.021 1122.64,-786.867 1127.71,-791.684\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 28 -->\n",
       "<g class=\"node\" id=\"node29\"><title>28</title>\n",
       "<polygon fill=\"#399de5\" points=\"1098,-655.5 1001,-655.5 1001,-587.5 1098,-587.5 1098,-655.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1049.5\" y=\"-640.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1049.5\" y=\"-625.3\">samples = 2</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1049.5\" y=\"-610.3\">value = [0, 2]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1049.5\" y=\"-595.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 27&#45;&gt;28 -->\n",
       "<g class=\"edge\" id=\"edge28\"><title>27-&gt;28</title>\n",
       "<path d=\"M1128.52,-698.907C1116.42,-687.211 1103.23,-674.457 1091.16,-662.78\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1093.42,-660.102 1083.8,-655.667 1088.56,-665.134 1093.42,-660.102\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 29 -->\n",
       "<g class=\"node\" id=\"node30\"><title>29</title>\n",
       "<polygon fill=\"#e78946\" points=\"1287,-663 1116,-663 1116,-580 1287,-580 1287,-663\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1201.5\" y=\"-647.8\">mean smoothness &lt;= 0.091</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1201.5\" y=\"-632.8\">gini = 0.117</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1201.5\" y=\"-617.8\">samples = 16</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1201.5\" y=\"-602.8\">value = [15, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1201.5\" y=\"-587.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 27&#45;&gt;29 -->\n",
       "<g class=\"edge\" id=\"edge29\"><title>27-&gt;29</title>\n",
       "<path d=\"M1181.26,-698.907C1183.49,-690.468 1185.87,-681.477 1188.19,-672.738\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1191.59,-673.584 1190.76,-663.021 1184.82,-671.791 1191.59,-673.584\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 30 -->\n",
       "<g class=\"node\" id=\"node31\"><title>30</title>\n",
       "<polygon fill=\"#399de5\" points=\"1152,-536.5 1055,-536.5 1055,-468.5 1152,-468.5 1152,-536.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1103.5\" y=\"-521.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1103.5\" y=\"-506.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1103.5\" y=\"-491.3\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1103.5\" y=\"-476.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 29&#45;&gt;30 -->\n",
       "<g class=\"edge\" id=\"edge30\"><title>29-&gt;30</title>\n",
       "<path d=\"M1167.5,-579.907C1157.89,-568.432 1147.42,-555.938 1137.79,-544.442\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1140.39,-542.086 1131.28,-536.667 1135.02,-546.58 1140.39,-542.086\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 31 -->\n",
       "<g class=\"node\" id=\"node32\"><title>31</title>\n",
       "<polygon fill=\"#e58139\" points=\"1283,-536.5 1170,-536.5 1170,-468.5 1283,-468.5 1283,-536.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1226.5\" y=\"-521.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1226.5\" y=\"-506.3\">samples = 15</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1226.5\" y=\"-491.3\">value = [15, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1226.5\" y=\"-476.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 29&#45;&gt;31 -->\n",
       "<g class=\"edge\" id=\"edge31\"><title>29-&gt;31</title>\n",
       "<path d=\"M1210.17,-579.907C1212.46,-569.204 1214.94,-557.615 1217.25,-546.776\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1220.75,-547.178 1219.41,-536.667 1213.9,-545.715 1220.75,-547.178\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 33 -->\n",
       "<g class=\"node\" id=\"node34\"><title>33</title>\n",
       "<polygon fill=\"#399de5\" points=\"1357,-774.5 1260,-774.5 1260,-706.5 1357,-706.5 1357,-774.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-759.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-744.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-729.3\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1308.5\" y=\"-714.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 32&#45;&gt;33 -->\n",
       "<g class=\"edge\" id=\"edge33\"><title>32-&gt;33</title>\n",
       "<path d=\"M1308.5,-817.907C1308.5,-807.204 1308.5,-795.615 1308.5,-784.776\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1312,-784.667 1308.5,-774.667 1305,-784.667 1312,-784.667\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 34 -->\n",
       "<g class=\"node\" id=\"node35\"><title>34</title>\n",
       "<polygon fill=\"#e5823a\" points=\"1529.5,-782 1375.5,-782 1375.5,-699 1529.5,-699 1529.5,-782\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1452.5\" y=\"-766.8\">perimeter error &lt;= 1.752</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1452.5\" y=\"-751.8\">gini = 0.013</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1452.5\" y=\"-736.8\">samples = 150</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1452.5\" y=\"-721.8\">value = [149, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1452.5\" y=\"-706.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 32&#45;&gt;34 -->\n",
       "<g class=\"edge\" id=\"edge34\"><title>32-&gt;34</title>\n",
       "<path d=\"M1358.46,-817.907C1370.19,-808.379 1382.78,-798.148 1394.81,-788.37\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1397.07,-791.043 1402.63,-782.021 1392.66,-785.61 1397.07,-791.043\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 35 -->\n",
       "<g class=\"node\" id=\"node36\"><title>35</title>\n",
       "<polygon fill=\"#eca06a\" points=\"1511.5,-663 1331.5,-663 1331.5,-580 1511.5,-580 1511.5,-663\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1421.5\" y=\"-647.8\">worst compactness &lt;= 0.176</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1421.5\" y=\"-632.8\">gini = 0.32</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1421.5\" y=\"-617.8\">samples = 5</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1421.5\" y=\"-602.8\">value = [4, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1421.5\" y=\"-587.8\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 34&#45;&gt;35 -->\n",
       "<g class=\"edge\" id=\"edge35\"><title>34-&gt;35</title>\n",
       "<path d=\"M1441.74,-698.907C1439.51,-690.468 1437.13,-681.477 1434.81,-672.738\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1438.18,-671.791 1432.24,-663.021 1431.41,-673.584 1438.18,-671.791\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 38 -->\n",
       "<g class=\"node\" id=\"node39\"><title>38</title>\n",
       "<polygon fill=\"#e58139\" points=\"1643,-655.5 1530,-655.5 1530,-587.5 1643,-587.5 1643,-655.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1586.5\" y=\"-640.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1586.5\" y=\"-625.3\">samples = 145</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1586.5\" y=\"-610.3\">value = [145, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1586.5\" y=\"-595.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 34&#45;&gt;38 -->\n",
       "<g class=\"edge\" id=\"edge38\"><title>34-&gt;38</title>\n",
       "<path d=\"M1498.99,-698.907C1512.51,-687.101 1527.27,-674.217 1540.75,-662.45\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1543.28,-664.881 1548.51,-655.667 1538.68,-659.608 1543.28,-664.881\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 36 -->\n",
       "<g class=\"node\" id=\"node37\"><title>36</title>\n",
       "<polygon fill=\"#399de5\" points=\"1446,-536.5 1349,-536.5 1349,-468.5 1446,-468.5 1446,-536.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1397.5\" y=\"-521.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1397.5\" y=\"-506.3\">samples = 1</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1397.5\" y=\"-491.3\">value = [0, 1]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1397.5\" y=\"-476.3\">class = benign</text>\n",
       "</g>\n",
       "<!-- 35&#45;&gt;36 -->\n",
       "<g class=\"edge\" id=\"edge36\"><title>35-&gt;36</title>\n",
       "<path d=\"M1413.17,-579.907C1410.98,-569.204 1408.6,-557.615 1406.38,-546.776\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1409.74,-545.76 1404.3,-536.667 1402.88,-547.166 1409.74,-545.76\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 37 -->\n",
       "<g class=\"node\" id=\"node38\"><title>37</title>\n",
       "<polygon fill=\"#e58139\" points=\"1577,-536.5 1464,-536.5 1464,-468.5 1577,-468.5 1577,-536.5\" stroke=\"black\"/>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1520.5\" y=\"-521.3\">gini = 0.0</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1520.5\" y=\"-506.3\">samples = 4</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1520.5\" y=\"-491.3\">value = [4, 0]</text>\n",
       "<text font-family=\"Times New Roman,serif\" font-size=\"14.00\" text-anchor=\"middle\" x=\"1520.5\" y=\"-476.3\">class = malignant</text>\n",
       "</g>\n",
       "<!-- 35&#45;&gt;37 -->\n",
       "<g class=\"edge\" id=\"edge37\"><title>35-&gt;37</title>\n",
       "<path d=\"M1455.85,-579.907C1465.56,-568.432 1476.13,-555.938 1485.86,-544.442\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"1488.65,-546.562 1492.44,-536.667 1483.3,-542.04 1488.65,-546.562\" stroke=\"black\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>svg{width:40% !important;height:40% !important;}</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph = Source(export_graphviz(estimators[-1], # najzlozitejsi model\n",
    "                               out_file=None,\n",
    "                               feature_names=feature_names,\n",
    "                               class_names=label_names,\n",
    "                               filled = True))\n",
    "\n",
    "display(SVG(graph.pipe(format='svg')))\n",
    "\n",
    "from IPython.display import HTML # toto je tu len pre to aby sa mi obrazok zmestil na obrazovku\n",
    "style = \"<style>svg{width:40% !important;height:40% !important;}</style>\"\n",
    "HTML(style)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Existujú viaceré spôsoby ako riešiť pretrénovanie a kontrolovať varianciu a bias\n",
    "\n",
    "## Získať viac dát :)\n",
    "\n",
    "## Učenie súborom metód (ensemble learning)\n",
    "toto nemusíte teraz vedieť, ale je dobre ak poznáte, že také niečo existuje\n",
    "\n",
    "* **Bagging** \n",
    "\n",
    "> Kombinovanie predikcii **nezávislých** modelov do jednej predikcie. Každý model musí byť **lepší ako náhoda**.\n",
    "\n",
    "> Používa sa na zníženie variancie\n",
    "\n",
    "* **Boosting** \n",
    "\n",
    "> Používa na zníženie biasu. Pozor, môže zvýšiť varianciu!\n",
    "\n",
    "> Iteratívne trénovanie ďalších klasifikátorov so zvýšenou váhou na tie pozorovania, na ktorých sa predchádzajúcim modelom nedarilo\n",
    "\n",
    "\n",
    "## Zjednodušenie modelu\n",
    "* Výber atribútov\n",
    "* Regularizácia\n",
    "* Orezávanie stromov\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Výber atribútov"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Prečo by som mal vyberať len niektoré atribúty?\n",
    "\n",
    "* redundancia - skryté závislosti medzi nimi \n",
    "* irelevancia - nemusia mat žiadny vplyv na predikovanú hodnotu\n",
    "* pretrénovanie - model sa dá natrénovať aj na náhodných dátach a na trénovanej sade bude fungovať. Na testovacej ale bude fungovať úplne strašne\n",
    "* prekliatie dimenzionality - pri veľkom počte atribútov potrebujem veľa dát na to aby som dostatočne pokryl priestor možných hodnôt\n",
    "* produktivita / rýchlosť - moja ako analytika a aj mojich modelov (trénovanie aj predikcia)\n",
    "* zrozumiteľnosť - ľahšie sa vysvetľuje model, ktorý používa menej atribútov\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Existujú 3 skupiny prístupov na výber atribútov\n",
    "\n",
    "> ## 1. Filter\n",
    "\n",
    "> ## 2. Wrapper\n",
    "\n",
    "> ## 3. Embedded\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 1. Filter\n",
    "\n",
    "Výber atribútov bez ohľadu na model, ktorý sa chystáme trénovať.\n",
    "\n",
    "Výber atribútov na základe zvolenej metriky atribútu.\n",
    "\n",
    "* rýchle\n",
    "* nezávislé na modeli (to je dobre aj zle: môžem ich použiť na rôzne modely, ale nedokážu využiť špecifické vlastnosti konkrétnych algoritmov)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Najjednoduchšia možnosť je vyhodiť atribúty, ktoré majú všade rovnaké hodnoty\n",
    "\n",
    "Pozor, nie malú varianciu. Hlavne pri nevyvážených triedach môžu byt práve takéto atribúty veľmi užitočné\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 2, 0, 3],\n",
       "       [0, 1, 4, 3],\n",
       "       [0, 1, 1, 3]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_selection import VarianceThreshold\n",
    "\n",
    "X = np.array([[0, 2, 0, 3], [0, 1, 4, 3], [0, 1, 1, 3]])\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 0],\n",
       "       [1, 4],\n",
       "       [1, 1]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "selector = VarianceThreshold(threshold=0.0)\n",
    "selector.fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Môžeme vyberať atribúty aj na základe závislosti atribútu a predikovanej hodnoty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 4)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from sklearn.feature_selection import SelectKBest\n",
    "from sklearn.feature_selection import chi2 # daju sa pouzit aj ine metriky\n",
    "\n",
    "iris = load_iris()\n",
    "X, y = iris.data, iris.target\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Metriku ako napríklad Chi kvadrát použijeme na kvantifikovanie závislosti k predikovanej premennej.\n",
    "\n",
    "Najlepšie atribúty necháme\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 2)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_new = SelectKBest(chi2, k=2).fit_transform(X, y)\n",
    "X_new.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Dajú sa používať rôzne metriky\n",
    "> Klasifikácia\n",
    "* chi2 - nezáporné čísla\n",
    "* mutual_info_classif - diskrétne dáta\n",
    "* f_classif - ANOVA medzi predikovanou premennou a atribútmi\n",
    "\n",
    "> Regresia\n",
    "* f_regression - F test medzi predikovanou hodnotou a atribútmi\n",
    "* mutual_info_regression - Mutual information na reálnych číslach\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Dá sa vyberať K najlepších alebo nejaký percentil alebo nechať počet atribútov na štatistický test\n",
    "\n",
    "* SelectKBest \n",
    "* SelectPercentile\n",
    "\n",
    "* SelectFpr - false positive rate\n",
    "* SelectFdr - false discovery rate  \n",
    "* SelectFwe - family wise error\n",
    "\n",
    "* GenericUnivariateSelect - Všetko dohromady a stratégia sa dá nastaviť parametrom\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Vlastnosti filtrov\n",
    "\n",
    "* väčšinou rýchle\n",
    "* nezávislé na modely (nepotrebujem opakovane trénovať model ale vybrané atribúty nemusia byt najvhodnejšie pre každý model)\n",
    "* väčšinou sa pozerajú len na vlastnosti dvojíc predikovaná premenná - atribút, kombinácie viacerých atribútov nezohľadňujú\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Varovanie, PCA sa často používa na redukciu dimenzionality ale nie na výber atribútov\n",
    "\n",
    "Je to častá chyba\n",
    "\n",
    "PCA **vytvára nové** atribúty ako **lineárnu kombináciu** pôvodných. Nevyberá z pôvodných.\n",
    "\n",
    "Ak použijeme dôležitosť atribútov, ktoré vstupujú do PCA, tak získame len ich varianciu. To je ekvivalentné s VarianceThreshold spomenutým vyššie. Aby sme toto získali, tak nemusíme počítať celé PCA. Stačí nám variancia každého atribútu oddelene. \n",
    "\n",
    "Navyše do PCA by mali vstupovať dáta s normalizovanou varianciou, takže by sme namiesto variance score pre každý atribút dostali stále rovnakú hodnotu.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 2. Wrapper"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Základná myšlienka\n",
    "\n",
    "hľadáme podmnožinu atribútov, na ktorej bude model dávať najlepšie výsledky\n",
    "\n",
    "Skúšame rôzne podmnožiny, opakovane trénujeme nové modely a vyberáme tú najlepšiu podmnožinu, na ktorej model funguje najlepšie\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Problém\n",
    "\n",
    "Ak máme dataset s N atribútmi, tak počet rôznych podmnožín je $2^N$\n",
    "\n",
    "To znamená, že by sme museli náš model natrénovať $2^N$ krát.\n",
    "\n",
    "Chcelo by to nájsť proces, ktorý minimalizuje počet pokusov a zároveň maximalizuje úspešnosť modelu\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Greedy prístupy\n",
    "\n",
    "Najčastejšie sa používajú greedy prístupy, ktoré postupne zväčšujú sadu atribútov (alebo zmenšujú) tak, že pridávajú (odoberajú) atribút tak aby sa čo najviac zvýšila (najmenej znížila) úspešnosť. Skúšajú rôzne podmnožiny atribútov, ktoré postupne zväčšujú alebo zmenšujú.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Knižnica Mlxtend\n",
    "\n",
    "* Sequential Forward Selection (SFS)\n",
    "> Postupne zväčšuje množinu atribútov o ten, ktorý najviac prispel k zlepšeniu\n",
    "\n",
    "> Začína s množinami veľkosti 1, vyberie najlepšiu a zafixuje atribút. K fixovanému atribútu pridá ďalší a vytvorí všetky možne podmnožiny veľkosti 2 s jedným zafixovaným atribútom. Vyberie najlepšiu ...\n",
    "\n",
    "* Sequential Backward Selection (SBS)\n",
    "> Postupne zmenšuje množinu atribútov o ten, ktorý najmenej pomáhal.\n",
    "\n",
    "* Sequential Floating Forward Selection (SFFS)\n",
    "> SFS s pokusom o vyhodenie už pridaných atribútov ak sa ukáže že veľmi nepomáhajú \n",
    "\n",
    "* Sequential Floating Backward Selection (SFBS)\n",
    "> SBS s pokusom o pridanie už raz vyhodeného atribútu\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Scikit-Learn\n",
    "\n",
    "* RFE - Recursive feature elimination\n",
    "> Postupne vyhadzovanie atribútov, ktoré majú v modeli najnižšiu váhu (potrebujeme aby to model vedel povedať) \n",
    "\n",
    "* RFECV - RFE with cross-validation\n",
    "> RFE s krízovou validáciou\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Priklad SFS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.datasets import load_iris\n",
    "\n",
    "iris = load_iris()\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "knn = KNeighborsClassifier(n_neighbors=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:    0.0s finished\n",
      "\n",
      "[2019-10-26 14:11:38] Features: 1/3 -- score: 0.96[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:    0.0s finished\n",
      "\n",
      "[2019-10-26 14:11:38] Features: 2/3 -- score: 0.9733333333333334[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s finished\n",
      "\n",
      "[2019-10-26 14:11:38] Features: 3/3 -- score: 0.9733333333333334"
     ]
    }
   ],
   "source": [
    "from mlxtend.feature_selection import SequentialFeatureSelector as SFS\n",
    "\n",
    "sfs1 = SFS(knn, k_features=3, forward=True,  floating=False, verbose=2, scoring='accuracy', cv=0)\n",
    "# pomocou tejto triedy vieme robit SFS, SFFS, SBS aj SFBS a dokonca aj pridat cross-validaciu\n",
    "\n",
    "sfs1 = sfs1.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Zoznamy najlepších podmnožín atribútov pre jednotlivé veľkosti podmnožín"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{1: {'feature_idx': (3,),\n",
       "  'cv_scores': array([0.96]),\n",
       "  'avg_score': 0.96,\n",
       "  'feature_names': ('3',)},\n",
       " 2: {'feature_idx': (2, 3),\n",
       "  'cv_scores': array([0.97333333]),\n",
       "  'avg_score': 0.9733333333333334,\n",
       "  'feature_names': ('2', '3')},\n",
       " 3: {'feature_idx': (1, 2, 3),\n",
       "  'cv_scores': array([0.97333333]),\n",
       "  'avg_score': 0.9733333333333334,\n",
       "  'feature_names': ('1', '2', '3')}}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sfs1.subsets_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 3. Embedded"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Hlavná myšlienka\n",
    "\n",
    "Skombinovať výhody filtrov a wrapprov\n",
    "\n",
    "Model, ktorý sa trénuje si bude priamo vyberať atribúty, ktoré sú pre neho najlepšie\n",
    "\n",
    "Len málo modelov to podporuje\n",
    "\n",
    "* Lineárne modely penalizované L1 (Lasso) alebo L1+L2 (Elastic Net) regularizáciou: SVM, Lineárna regresia, Logistická regresia ...\n",
    "\n",
    "> Regularizácia zavádza do modelu penalizáciu za počet / veľkosť váh atribútov modelu. Nie je tam len chyba predikcie. Prirodzene sa tak vyberá jednoduchší model.\n",
    "\n",
    "* RandomForest\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 4)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.feature_selection import SelectFromModel\n",
    "iris = load_iris()\n",
    "X, y = iris.data, iris.target\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Po natrénovaní náhodného lesu viem vybrať dôležitosť atribútov.\n",
    "\n",
    "Náhodný les je zjednodušene povedane: skupina stromov.\n",
    "\n",
    "Viem sa pozrieť aké atribúty používajú stromy na rozhodovanie a ako vysoko sú v nich a podľa toho odvodiť ich dôležitosť.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\sevo_\\AppData\\Roaming\\Python\\Python37\\site-packages\\sklearn\\ensemble\\forest.py:245: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([0.05472823, 0.01440036, 0.37523871, 0.5556327 ])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = RandomForestClassifier()\n",
    "clf = clf.fit(X, y)\n",
    "clf.feature_importances_  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 2)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = SelectFromModel(clf, prefit=True)\n",
    "X_new = model.transform(X)\n",
    "X_new.shape  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Čo si odniesť z prednášky?\n",
    "\n",
    "* Rozhodovacie stromy sú pomerne jednoduchý ale zároveň veľmi silný nástroj.\n",
    "\n",
    "* Pri stromoch sa veľmi dobre interpretuje natrénovaný model pomocou pravidiel.\n",
    "\n",
    "* Pozor na pretrénovanie (nie len pri stromoch)\n",
    "\n",
    "* Výber atribútov je dobrý na redukciu problému prekliatia dimenzionality.\n",
    "\n",
    "> * Ak používate nejaký lineárny model alebo les, tak je zbytočné robiť filtre a ešte viac zbytočné robiť wrappre. Môžete využiť to, že sa dajú použiť na embedded výber atribútov\n",
    "\n",
    "> * Ak nemáte čas na opakované trénovanie modelu, tak filtre môžu byt dostatočný hotfix. Treba ale zvážiť akú vlastnosť atribútov chcete použiť na nájdenie tých najdôležitejších. \n",
    "\n",
    "> * Ak máte čas spustiť trénovanie viac krát, tak asi najlepšia možnosť je SFFS alebo SFECV\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"img/potato-tree.jpg\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}