03_exercise_scikit_learn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table>\n",
    "<tr>\n",
    "  <td><img src=\"figures/iris_setosa.jpg\"></td>\n",
    "  <td><img src=\"figures/iris_versicolor.jpg\"></td>\n",
    "  <td><img src=\"figures/iris_virginica.jpg\"></td>\n",
    "</tr>\n",
    "\n",
    "<tr>\n",
    "  <td>Iris Setosa</td><td>Iris Versicolor</td><td>Iris Virginica</td>\n",
    "</tr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From <a href=\"https://en.wikipedia.org/wiki/Iris_flower_data_set\">Wikipedia</a>:\n",
    "\n",
    "The Iris flower data set or Fisher's Iris data set is a multivariate data set introduced by Ronald Fisher in his 1936 paper The use of multiple measurements in taxonomic problems as an example of linear discriminant analysis. It is sometimes called Anderson's Iris data set because Edgar Anderson collected the data to quantify the morphologic variation of Iris flowers of three related species. Two of the three species were collected in the Gaspé Peninsula \"all from the same pasture, and picked on the same day and measured at the same time by the same person with the same apparatus\".\n",
    "\n",
    "<img src=\"figures/petal_sepal.jpg\" alt=\"Sepal\" style=\"width: 25%; float: left; padding: 1em;\"/>\n",
    "\n",
    "The data set consists of 50 samples from each of three species of Iris (Iris setosa, Iris virginica and Iris versicolor). Four features were measured from each sample: the length and the width of the sepals and petals, in centimetres. Based on the combination of these four features, Fisher developed a linear discriminant model to distinguish the species from each other.\n",
    "\n",
    "<br/>\n",
    "\"Petal-sepal\". Licensed under CC BY-SA 3.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Petal-sepal.jpg#/media/File:Petal-sepal.jpg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import pandas as pd\n",
    "import sklearn\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise\n",
    "\n",
    "In this exercise, we train a classifier to tell us the type of a flower, based on several of its measured properties."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the data\n",
    "\n",
    "1. Using the ``read_csv`` command in Pandas, load the dataset in the file ``data/iris.csv``.\n",
    "2. Use the ``head`` method of the resulting DataFrame to see the first few entries.\n",
    "3. Use the ``tail`` method to see the last few entries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "iris = ...\n",
    "iris.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "iris.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examine the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use ``groupby`` to group measurements by flower name and then display summary statistics.  Look at the numbers, and see if you can identify any big differences between species:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "iris.groupby('Name').describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the data\n",
    "\n",
    "- Use Seaborn's ``pairplot`` function to display the different features of the data.\n",
    "- Remember to set the keyword `hue` to the correct column name so that the classes are colored separately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that this dataset should be very easy to classify."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, use the ``pop`` method of the DataFrame to grab the labels from the Name column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data = iris.copy()\n",
    "labels = ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "labels[:15]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Transform labels to numbers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's transform those text labels to numbers (0, 1, or 2) respectively.  Strictly speaking, this isn't necessary, since NumPy arrays can contain objects, and comparisons work fine on those.  But in general, working with numbers speed things up a lot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "encoder = LabelEncoder()\n",
    "labels = ...\n",
    "labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Split data into training & testing data\n",
    "\n",
    "Using ``sklearn.cross_validation.train_test_split``, split the dataset and labels into two: training data and testing data.\n",
    "\n",
    "**Since this is such an easy problem, we only want 0.1% of the samples to go towards training.**\n",
    "\n",
    "Note, it's important to pass both data and labels into the ``train_test_split`` function together so that they can be split the same way (i.e., do not run ``train_test_split`` once for each).\n",
    "\n",
    "**Hint:** *If you set the ``random_state`` variable, you will always get the same split, and your experiments will be comparable.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data_train, data_test, labels_train, labels_test = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the lengths of the training and testing data and labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "len(data_train), len(data_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classify and evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Gaussian Naive Bayes classifier is a good entry-level benchmark"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "nb = GaussianNB()\n",
    "nb.fit(...)\n",
    "labels_predicted_nb = ...\n",
    "accuracy_score(labels_test, labels_predicted_nb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now, try the evaluation, but with a RandomForest classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "rf = RandomForestClassifier(random_state=42)\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's look at the feature importances, just for the fun of it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "list(zip(iris.columns, rf.feature_importances_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Extra/advanced exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we're going to work with a digits dataset.  Each digit is represented as a matrix, with values representing intensity (0 is black, 16 is white).  To form a feature vector, the matrix is \"unravelled\", i.e. all the values are unpacked from the matrix into a single, long vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_digits\n",
    "digits = load_digits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(6, 6))  # figure size in inches\n",
    "fig.subplots_adjust(left=0, right=1, bottom=0, top=1, hspace=0.05, wspace=0.05)\n",
    "\n",
    "# plot the digits: each image is 8x8 pixels\n",
    "for i in range(64):\n",
    "    ax = fig.add_subplot(8, 8, i + 1, xticks=[], yticks=[])\n",
    "    ax.imshow(digits.images[i], cmap=plt.cm.binary, interpolation='nearest')\n",
    "    \n",
    "    # label the image with the target value\n",
    "    ax.text(0, 7, str(digits.target[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualization\n",
    "\n",
    "A good first-step for many problems is to visualize the data using one of the\n",
    "*Dimensionality Reduction* techniques we saw earlier.  We'll start with the\n",
    "most straightforward one, Principal Component Analysis (PCA).\n",
    "\n",
    "PCA seeks orthogonal linear combinations of the features which show the greatest\n",
    "variance, and as such, can help give you a good idea of the structure of the\n",
    "data set.  Here we'll use `RandomizedPCA`, because it's faster for large `N`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.decomposition import RandomizedPCA\n",
    "pca = RandomizedPCA(n_components=2, random_state=1999)\n",
    "proj = pca.fit_transform(digits.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.scatter(proj[:, 0], proj[:, 1], c=digits.target, cmap='RdBu')\n",
    "plt.colorbar();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we see that the digits do cluster fairly well, so we can expect even\n",
    "a fairly naive classification scheme to do a decent job separating them.\n",
    "\n",
    "A weakness of PCA is that it produces a linear dimensionality reduction:\n",
    "this may miss some interesting relationships in the data.  If we want to\n",
    "see a nonlinear mapping  of the data, we can use one of the several\n",
    "methods in the `manifold` module.  Here we'll use Isomap (ISOmetric MAPping)\n",
    "which is a manifold learning method based on graph theory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.manifold import Isomap\n",
    "iso = Isomap(n_neighbors=5, n_components=2)\n",
    "proj = iso.fit_transform(digits.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.scatter(proj[:, 0], proj[:, 1], c=digits.target, cmap='RdBu')\n",
    "plt.colorbar();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These visualizations show that it classification should be possible!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now, train classifiers as we did with the Iris dataset, and see how they do!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}