Intro_to_pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro to pandas\n",
    "https://colab.research.google.com/notebooks/mlcc/intro_to_pandas.ipynb?utm_source=mlcc&utm_campaign=colab-external&utm_medium=referral&utm_content=pandas-colab&hl=en\n",
    "\n",
    "## Basic concepts\n",
    "To import *panads* API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "\n",
    "import pandas as pd\n",
    "pd.__version__\n",
    "\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The primary data structures in *pandas* are implemented as two classes:\n",
    "  **DataFrame**, can imagine as a relational data table with rows and named columns\n",
    "  **Series**, which is asingle column\n",
    "  \n",
    "A `DataFrame` contains one or more `Series` and a name for each `Series`\n",
    "\n",
    "The data frame is a commonly ised abstraction for data manipulation (like what you can do is Spark and R)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can create a `Series` by constructing a `Series` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    San Fransisco\n",
       "1         San Jose\n",
       "2       Sacremento\n",
       "dtype: object"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(['San Fransisco', 'San Jose', 'Sacremento'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`DataFrame` objects can be created by passing a `dict` mapping `string` column names to their respective `Series`. If the `Series` don't match in length, missing values are filled with NA/NaN values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City name</th>\n",
       "      <th>Population</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>San Fransisco</td>\n",
       "      <td>852469</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>San Jose</td>\n",
       "      <td>1015785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sacremento</td>\n",
       "      <td>485199</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       City name  Population\n",
       "0  San Fransisco      852469\n",
       "1       San Jose     1015785\n",
       "2     Sacremento      485199"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city_names = pd.Series(['San Fransisco', 'San Jose', 'Sacremento'])\n",
    "population = pd.Series([852469, 1015785, 485199])\n",
    "\n",
    "pd.DataFrame({ 'City name': city_names, 'Population': population})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But most of the time, you load an entrie file into a `DataFrame`. The following example loads a file with California housing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>17000.000000</td>\n",
       "      <td>17000.000000</td>\n",
       "      <td>17000.000000</td>\n",
       "      <td>17000.000000</td>\n",
       "      <td>17000.000000</td>\n",
       "      <td>17000.000000</td>\n",
       "      <td>17000.000000</td>\n",
       "      <td>17000.000000</td>\n",
       "      <td>17000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-119.562108</td>\n",
       "      <td>35.625225</td>\n",
       "      <td>28.589353</td>\n",
       "      <td>2643.664412</td>\n",
       "      <td>539.410824</td>\n",
       "      <td>1429.573941</td>\n",
       "      <td>501.221941</td>\n",
       "      <td>3.883578</td>\n",
       "      <td>207300.912353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2.005166</td>\n",
       "      <td>2.137340</td>\n",
       "      <td>12.586937</td>\n",
       "      <td>2179.947071</td>\n",
       "      <td>421.499452</td>\n",
       "      <td>1147.852959</td>\n",
       "      <td>384.520841</td>\n",
       "      <td>1.908157</td>\n",
       "      <td>115983.764387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-124.350000</td>\n",
       "      <td>32.540000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.499900</td>\n",
       "      <td>14999.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-121.790000</td>\n",
       "      <td>33.930000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>1462.000000</td>\n",
       "      <td>297.000000</td>\n",
       "      <td>790.000000</td>\n",
       "      <td>282.000000</td>\n",
       "      <td>2.566375</td>\n",
       "      <td>119400.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-118.490000</td>\n",
       "      <td>34.250000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2127.000000</td>\n",
       "      <td>434.000000</td>\n",
       "      <td>1167.000000</td>\n",
       "      <td>409.000000</td>\n",
       "      <td>3.544600</td>\n",
       "      <td>180400.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>-118.000000</td>\n",
       "      <td>37.720000</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>3151.250000</td>\n",
       "      <td>648.250000</td>\n",
       "      <td>1721.000000</td>\n",
       "      <td>605.250000</td>\n",
       "      <td>4.767000</td>\n",
       "      <td>265000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>-114.310000</td>\n",
       "      <td>41.950000</td>\n",
       "      <td>52.000000</td>\n",
       "      <td>37937.000000</td>\n",
       "      <td>6445.000000</td>\n",
       "      <td>35682.000000</td>\n",
       "      <td>6082.000000</td>\n",
       "      <td>15.000100</td>\n",
       "      <td>500001.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          longitude      latitude  housing_median_age   total_rooms  \\\n",
       "count  17000.000000  17000.000000        17000.000000  17000.000000   \n",
       "mean    -119.562108     35.625225           28.589353   2643.664412   \n",
       "std        2.005166      2.137340           12.586937   2179.947071   \n",
       "min     -124.350000     32.540000            1.000000      2.000000   \n",
       "25%     -121.790000     33.930000           18.000000   1462.000000   \n",
       "50%     -118.490000     34.250000           29.000000   2127.000000   \n",
       "75%     -118.000000     37.720000           37.000000   3151.250000   \n",
       "max     -114.310000     41.950000           52.000000  37937.000000   \n",
       "\n",
       "       total_bedrooms    population    households  median_income  \\\n",
       "count    17000.000000  17000.000000  17000.000000   17000.000000   \n",
       "mean       539.410824   1429.573941    501.221941       3.883578   \n",
       "std        421.499452   1147.852959    384.520841       1.908157   \n",
       "min          1.000000      3.000000      1.000000       0.499900   \n",
       "25%        297.000000    790.000000    282.000000       2.566375   \n",
       "50%        434.000000   1167.000000    409.000000       3.544600   \n",
       "75%        648.250000   1721.000000    605.250000       4.767000   \n",
       "max       6445.000000  35682.000000   6082.000000      15.000100   \n",
       "\n",
       "       median_house_value  \n",
       "count        17000.000000  \n",
       "mean        207300.912353  \n",
       "std         115983.764387  \n",
       "min          14999.000000  \n",
       "25%         119400.000000  \n",
       "50%         180400.000000  \n",
       "75%         265000.000000  \n",
       "max         500001.000000  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "california_housing_dataframe = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_train.csv\", sep=\",\")\n",
    "california_housing_dataframe.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`DataFrame.describe` shows intersting statistics about a `DataFrame`. Another useful function is `DataFrame.head`, which displays the first few records of a `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-114.31</td>\n",
       "      <td>34.19</td>\n",
       "      <td>15.0</td>\n",
       "      <td>5612.0</td>\n",
       "      <td>1283.0</td>\n",
       "      <td>1015.0</td>\n",
       "      <td>472.0</td>\n",
       "      <td>1.4936</td>\n",
       "      <td>66900.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-114.47</td>\n",
       "      <td>34.40</td>\n",
       "      <td>19.0</td>\n",
       "      <td>7650.0</td>\n",
       "      <td>1901.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>463.0</td>\n",
       "      <td>1.8200</td>\n",
       "      <td>80100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-114.56</td>\n",
       "      <td>33.69</td>\n",
       "      <td>17.0</td>\n",
       "      <td>720.0</td>\n",
       "      <td>174.0</td>\n",
       "      <td>333.0</td>\n",
       "      <td>117.0</td>\n",
       "      <td>1.6509</td>\n",
       "      <td>85700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-114.57</td>\n",
       "      <td>33.64</td>\n",
       "      <td>14.0</td>\n",
       "      <td>1501.0</td>\n",
       "      <td>337.0</td>\n",
       "      <td>515.0</td>\n",
       "      <td>226.0</td>\n",
       "      <td>3.1917</td>\n",
       "      <td>73400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-114.57</td>\n",
       "      <td>33.57</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1454.0</td>\n",
       "      <td>326.0</td>\n",
       "      <td>624.0</td>\n",
       "      <td>262.0</td>\n",
       "      <td>1.9250</td>\n",
       "      <td>65500.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   longitude  latitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0    -114.31     34.19                15.0       5612.0          1283.0   \n",
       "1    -114.47     34.40                19.0       7650.0          1901.0   \n",
       "2    -114.56     33.69                17.0        720.0           174.0   \n",
       "3    -114.57     33.64                14.0       1501.0           337.0   \n",
       "4    -114.57     33.57                20.0       1454.0           326.0   \n",
       "\n",
       "   population  households  median_income  median_house_value  \n",
       "0      1015.0       472.0         1.4936             66900.0  \n",
       "1      1129.0       463.0         1.8200             80100.0  \n",
       "2       333.0       117.0         1.6509             85700.0  \n",
       "3       515.0       226.0         3.1917             73400.0  \n",
       "4       624.0       262.0         1.9250             65500.0  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "california_housing_dataframe.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANother powerful feature of *pandas* is graphing. For example, `DataFrame.hist` lets you quickly study the distribution of values in a column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[<matplotlib.axes._subplots.AxesSubplot object at 0x000000000B8FB1D0>]],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGM5JREFUeJzt3X+Q3HV9x/HnS36JhCHhR64hiV5aggqNpnoGlM70QIUI\ntsGpOKERAqJpZ8IMtrE1OGNBMS12RBwq0sYSCVaNqUBJAQsxsqW2AyRBSggpcsIJISERE34cIO3R\nd//4fk6+Hnu3e7m93bv9vB4zN/fdz/ez3+/nfbu3r/3+2O8qIjAzs/y8rtUDMDOz1nAAmJllygFg\nZpYpB4CZWaYcAGZmmXIAmJllygFgoyKpV9L7mri+Pkm/2az1NZqkkHRMmv47SZ9t9ZgsX/u3egBm\nIxERk1o9hkaJiD9p9Rgsb94CMDPLlAPAGmGupAckPSvpu5JeDyDpE5J6JO2RtE7S0am9M+0K+dUW\nqKSKpI+n6WMk/Vta3tOSvlvqV96Fcp2kqyXdKul5SfdI+q1S31MlPZyW87W0zI8PV4ik8yT9h6Qr\nJT0j6VFJ70ntT0jaLWlxqf9Bkr4k6XFJu9JunYNL8/9c0k5JOyR9bNC6rpP0hTQ9RdItkn4uaW+a\nnjHo73NZGtvzku6QdGStB0bSP0l6Kv0N7pJ0fGneEZL+RdJzkjZK+oKkH5Xmv0XS+vT4PSzpI7XW\nZxOLA8Aa4SPAfGAW8DbgPEmnAH+d5k0DfgasqXN5lwF3AFOAGcDfDtP3bOBzqW8PsAIgvTh+D7gY\nOAJ4GHhPnes/AXgg3e/badzvAo4BPgp8VdLArqgvAscCc9P86cBfpjHMBz4FvB+YDQx3rOR1wDeA\nNwFvBF4Cvjqozx8B5wNTgQPTsmv5flr3VOA+4FuleVcDLwC/ASxOP6SxHwKsT/VPpfg7f60cIDbx\nOQCsEa6KiB0RsQf4F4oXw0XAqoi4LyJepnghfrekzjqW978UL4RHR8QvI+JHw/S9MSLujYh+ihe3\nuan9dGBrRNyY5l0FPFVnPY9FxDci4hXgu8BM4PMR8XJE3AH8D3CMJAGfAP40IvZExPPAXwEL03I+\nAnwjIh6MiBeAS4daYUT8IiJuiIgX03JWAL83qNs3IuInEfESsLZU65AiYlVEPJ8eg0uBt0s6TNJ+\nwB8Cl6R1PgSsLt31g0Bv+jv0R8R9wA3Ah2ut0yYOB4A1QvmF9UVgEnA0xbt+ACKiD/gFxTvkWv4C\nEHCvpK2Dd53UsW7S+p8orT+A7XWsG2BXafqldP/BbZOAo4A3AJvT7qJngH9N7a8ZA6W/x2CS3iDp\n7yX9TNJzwF3A5PRCPWCoWoda5n6SLpf007TM3jTryDTG/QeNrzz9JuCEgbpSbYsothasTfgsIBsr\nOyheRIBf7VI4AniSYrcDFC+ez6XpX72wRMRTFO+skfS7wA8k3RURPSNY/06K3UcD61f5doM8TREG\nx0fEk0OMYWbp9huHWdYy4M3ACRHxlKS5wI8pgnBf/RGwgGLXUy9wGLA3LfPnQD/F3+QnqX95rE8A\n/xYR7x/F+m2c8xaAjZVvA+dLmivpIIpdI/dERG9E/JwiCD6a3qV+DCgfvD2rdAB0LxDAKyNc/63A\nHElnpoPNS2nwu9eI+D/g68CVkqYCSJou6bTUZS3F8ZDjJL0BuGSYxR1KESbPSDq8Rt96HQq8TLHl\n9QaKx2Bg7K8ANwKXpq2PtwDnlu57C3CspHMkHZB+3iXprQ0Yl40TDgAbExGxAfgsxX7jnRQv8AtL\nXT4B/DnFi9PxwH+W5r0LuEdSH7AOuCgiHhvh+p8GzgL+Jq3jOGATxQtiI32a4uDz3Wk3yw8o3skT\nEd8HvgL8MPX54TDL+QpwMMVWxd0Uu5JG63qK3U5PAg+l5ZZdSLFV8BTwTeA7pL9POg5xKsVjtiP1\n+SJwUAPGZeOE/IUwlgNJr6M4BrAoIu5s9XjGI0lfBH4jIhbX7GxtwVsA1rYknSZpctoF9RmKfd+D\n3wVnK53n/zYV5gEXADe1elzWPA4Aa2fvBn5KsVvl94EzI+Kl9GGtvio/f9fa4Y6cpEVD1LK1jrsf\nSnEc4AWK4xVXADeP5XhtfPEuIDOzTHkLwMwsU+P6cwBHHnlkdHZ21uz3wgsvcMghh4z9gMaBnGoF\n19vOcqoVmlvv5s2bn46Io2r1G9cB0NnZyaZNm2r2q1QqdHd3j/2AxoGcagXX285yqhWaW6+kIT91\nXuZdQGZmmXIAmJllqmYASHq9pHsl/Ve6MNfnUvssFddff0TFNeAPTO0Hpds9aX5naVkXp/aHSx+X\nNzOzFqhnC+Bl4JSIeDvF5WfnSzqR4mPhV0bEbIrrtVyQ+l8A7I2IY4ArUz8kHUfxsfLjKa4d/7VB\nVzo0M7MmqhkAUehLNw9IPwGcQvGFG1BcR/zMNL2AV68r/j3gvelKjAuANema6o9RXBtlXkOqMDOz\nEavrLKD0Tn0zxTceXU3x6cpn0hdtQHGNlYHrvE8nXVc8IvolPUtxGeDp/PrH8Mv3Ka9rCbAEoKOj\ng0qlUnN8fX19dfVrBznVCq63neVUK4zPeusKgHTp2LmSJlNcK6TaJWEHPlJc7frlMUz74HWtBFYC\ndHV1RT2nTeV0OllOtYLrbWc51Qrjs94RnQUUEc8AFeBEim8rGgiQGRSXjIXinf1MgDT/MGBPub3K\nfczMrMnqOQvoqPTOH0kHU3y70DbgTl79ftDFvHoRqXW8+uXSHwZ+mL6Obx2wMJ0lNIvii6rvbVQh\nZmY2MvXsApoGrE7HAV4HrI2IWyQ9BKyR9AWKr667NvW/FvimpB6Kd/4LASJiq6S1FF9M0Q8sTbuW\nzGwEOpff2rJ1915+RsvWbY1XMwAi4gHgd6q0P0qVs3gi4pcU38RUbVkrgBUjH6aZmTWaPwlsZpYp\nB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWaYcAGZmmXIAmJll\nygFgZpYpB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWaYcAGZm\nmXIAmJllygFgZpYpB4CZWaZqBoCkmZLulLRN0lZJF6X2SyU9Ken+9HN66T4XS+qR9LCk00rt81Nb\nj6TlY1OSmZnVY/86+vQDyyLiPkmHApslrU/zroyIL5U7SzoOWAgcDxwN/EDSsWn21cD7ge3ARknr\nIuKhRhRiZmYjUzMAImInsDNNPy9pGzB9mLssANZExMvAY5J6gHlpXk9EPAogaU3q6wAwM2sBRUT9\nnaVO4C7gt4E/A84DngM2UWwl7JX0VeDuiPjHdJ9rge+nRcyPiI+n9nOAEyLiwkHrWAIsAejo6Hjn\nmjVrao6rr6+PSZMm1V3HRJZTreB6q9ny5LNNGs1rzZl+WMOW5cd27Jx88smbI6KrVr96dgEBIGkS\ncAPwyYh4TtI1wGVApN9XAB8DVOXuQfXjDa9Jn4hYCawE6Orqiu7u7ppjq1Qq1NOvHeRUK7jeas5b\nfmtzBlNF76Luhi3Lj23r1RUAkg6gePH/VkTcCBARu0rzvw7ckm5uB2aW7j4D2JGmh2o3M7Mmq+cs\nIAHXAtsi4sul9mmlbh8CHkzT64CFkg6SNAuYDdwLbARmS5ol6UCKA8XrGlOGmZmNVD1bACcB5wBb\nJN2f2j4DnC1pLsVunF7gjwEiYquktRQHd/uBpRHxCoCkC4Hbgf2AVRGxtYG1mJnZCNRzFtCPqL5f\n/7Zh7rMCWFGl/bbh7mdmZs3jTwKbmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwA\nZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlqm6vxLSbDidY/Q1hcvm9A/7FYi9l58x\nJus1y4G3AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPL\nlC8FYRPaWF2Coh6+DIVNdDW3ACTNlHSnpG2Stkq6KLUfLmm9pEfS7ympXZKuktQj6QFJ7ygta3Hq\n/4ikxWNXlpmZ1VLPLqB+YFlEvBU4EVgq6ThgObAhImYDG9JtgA8As9PPEuAaKAIDuAQ4AZgHXDIQ\nGmZm1nw1AyAidkbEfWn6eWAbMB1YAKxO3VYDZ6bpBcD1UbgbmCxpGnAasD4i9kTEXmA9ML+h1ZiZ\nWd0UEfV3ljqBu4DfBh6PiMmleXsjYoqkW4DLI+JHqX0D8GmgG3h9RHwhtX8WeCkivjRoHUsothzo\n6Oh455o1a2qOq6+vj0mTJtVdx0Q2Xmvd8uSzY7LcjoNh10tjsuhRmzP9sIYvs57Hd6z+1vVoZM3j\n9bk8VppZ78knn7w5Irpq9av7ILCkScANwCcj4jlJQ3at0hbDtP96Q8RKYCVAV1dXdHd31xxbpVKh\nnn7tYLzWOtw1+0dj2Zx+rtgyPs9V6F3U3fBl1vP4jtXfuh6NrHm8PpfHynist67TQCUdQPHi/62I\nuDE170q7dki/d6f27cDM0t1nADuGaTczsxao5ywgAdcC2yLiy6VZ64CBM3kWAzeX2s9NZwOdCDwb\nETuB24FTJU1JB39PTW1mZtYC9WxbnwScA2yRdH9q+wxwObBW0gXA48BZad5twOlAD/AicD5AROyR\ndBmwMfX7fETsaUgVZi0wFp9BqPUVmGaNVDMA0sHcoXb4v7dK/wCWDrGsVcCqkQzQzMzGhi8FYWaW\nKQeAmVmmHABmZplyAJiZZcoBYGaWqfH5EUvbZ628PLKZTSzeAjAzy5QDwMwsUw4AM7NMOQDMzDLl\nADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NM\n+fsAzMyG0ajv2Fg2p5/zRrCs3svPaMh6h+MtADOzTDkAzMwy5QAwM8uUA8DMLFM1A0DSKkm7JT1Y\nartU0pOS7k8/p5fmXSypR9LDkk4rtc9PbT2Slje+FDMzG4l6zgK6DvgqcP2g9isj4kvlBknHAQuB\n44GjgR9IOjbNvhp4P7Ad2ChpXUQ8NIqxm1mTNeqMGBjZWTHNOCMmRzUDICLuktRZ5/IWAGsi4mXg\nMUk9wLw0ryciHgWQtCb1dQCYmbXIaD4HcKGkc4FNwLKI2AtMB+4u9dme2gCeGNR+QrWFSloCLAHo\n6OigUqnUHEhfX19d/dpBrVqXzelv3mCaoOPg9qtpODnVO5JaW/n/3ajHY6SPbTNq3tcAuAa4DIj0\n+wrgY4Cq9A2qH2uIaguOiJXASoCurq7o7u6uOZhKpUI9/dpBrVpH8kGTiWDZnH6u2JLP5xVzqnck\ntfYu6h7bwQyjUf9TI31sm1HzPj3TImLXwLSkrwO3pJvbgZmlrjOAHWl6qHYzM2uBfToNVNK00s0P\nAQNnCK0DFko6SNIsYDZwL7ARmC1plqQDKQ4Ur9v3YZuZ2WjV3AKQ9B2gGzhS0nbgEqBb0lyK3Ti9\nwB8DRMRWSWspDu72A0sj4pW0nAuB24H9gFURsbXh1ZiZWd3qOQvo7CrN1w7TfwWwokr7bcBtIxqd\nmZmNGX8S2MwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NMOQDMzDKVx0VHzGxCa+RlqO1V3gIw\nM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkA\nzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFM1A0DSKkm7JT1Yajtc0npJj6TfU1K7\nJF0lqUfSA5LeUbrP4tT/EUmLx6YcMzOrVz1bANcB8we1LQc2RMRsYEO6DfABYHb6WQJcA0VgAJcA\nJwDzgEsGQsPMzFqjZgBExF3AnkHNC4DVaXo1cGap/foo3A1MljQNOA1YHxF7ImIvsJ7XhoqZmTXR\n/vt4v46I2AkQETslTU3t04EnSv22p7ah2l9D0hKKrQc6OjqoVCo1B9PX11dXv3ZQq9Zlc/qbN5gm\n6Di4/WoaTk715lQrjLzeZrym7WsADEVV2mKY9tc2RqwEVgJ0dXVFd3d3zZVWKhXq6dcOatV63vJb\nmzeYJlg2p58rtjT6aTp+5VRvTrXCyOvtXdQ9doNJ9vUsoF1p1w7p9+7Uvh2YWeo3A9gxTLuZmbXI\nvgbAOmDgTJ7FwM2l9nPT2UAnAs+mXUW3A6dKmpIO/p6a2szMrEVqbo9I+g7QDRwpaTvF2TyXA2sl\nXQA8DpyVut8GnA70AC8C5wNExB5JlwEbU7/PR8TgA8tmZtZENQMgIs4eYtZ7q/QNYOkQy1kFrBrR\n6CaozjHcD79sTn/b7ec3s9bwJ4HNzDLlADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4A\nM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QD\nwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NM7d/qAYylzuW3tnoIZmbjlrcAzMwyNaoAkNQr\naYuk+yVtSm2HS1ov6ZH0e0pql6SrJPVIekDSOxpRgJmZ7ZtGbAGcHBFzI6Ir3V4ObIiI2cCGdBvg\nA8Ds9LMEuKYB6zYzs300FruAFgCr0/Rq4MxS+/VRuBuYLGnaGKzfzMzqMNoACOAOSZslLUltHRGx\nEyD9nprapwNPlO67PbWZmVkLjPYsoJMiYoekqcB6Sf89TF9VaYvXdCqCZAlAR0cHlUql5iD6+vqq\n9ls2p7/mfSeajoPbs66huN72lVOtMPJ663ntG61RBUBE7Ei/d0u6CZgH7JI0LSJ2pl08u1P37cDM\n0t1nADuqLHMlsBKgq6sruru7a46jUqlQrd95bXga6LI5/Vyxpa3P3v01rrd95VQrjLze3kXdYzeY\nZJ93AUk6RNKhA9PAqcCDwDpgceq2GLg5Ta8Dzk1nA50IPDuwq8jMzJpvNPHbAdwkaWA5346If5W0\nEVgr6QLgceCs1P824HSgB3gROH8U6zYzs1Ha5wCIiEeBt1dp/wXw3irtASzd1/WZmVlj+ZPAZmaZ\ncgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZ\nphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBm\nlikHgJlZphwAZmaZcgCYmWXKAWBmlqmmB4Ck+ZIeltQjaXmz129mZoWmBoCk/YCrgQ8AxwFnSzqu\nmWMwM7NCs7cA5gE9EfFoRPwPsAZY0OQxmJkZoIho3sqkDwPzI+Lj6fY5wAkRcWGpzxJgSbr5ZuDh\nOhZ9JPB0g4c7XuVUK7jedpZTrdDcet8UEUfV6rR/M0ZSoiptv5ZAEbESWDmihUqbIqJrNAObKHKq\nFVxvO8upVhif9TZ7F9B2YGbp9gxgR5PHYGZmND8ANgKzJc2SdCCwEFjX5DGYmRlN3gUUEf2SLgRu\nB/YDVkXE1gYsekS7jCa4nGoF19vOcqoVxmG9TT0IbGZm44c/CWxmlikHgJlZpiZ0ALT7ZSUkrZK0\nW9KDpbbDJa2X9Ej6PaWVY2wUSTMl3Slpm6Stki5K7e1a7+sl3Svpv1K9n0vtsyTdk+r9bjpZom1I\n2k/SjyXdkm63bb2SeiVtkXS/pE2pbVw9nydsAGRyWYnrgPmD2pYDGyJiNrAh3W4H/cCyiHgrcCKw\nND2e7Vrvy8ApEfF2YC4wX9KJwBeBK1O9e4ELWjjGsXARsK10u93rPTki5pbO/x9Xz+cJGwBkcFmJ\niLgL2DOoeQGwOk2vBs5s6qDGSETsjIj70vTzFC8S02nfeiMi+tLNA9JPAKcA30vtbVMvgKQZwBnA\nP6Tboo3rHcK4ej5P5ACYDjxRur09tbW7jojYCcWLJjC1xeNpOEmdwO8A99DG9abdIfcDu4H1wE+B\nZyKiP3Vpt+f0V4C/AP4v3T6C9q43gDskbU6XuIFx9nxu9qUgGqnmZSVs4pE0CbgB+GREPFe8SWxP\nEfEKMFfSZOAm4K3VujV3VGND0geB3RGxWVL3QHOVrm1Rb3JSROyQNBVYL+m/Wz2gwSbyFkCul5XY\nJWkaQPq9u8XjaRhJB1C8+H8rIm5MzW1b74CIeAaoUBz7mCxp4I1ZOz2nTwL+QFIvxe7aUyi2CNq1\nXiJiR/q9myLg5zHOns8TOQByvazEOmBxml4M3NzCsTRM2h98LbAtIr5cmtWu9R6V3vkj6WDgfRTH\nPe4EPpy6tU29EXFxRMyIiE6K/9UfRsQi2rReSYdIOnRgGjgVeJBx9nye0J8ElnQ6xbuIgctKrGjx\nkBpK0neAborLyO4CLgH+GVgLvBF4HDgrIgYfKJ5wJP0u8O/AFl7dR/wZiuMA7Vjv2ygOAu5H8UZs\nbUR8XtJvUrxDPhz4MfDRiHi5dSNtvLQL6FMR8cF2rTfVdVO6uT/w7YhYIekIxtHzeUIHgJmZ7buJ\nvAvIzMxGwQFgZpYpB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWab+H02b8FPpY7H+AAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb8e8f60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "california_housing_dataframe.hist('housing_median_age')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing Data\n",
    "You can access `DataFrame` data using familiar Python dict/list operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.series.Series'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0    San Fransisco\n",
       "1         San Jose\n",
       "2       Sacremento\n",
       "Name: City name, dtype: object"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities = pd.DataFrame({'City name': city_names, 'Population': population})\n",
    "print(type(cities['City name']))\n",
    "cities['City name']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'str'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'San Jose'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(type(cities['City name'][1]))\n",
    "cities['City name'][1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City name</th>\n",
       "      <th>Population</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>San Fransisco</td>\n",
       "      <td>852469</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>San Jose</td>\n",
       "      <td>1015785</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       City name  Population\n",
       "0  San Fransisco      852469\n",
       "1       San Jose     1015785"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(type(cities[0:2]))\n",
    "cities[0:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`pandas` also provides a rich API for advanced indexing and selection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manipulating Data\n",
    "You can apply Python's basic arithmetic operations to `Series`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     852.469\n",
       "1    1015.785\n",
       "2     485.199\n",
       "dtype: float64"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "population / 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy is a popular toolkit for scientific computing. *pandas* `Series` can be used as arguments to most NumPy functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    13.655892\n",
       "1    13.831172\n",
       "2    13.092314\n",
       "dtype: float64"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "np.log(population)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more complex single-column transformations, you can use `Series.apply`. Like Python map function, `Series.apply` accepts as an argument a lambda function, which is applied to each value.\n",
    "\n",
    "The example below creates a new `Series` that indicates whether `population` is over one million:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1     True\n",
       "2    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "population.apply(lambda val: val > 1000000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modifying `DataFrames` is also straightforward. For example, the following code adds two `Series` to an existing `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City name</th>\n",
       "      <th>Population</th>\n",
       "      <th>Area square miles</th>\n",
       "      <th>Population density</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>San Fransisco</td>\n",
       "      <td>852469</td>\n",
       "      <td>46.87</td>\n",
       "      <td>18187.945381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>San Jose</td>\n",
       "      <td>1015785</td>\n",
       "      <td>176.53</td>\n",
       "      <td>5754.177760</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sacremento</td>\n",
       "      <td>485199</td>\n",
       "      <td>97.92</td>\n",
       "      <td>4955.055147</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       City name  Population  Area square miles  Population density\n",
       "0  San Fransisco      852469              46.87        18187.945381\n",
       "1       San Jose     1015785             176.53         5754.177760\n",
       "2     Sacremento      485199              97.92         4955.055147"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities['Area square miles'] = pd.Series([46.87, 176.53, 97.92])\n",
    "cities['Population density'] = cities['Population'] / cities['Area square miles']\n",
    "cities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise #1\n",
    "Modify the `cities` table by adding a new boolean column that is True if and only if *both* of the following are True:\n",
    "  - The City is named after a saint.\n",
    "  - The city has an area greater than 50 square miles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City name</th>\n",
       "      <th>Population</th>\n",
       "      <th>Area square miles</th>\n",
       "      <th>Population density</th>\n",
       "      <th>Is wide and has saint name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>San Fransisco</td>\n",
       "      <td>852469</td>\n",
       "      <td>46.87</td>\n",
       "      <td>18187.945381</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>San Jose</td>\n",
       "      <td>1015785</td>\n",
       "      <td>176.53</td>\n",
       "      <td>5754.177760</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sacremento</td>\n",
       "      <td>485199</td>\n",
       "      <td>97.92</td>\n",
       "      <td>4955.055147</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       City name  Population  Area square miles  Population density  \\\n",
       "0  San Fransisco      852469              46.87        18187.945381   \n",
       "1       San Jose     1015785             176.53         5754.177760   \n",
       "2     Sacremento      485199              97.92         4955.055147   \n",
       "\n",
       "   Is wide and has saint name  \n",
       "0                       False  \n",
       "1                        True  \n",
       "2                       False  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities['Is wide and has saint name'] = cities['City name'].apply(lambda name: \"San\" in name) & cities['Area square miles'].apply(lambda area: area > 50)\n",
    "cities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Indexes\n",
    "Both `Series` and `DataFrame` objects also define an `index` property that assigns an identifier value to each `Series` item or `DataFrame` row.\n",
    "By default, at construction, `pandas` assigns index values that reflect the ordering of the source data. Once created, the index values are stable; thatis, they do not change when data is reorederd."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=3, step=1)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city_names.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=3, step=1)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Call `DataFrame.reindex` to manually reorder the rows. For example, the following has the same effect as sorting by city name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City name</th>\n",
       "      <th>Population</th>\n",
       "      <th>Area square miles</th>\n",
       "      <th>Population density</th>\n",
       "      <th>Is wide and has saint name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sacremento</td>\n",
       "      <td>485199</td>\n",
       "      <td>97.92</td>\n",
       "      <td>4955.055147</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>San Fransisco</td>\n",
       "      <td>852469</td>\n",
       "      <td>46.87</td>\n",
       "      <td>18187.945381</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>San Jose</td>\n",
       "      <td>1015785</td>\n",
       "      <td>176.53</td>\n",
       "      <td>5754.177760</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       City name  Population  Area square miles  Population density  \\\n",
       "2     Sacremento      485199              97.92         4955.055147   \n",
       "0  San Fransisco      852469              46.87        18187.945381   \n",
       "1       San Jose     1015785             176.53         5754.177760   \n",
       "\n",
       "   Is wide and has saint name  \n",
       "2                       False  \n",
       "0                       False  \n",
       "1                        True  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities.reindex([2, 0, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reindexing is a great way to shuffle (randomise) a `DataFrame`. In the example below, we take the index, wich is array-like, and pass it to NumPy's `random.permutation` function, which shuffles its values in place. Calling `reindex` with this shuffled array causes the `DataFrame` rows to be shuffled in the same way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City name</th>\n",
       "      <th>Population</th>\n",
       "      <th>Area square miles</th>\n",
       "      <th>Population density</th>\n",
       "      <th>Is wide and has saint name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sacremento</td>\n",
       "      <td>485199</td>\n",
       "      <td>97.92</td>\n",
       "      <td>4955.055147</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>San Jose</td>\n",
       "      <td>1015785</td>\n",
       "      <td>176.53</td>\n",
       "      <td>5754.177760</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>San Fransisco</td>\n",
       "      <td>852469</td>\n",
       "      <td>46.87</td>\n",
       "      <td>18187.945381</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       City name  Population  Area square miles  Population density  \\\n",
       "2     Sacremento      485199              97.92         4955.055147   \n",
       "1       San Jose     1015785             176.53         5754.177760   \n",
       "0  San Fransisco      852469              46.87        18187.945381   \n",
       "\n",
       "   Is wide and has saint name  \n",
       "2                       False  \n",
       "1                        True  \n",
       "0                       False  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities.reindex(np.random.permutation(cities.index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise #2\n",
    "The `reindex` method allows index values that are not in the original `DataFrame`'s index values. Tyr it and see what happens if you use such values! Why do you think this is allowed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City name</th>\n",
       "      <th>Population</th>\n",
       "      <th>Area square miles</th>\n",
       "      <th>Population density</th>\n",
       "      <th>Is wide and has saint name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sacremento</td>\n",
       "      <td>485199.0</td>\n",
       "      <td>97.92</td>\n",
       "      <td>4955.055147</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>San Jose</td>\n",
       "      <td>1015785.0</td>\n",
       "      <td>176.53</td>\n",
       "      <td>5754.177760</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sacremento</td>\n",
       "      <td>485199.0</td>\n",
       "      <td>97.92</td>\n",
       "      <td>4955.055147</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    City name  Population  Area square miles  Population density  \\\n",
       "2  Sacremento    485199.0              97.92         4955.055147   \n",
       "5         NaN         NaN                NaN                 NaN   \n",
       "1    San Jose   1015785.0             176.53         5754.177760   \n",
       "2  Sacremento    485199.0              97.92         4955.055147   \n",
       "\n",
       "  Is wide and has saint name  \n",
       "2                      False  \n",
       "5                        NaN  \n",
       "1                       True  \n",
       "2                      False  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities.reindex([2, 5, 1, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This behavior is desirable because indexes are often strings pulled from the actual data.\n",
    "In this case, allowing \"missing\" indices makes it easy to reindex using an external list, as you don't have to worry about sanitizing the input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}