Final-Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CMPINF0010 Lab Final Project\n",
    "\n",
    "Your final project is, at least ideally, the conjoining of everything you've learned in the course so far. You'll be using many of the big ideas you've learned, your Python skills, command-line and `git`, and plenty of pandas and data viz. \n",
    "\n",
    "You will be working in teams (assigned below) to make a data driven argument that answers the following question:\n",
    "\n",
    "## What is the best neighborhood in Pittsburgh?\n",
    "\n",
    "Using data from the WPRDC, you will create a data driven argument to support your claim about the “best” neighborhood in Pittsburgh.\n",
    "\n",
    "With your group members, you will be creating a Jupyter notebook to demonstrate your argument and the data analysis you did to support it. You will present your arguments to the class in the last few weeks of lab.\n",
    "\n",
    "To answer this question you need to do the following:\n",
    "\n",
    "* Come up with a team name!\n",
    "* As a group, come up with some way of defining and measuring \"bestness.\" This doesn't have to be a serious metric, it could be whimsical or secretly \"worstness.”\n",
    "* Use at least two datasets in your argument.\n",
    "* Create a git repository to store your data and notebooks and code.\n",
    "\n",
    "\n",
    "**Note**: There is a lot of subjectivity here, to wit, *what does \"best\" mean?*. One of your tasks in this project is to come up with your own metric for “best” and then use it to analyze data to determine the best neighborhood. The goal of this final project is to work as a team to develop a metric, apply it, and write up the results.\n",
    "\n",
    "You could define \"best\" as the smartest and then define smartness as \"number of advanced degree holders living in the neighborhood\". Or you might also define best as the ratio of the ”number of potholes\" and the “number of trees” in the neighborhood (lower or higher, the decision is up to you). How you want to measure bestness is up to you.\n",
    "\n",
    "----\n",
    "\n",
    "You'll be dealing with WPRDC data to talk about Pittsburgh, so here's a guide to working with the WPRDC."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The WPRDC, or, how to deal with open data\n",
    "\n",
    "Last week, you dug through the WPRDC and had a chance to see the vast and varied data available there. During this project, you'll be dealing with quite a few datasets and data sources, but luckily they'll all be *open data*. Open data is, according to [Albert Lin](http://www.wprdc.org/news/so-you-want-to-use-open-data/) of the WPRDC, \"a complete set of primary data made easily and permanently available in a timely fashion using electronic, machine readable, open file formats. Cost should not pose a barrier to accessing information, and no unreasonable restrictions should limit accessibility, sharing and re-use.\" \n",
    "\n",
    "### A brief guide to the WPRDC\n",
    "\n",
    "Your primary source of data to analyze will be the Western Pennsylvania Regional Data Center, or WPRDC. We've used WPRDC data a lot over the past few weeks, but this is the first time you'll be exploring the vast expanses of its data before. \n",
    "\n",
    "There's a *lot* of data available, so it's important to know how to search through it and find what you want. \n",
    "\n",
    "The primary way you'll find relevant datasets is through **keyword search**. From the master list of all datasets available at [data.wprdc.org](https://data.wprdc.org), just search for what you're looking for! For example, when I was looking for a list of all of the bus stops on Port Authority routes, I searched \"bus stops,\" and the first (and only) result was exactly what I wanted: [this lovely dataset](https://data.wprdc.org/dataset/port-authority-of-allegheny-county-transit-stops).\n",
    "\n",
    "However, searching around can be fruitless if you're not exactly sure what kind of data are available; fortunately, the WPRDC groups their datasets into topic-centered **categories**, like health, public safety, and housing. The full list of categories is available here: [data.wprdc.org/group](https://data.wprdc.org/group). If you can't find the specific data you want by searching for it, or if you don't know what data you want exactly, try browsing through the relevant category or categories and see what you find.\n",
    "\n",
    "Finally, you can also sort by **publisher**. Each government agency or private group that displays data through the WPRDC has their own page listing all of the data they provide. You can view all of the organizations here: [data.wprdc.org/organization](https://data.wprdc.org/organization)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What to do once you've got your data\n",
    "\n",
    "When you've found a dataset you're going to use in your analysis, it's important to consider how you should treat your data. There are basically two relevant vectors to consider here: streaming the data online, or downloading a copy locally. The good news is that you've done both of these kinds of data use before!\n",
    "\n",
    "**Streaming the data** is essentially just reading in your data from a link on the internet. We did this with the 311 data in a past week: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run this if you have data\n",
    "\n",
    "import ssl\n",
    "\n",
    "ssl._create_default_https_context = ssl._create_unverified_context"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>REQUEST_ID</th>\n",
       "      <th>REQUEST_TYPE</th>\n",
       "      <th>REQUEST_ORIGIN</th>\n",
       "      <th>STATUS</th>\n",
       "      <th>DEPARTMENT</th>\n",
       "      <th>NEIGHBORHOOD</th>\n",
       "      <th>COUNCIL_DISTRICT</th>\n",
       "      <th>WARD</th>\n",
       "      <th>TRACT</th>\n",
       "      <th>PUBLIC_WORKS_DIVISION</th>\n",
       "      <th>PLI_DIVISION</th>\n",
       "      <th>POLICE_ZONE</th>\n",
       "      <th>FIRE_ZONE</th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>GEO_ACCURACY</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CREATED_ON</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-12-15 14:53:00</th>\n",
       "      <td>203364.0</td>\n",
       "      <td>Street Obstruction/Closure</td>\n",
       "      <td>Call Center</td>\n",
       "      <td>1</td>\n",
       "      <td>DOMI - Permits</td>\n",
       "      <td>Central Northside</td>\n",
       "      <td>1.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>4.200322e+10</td>\n",
       "      <td>1.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1-7</td>\n",
       "      <td>-80.016716</td>\n",
       "      <td>40.454144</td>\n",
       "      <td>EXACT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-11-29 09:54:00</th>\n",
       "      <td>200800.0</td>\n",
       "      <td>Graffiti</td>\n",
       "      <td>Control Panel</td>\n",
       "      <td>1</td>\n",
       "      <td>Police - Zones 1-6</td>\n",
       "      <td>South Side Flats</td>\n",
       "      <td>3.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>4.200316e+10</td>\n",
       "      <td>3.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4-24</td>\n",
       "      <td>-79.969952</td>\n",
       "      <td>40.429243</td>\n",
       "      <td>APPROXIMATE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-12-01 13:23:00</th>\n",
       "      <td>201310.0</td>\n",
       "      <td>Litter</td>\n",
       "      <td>Call Center</td>\n",
       "      <td>1</td>\n",
       "      <td>DPW - Street Maintenance</td>\n",
       "      <td>Troy Hill</td>\n",
       "      <td>1.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>4.200324e+10</td>\n",
       "      <td>1.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1-2</td>\n",
       "      <td>-79.985859</td>\n",
       "      <td>40.459716</td>\n",
       "      <td>EXACT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-11-22 14:54:00</th>\n",
       "      <td>200171.0</td>\n",
       "      <td>Water Main Break</td>\n",
       "      <td>Call Center</td>\n",
       "      <td>0</td>\n",
       "      <td>Pittsburgh Water and Sewer Authority</td>\n",
       "      <td>Banksville</td>\n",
       "      <td>2.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>4.200320e+10</td>\n",
       "      <td>5.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4-9</td>\n",
       "      <td>-80.034210</td>\n",
       "      <td>40.406969</td>\n",
       "      <td>EXACT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-10-12 12:46:00</th>\n",
       "      <td>193043.0</td>\n",
       "      <td>Guide Rail</td>\n",
       "      <td>Call Center</td>\n",
       "      <td>1</td>\n",
       "      <td>DPW - Construction Division</td>\n",
       "      <td>East Hills</td>\n",
       "      <td>9.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>4.200313e+10</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3-19</td>\n",
       "      <td>-79.876582</td>\n",
       "      <td>40.451226</td>\n",
       "      <td>EXACT</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     REQUEST_ID                REQUEST_TYPE REQUEST_ORIGIN  \\\n",
       "CREATED_ON                                                                   \n",
       "2017-12-15 14:53:00    203364.0  Street Obstruction/Closure    Call Center   \n",
       "2017-11-29 09:54:00    200800.0                    Graffiti  Control Panel   \n",
       "2017-12-01 13:23:00    201310.0                      Litter    Call Center   \n",
       "2017-11-22 14:54:00    200171.0            Water Main Break    Call Center   \n",
       "2017-10-12 12:46:00    193043.0                  Guide Rail    Call Center   \n",
       "\n",
       "                     STATUS                            DEPARTMENT  \\\n",
       "CREATED_ON                                                          \n",
       "2017-12-15 14:53:00       1                        DOMI - Permits   \n",
       "2017-11-29 09:54:00       1                    Police - Zones 1-6   \n",
       "2017-12-01 13:23:00       1              DPW - Street Maintenance   \n",
       "2017-11-22 14:54:00       0  Pittsburgh Water and Sewer Authority   \n",
       "2017-10-12 12:46:00       1           DPW - Construction Division   \n",
       "\n",
       "                          NEIGHBORHOOD  COUNCIL_DISTRICT  WARD         TRACT  \\\n",
       "CREATED_ON                                                                     \n",
       "2017-12-15 14:53:00  Central Northside               1.0  22.0  4.200322e+10   \n",
       "2017-11-29 09:54:00   South Side Flats               3.0  16.0  4.200316e+10   \n",
       "2017-12-01 13:23:00          Troy Hill               1.0  24.0  4.200324e+10   \n",
       "2017-11-22 14:54:00         Banksville               2.0  20.0  4.200320e+10   \n",
       "2017-10-12 12:46:00         East Hills               9.0  13.0  4.200313e+10   \n",
       "\n",
       "                     PUBLIC_WORKS_DIVISION  PLI_DIVISION  POLICE_ZONE  \\\n",
       "CREATED_ON                                                              \n",
       "2017-12-15 14:53:00                    1.0          22.0          1.0   \n",
       "2017-11-29 09:54:00                    3.0          16.0          3.0   \n",
       "2017-12-01 13:23:00                    1.0          24.0          1.0   \n",
       "2017-11-22 14:54:00                    5.0          20.0          6.0   \n",
       "2017-10-12 12:46:00                    2.0          13.0          5.0   \n",
       "\n",
       "                    FIRE_ZONE          X          Y GEO_ACCURACY  \n",
       "CREATED_ON                                                        \n",
       "2017-12-15 14:53:00       1-7 -80.016716  40.454144        EXACT  \n",
       "2017-11-29 09:54:00      4-24 -79.969952  40.429243  APPROXIMATE  \n",
       "2017-12-01 13:23:00       1-2 -79.985859  40.459716        EXACT  \n",
       "2017-11-22 14:54:00       4-9 -80.034210  40.406969        EXACT  \n",
       "2017-10-12 12:46:00      3-19 -79.876582  40.451226        EXACT  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load the 311 data directly from the WPRDC and parse dates directly\n",
    "pgh_311_data = pd.read_csv(\"https://data.wprdc.org/datastore/dump/76fda9d0-69be-4dd5-8108-0de7907fc5a4\",\n",
    "                           index_col=\"CREATED_ON\", \n",
    "                           parse_dates=True)\n",
    "pgh_311_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you see here, we can just give `pandas` a link to a data file from the internet and it'll just handle it; it's pretty great that way. And it was good to use that for something that keeps updating continuously, like Pittsburgh 311 calls. (As I write this, the most recent update was 4 minutes ago.) \n",
    "\n",
    "However, links on the web are *unstable*: they can move or be taken down, and there's no guarantee that there will be a good \n",
    "internet connection everywhere. So, it's always a good idea to have a copy of any online data as a backup.\n",
    "\n",
    "If you're dealing with data that isn't actively changing, like results from a diabetes study that occured in 2015, it's probably best to **use a local copy of the data** for your analysis. You've done this a lot; it's why our data analysis labs have been in GitHub repos that you `git clone` instead of just downloading a notebook. Managing a bunch of files can be fiddly, but GitHub makes it easy to distribute and manage projects. You can download local copies of WPRDC data from the site and use them in your projects like any other file we've worked with, using the `read_table()` and `read_csv()` functions in `pandas`. \n",
    "\n",
    "**Note**: You may encounter some weird filetypes that we haven't dealt with in class. One I ran into was a `.dbf` file, which is an older type of database. To deal with `.dbf` files, we can use a module called `geopandas`, which normally is used for doing spatial/map stuff in data analysis. (You'll use `geopandas` elsewhere in your project, most likely.)\n",
    "\n",
    "To read in a `.dbf` file (I'm using the Port Authority's bus stop data as an example), do the following: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "ename": "OSError",
     "evalue": "no such file or directory: 'BusStops_data/PAAC_Stops_1611.dbf'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mOSError\u001b[0m                                   Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-6-d9f932e9620f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mdbf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34mr'BusStops_data/PAAC_Stops_1611.dbf'\u001b[0m \u001b[0;31m# this opens the database as a readable file\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mtable\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_file\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdbf\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0mpdtable\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtable\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/ihome/crc/install/python/miniconda3-3.7/lib/python3.7/site-packages/geopandas/io/file.py\u001b[0m in \u001b[0;36mread_file\u001b[0;34m(filename, bbox, **kwargs)\u001b[0m\n\u001b[1;32m     63\u001b[0m         \u001b[0mreader\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfiona\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     64\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 65\u001b[0;31m     \u001b[0;32mwith\u001b[0m \u001b[0mreader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath_or_bytes\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mfeatures\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     66\u001b[0m         \u001b[0mcrs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfeatures\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcrs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     67\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mbbox\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/ihome/crc/install/python/miniconda3-3.7/lib/python3.7/site-packages/fiona/__init__.py\u001b[0m in \u001b[0;36mopen\u001b[0;34m(path, mode, driver, schema, crs, encoding, layer, vfs, enabled_drivers, crs_wkt)\u001b[0m\n\u001b[1;32m    160\u001b[0m                 \u001b[0;32mraise\u001b[0m \u001b[0mIOError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"no such archive file: %r\"\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0marchive\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    161\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0mpath\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;34m'-'\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mos\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexists\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 162\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mIOError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"no such file or directory: %r\"\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0mpath\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    163\u001b[0m         c = Collection(path, mode, driver=driver, encoding=encoding,\n\u001b[1;32m    164\u001b[0m                        \u001b[0mlayer\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mlayer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvsi\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvsi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marchive\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0marchive\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mOSError\u001b[0m: no such file or directory: 'BusStops_data/PAAC_Stops_1611.dbf'"
     ]
    }
   ],
   "source": [
    "import geopandas as gpd\n",
    "\n",
    "dbf = r'BusStops_data/PAAC_Stops_1611.dbf' # this opens the database as a readable file\n",
    "\n",
    "table = gpd.read_file(dbf)\n",
    "\n",
    "pdtable = pd.DataFrame(table)\n",
    "pdtable.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`geopandas` can read some filetypes that `pandas` doesn't natively support. If you encounter any other filetypes and the normal functions don't work, remember to use [Read-Search-Ask](https://medium.freecodecamp.org/read-search-dont-be-afraid-to-ask-743a23c411b4) and try to find a solution on the internet. Someone's dealt with your situation before."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Groups \n",
    "\n",
    "You'll be broken into groups, which were generated randomly (using pandas)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "students = pd.read_csv(\"students.csv\")\n",
    "students = students.sample(len(students))\n",
    "groups = np.array_split(students, 12)\n",
    "\n",
    "glist = pd.DataFrame()\n",
    "i = 0\n",
    "\n",
    "while i < len(groups):\n",
    "    curr = groups[i]\n",
    "    curr[\"Group Number\"] = i\n",
    "    glist = glist.append(curr)\n",
    "    i += 1\n",
    "\n",
    "glist = glist.sort_index()\n",
    "glist.to_csv(\"groups.csv\")\n",
    "glist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you've found your group partner, link up with them (with the emails provided, or through whatever other means you wish). Each group will turn in **one copy of the project**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Progress Presentations\n",
    "\n",
    "In the last two weeks of lab, your group will need to present your argument for what is the best neighborhood in Pittsburgh. You will also have to talk about your metric and how you applied it. Expect to have a discussion about your team’s metric and get feedback to incorporate into the final report. \n",
    "\n",
    "Presentations should last a maximum of 10 minutes, with 5 minutes of discussion.\n",
    "\n",
    "You can prepare slides or you can present from a notebook directly. We understand your analysis will be in-progress, but you should have some initial results. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final Report\n",
    "\n",
    "It is best to think about your final project as a data-driven report. You will need to put everything into a Jupyter notebook with the following structure:\n",
    "\n",
    "* **Introduction:** Introduce the project, your approach, talk about the process of how you came up with the metric and some alternatives you may have explored.\n",
    "* **The Metric:** Describe your metric, what features are you measuring. What datasets  are you using?\n",
    "* **The Best Neighborhood:** Apply the metric from the previous section to determine the best neighborhood in Pittsburgh. Beyond just executing code, provide narrative about why you think this is the best neighborhood. Incorporate a data visualization, perhaps to rank all of the neighborhoods or show a neighborhood’s best-ness over time. The key is to make a data driven argument.\n",
    "* **Conclusion:** Reflect on how the data driven determination of “best neighborhood” is the same or different from your personal favorite neighborhood. Each member of the group should write their own response to this.\n",
    "\n",
    "### Tips\n",
    "\n",
    "* Make this fun! Don’t make this harder than it needs to be. You define your metrics, so you are entirely free to pick a metric that is easy to calculate, but reveals something fun about Pittsburgh. You can also be as fancy as you want, and if you want to do something big and impressive, go for it!\n",
    "* Show your work, but not all of it. When working on a project, I often have a couple sets of notebooks. One is a set of working notebooks where I do my data exploration, hack on the code, and go down wrong paths. When I finally am able to pull together the code that generates the result I want, I move just that code into my “final draft.” I only want to see the data cleaning, transformation, analysis, and visualization associated with the specific argument you are making.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Collaboration\n",
    "\n",
    "You should use a shared GitHub repository to collaborate with your group, so that you can work on the project and be sure that you all have the same version. One person should create a repository for the project, and then they'll need to invite the other members of the group to collaborate. You can see how to do that in this [GitHub article](https://help.github.com/articles/inviting-collaborators-to-a-personal-repository/). \n",
    "\n",
    "For programming, I'd point your attention to the software development concept of \"pair programming\". It's defined [here](https://www.agilealliance.org/glossary/pairing/): \n",
    "\n",
    "> \"Pair programming consists of two programmers sharing a single workstation (one screen, keyboard and mouse among the pair). The programmer at the keyboard is usually called the \"driver\", the other, also actively involved in the programming task but focusing more on overall direction is the \"navigator\"; it is expected that the programmers swap roles every few minutes or so.\n",
    "\n",
    "If you and your partners want to work together to solve the larger coding/data analysis problems, pair programming can be a good framework to try to operate in. It, however, is just a suggestion.\n",
    "\n",
    "## Completing the Report and Presentation\n",
    "\n",
    "The **final report** is due by **11:59pm, on Friday 6 December 2019**. \n",
    "\n",
    "You'll have this week and next to work on your presentation and the project in lab and independently, with your groups.\n",
    "\n",
    "The final two lab sessions will consist of **student presentations**: each group will give a short presentation on their chosen neighborhood. This presentation will be a lot like your lab lecture; you'll be presenting from your Jupyter notebook. You should prepare this with your group ahead of time, taking the time to write yourself speaker notes and get a sense of who will be presenting what.\n",
    "\n",
    "Next week, the groups will select their **presentation times**. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}