DB0201EN-PeerAssign-v5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/images/SN_web_lightmode.png\" width=\"300\" alt=\"cognitiveclass.ai logo\">\n",
    "</center>\n",
    "\n",
    "<h1 align=center><font size = 5>Assignment: Notebook for Peer Assignment</font></h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "Using this Python notebook you will:\n",
    "\n",
    "1.  Understand three Chicago datasets  \n",
    "2.  Load the three datasets into three tables in a Db2 database\n",
    "3.  Execute SQL queries to answer assignment questions \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understand the datasets\n",
    "\n",
    "To complete the assignment problems in this notebook you will be using three datasets that are available on the city of Chicago's Data Portal:\n",
    "\n",
    "1.  <a href=\"https://data.cityofchicago.org/Health-Human-Services/Census-Data-Selected-socioeconomic-indicators-in-C/kn9c-c2s2?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDB0201ENSkillsNetwork22-2022-01-01\">Socioeconomic Indicators in Chicago</a>\n",
    "2.  <a href=\"https://data.cityofchicago.org/Education/Chicago-Public-Schools-Progress-Report-Cards-2011-/9xs2-f89t?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDB0201ENSkillsNetwork22-2022-01-01\">Chicago Public Schools</a>\n",
    "3.  <a href=\"https://data.cityofchicago.org/Public-Safety/Crimes-2001-to-present/ijzp-q8t2?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDB0201ENSkillsNetwork22-2022-01-01\">Chicago Crime Data</a>\n",
    "\n",
    "### 1. Socioeconomic Indicators in Chicago\n",
    "\n",
    "This dataset contains a selection of six socioeconomic indicators of public health significance and a “hardship index,” for each Chicago community area, for the years 2008 – 2012.\n",
    "\n",
    "A detailed description of this dataset and the original dataset can be obtained from the Chicago Data Portal at:\n",
    "[https://data.cityofchicago.org/Health-Human-Services/Census-Data-Selected-socioeconomic-indicators-in-C/kn9c-c2s2](https://data.cityofchicago.org/Health-Human-Services/Census-Data-Selected-socioeconomic-indicators-in-C/kn9c-c2s2?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDB0201ENSkillsNetwork22-2022-01-01&cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ)\n",
    "\n",
    "### 2. Chicago Public Schools\n",
    "\n",
    "This dataset shows all school level performance data used to create CPS School Report Cards for the 2011-2012 school year. This dataset is provided by the city of Chicago's Data Portal.\n",
    "\n",
    "A detailed description of this dataset and the original dataset can be obtained from the Chicago Data Portal at:\n",
    "[https://data.cityofchicago.org/Education/Chicago-Public-Schools-Progress-Report-Cards-2011-/9xs2-f89t](https://data.cityofchicago.org/Education/Chicago-Public-Schools-Progress-Report-Cards-2011-/9xs2-f89t?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDB0201ENSkillsNetwork22-2022-01-01&cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ)\n",
    "\n",
    "### 3. Chicago Crime Data\n",
    "\n",
    "This dataset reflects reported incidents of crime (with the exception of murders where data exists for each victim) that occurred in the City of Chicago from 2001 to present, minus the most recent seven days. \n",
    "\n",
    "A detailed description of this dataset and the original dataset can be obtained from the Chicago Data Portal at:\n",
    "[https://data.cityofchicago.org/Public-Safety/Crimes-2001-to-present/ijzp-q8t2](https://data.cityofchicago.org/Public-Safety/Crimes-2001-to-present/ijzp-q8t2?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDB0201ENSkillsNetwork22-2022-01-01&cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the datasets\n",
    "\n",
    "This assignment requires you to have these three tables populated with a subset of the whole datasets.\n",
    "\n",
    "In many cases the dataset to be analyzed is available as a .CSV (comma separated values) file, perhaps on the internet. Click on the links below to download and save the datasets (.CSV files):\n",
    "\n",
    "- <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/data/ChicagoCensusData.csv\" target=\"_blank\">Chicago Census Data</a>\n",
    "\n",
    "- <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/data/ChicagoPublicSchools.csv\" target=\"_blank\">Chicago Public Schools</a>\n",
    "\n",
    "- <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/data/ChicagoCrimeData.csv\" target=\"_blank\">Chicago Crime Data</a>\n",
    "\n",
    "**NOTE**: For the learners who are encountering issues with loading from .csv in DB2 on Firefox, you can download the .txt files and load the data with those:\n",
    "\n",
    "- <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/data/ChicagoCensusData.txt\" target=\"_blank\">Chicago Census Data</a>\n",
    "    \n",
    "- <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/data/ChicagoPublicSchools.txt\" target=\"_blank\">Chicago Public Schools</a>\n",
    "  \n",
    "- <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/data/ChicagoCrimeData.txt\" target=\"_blank\">Chicago Crime Data</a>\n",
    "\n",
    "**NOTE:** Ensure you have downloaded the datasets using the links above instead of directly from the Chicago Data Portal. The versions linked here are subsets of the original datasets and have some of the column names modified to be more database friendly which will make it easier to complete this assignment.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Store the datasets in database tables\n",
    "\n",
    "To analyze the data using SQL, it first needs to be stored in the database.\n",
    "\n",
    "While it is easier to read the dataset into a Pandas dataframe and then PERSIST it into the database as we saw in Week 3 Lab 3, it results in mapping to default datatypes which may not be optimal for SQL querying. For example a long textual field may map to a CLOB instead of a VARCHAR. \n",
    "\n",
    "Therefore, **it is highly recommended to manually load the table using the database console LOAD tool, as indicated in Week 2 Lab 1 Part II**. The only difference with that lab is that in Step 5 of the instructions you will need to click on create \"(+) New Table\" and specify the name of the table you want to create and then click \"Next\". \n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/images/LoadingData.png\">\n",
    "\n",
    "##### Now open the Db2 console, open the LOAD tool, Select / Drag the .CSV file for the first dataset, Next create a New Table, and then follow the steps on-screen instructions to load the data. Name the new tables as follows:\n",
    "\n",
    "1.  **CENSUS_DATA**\n",
    "2.  **CHICAGO_PUBLIC_SCHOOLS**\n",
    "3.  **CHICAGO_CRIME_DATA**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Connect to the database\n",
    "\n",
    "Let us first load the SQL extension and establish a connection with the database\n",
    "\n",
    "The following required modules are pre-installed in the Skills Network Labs environment. However if you run this notebook commands in a different Jupyter environment (e.g. Watson Studio or Ananconda) you may need to install these libraries by removing the `#` sign before `!pip` in the code cell below.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting ibm_db==3.1.0\n",
      "Collecting ibm_db_sa==0.3.3\n",
      "Collecting sqlalchemy>=0.7.3 (from ibm_db_sa==0.3.3)\n",
      "  Using cached https://files.pythonhosted.org/packages/9b/87/f048087411ddd2a80171e169c5b74a663f6a9da2fe059890f3596f6e7e91/SQLAlchemy-2.0.4-cp37-cp37m-win_amd64.whl\n",
      "Collecting typing-extensions>=4.2.0 (from sqlalchemy>=0.7.3->ibm_db_sa==0.3.3)\n",
      "  Using cached https://files.pythonhosted.org/packages/31/25/5abcd82372d3d4a3932e1fa8c3dbf9efac10cc7c0d16e78467460571b404/typing_extensions-4.5.0-py3-none-any.whl\n",
      "Collecting greenlet!=0.4.17; platform_machine == \"aarch64\" or (platform_machine == \"ppc64le\" or (platform_machine == \"x86_64\" or (platform_machine == \"amd64\" or (platform_machine == \"AMD64\" or (platform_machine == \"win32\" or platform_machine == \"WIN32\"))))) (from sqlalchemy>=0.7.3->ibm_db_sa==0.3.3)\n",
      "  Using cached https://files.pythonhosted.org/packages/fa/9a/e0e99a4aa93b16dd58881acb55ac1e2fb011475f2e46cf87843970001882/greenlet-2.0.2-cp37-cp37m-win_amd64.whl\n",
      "Collecting importlib-metadata; python_version < \"3.8\" (from sqlalchemy>=0.7.3->ibm_db_sa==0.3.3)\n",
      "  Using cached https://files.pythonhosted.org/packages/26/a7/9da7d5b23fc98ab3d424ac2c65613d63c1f401efb84ad50f2fa27b2caab4/importlib_metadata-6.0.0-py3-none-any.whl\n",
      "Collecting zipp>=0.5 (from importlib-metadata; python_version < \"3.8\"->sqlalchemy>=0.7.3->ibm_db_sa==0.3.3)\n",
      "  Using cached https://files.pythonhosted.org/packages/a8/7d/90189265f0a9bcdf79b1143b77b0ef6dca0a5f13783f1e1efa4d7d7eafca/zipp-3.14.0-py3-none-any.whl\n",
      "Installing collected packages: ibm-db, typing-extensions, greenlet, zipp, importlib-metadata, sqlalchemy, ibm-db-sa\n",
      "  Found existing installation: ibm-db 3.1.0\n",
      "    Uninstalling ibm-db-3.1.0:\n",
      "      Successfully uninstalled ibm-db-3.1.0\n",
      "  Found existing installation: typing-extensions 4.5.0\n",
      "    Uninstalling typing-extensions-4.5.0:\n",
      "      Successfully uninstalled typing-extensions-4.5.0\n",
      "  Found existing installation: greenlet 2.0.2\n",
      "    Uninstalling greenlet-2.0.2:\n",
      "      Successfully uninstalled greenlet-2.0.2\n",
      "  Found existing installation: zipp 3.14.0\n",
      "    Uninstalling zipp-3.14.0:\n",
      "      Successfully uninstalled zipp-3.14.0\n",
      "  Found existing installation: importlib-metadata 6.0.0\n",
      "    Uninstalling importlib-metadata-6.0.0:\n",
      "      Successfully uninstalled importlib-metadata-6.0.0\n",
      "  Found existing installation: SQLAlchemy 1.3.24\n",
      "    Uninstalling SQLAlchemy-1.3.24:\n",
      "      Successfully uninstalled SQLAlchemy-1.3.24\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\\\anaconda3\\\\lib\\\\site-packages\\\\~qlalchemy\\\\cprocessors.cp37-win_amd64.pyd'\n",
      "Consider using the `--user` option or check the permissions.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uninstalling SQLAlchemy-2.0.4:\n",
      "  Successfully uninstalled SQLAlchemy-2.0.4\n",
      "Collecting sqlalchemy==1.3.24\n",
      "  Using cached https://files.pythonhosted.org/packages/b4/fc/5795b67a697a067d2a8c31535ed2c14a31b24f0edc97e4df23004d0f5c9d/SQLAlchemy-1.3.24-cp37-cp37m-win_amd64.whl\n",
      "Installing collected packages: sqlalchemy\n",
      "Successfully installed sqlalchemy-1.3.24\n",
      "Requirement already satisfied: ipython-sql in c:\\anaconda3\\lib\\site-packages (0.4.1)\n",
      "Requirement already satisfied: six in c:\\anaconda3\\lib\\site-packages (from ipython-sql) (1.12.0)\n",
      "Requirement already satisfied: ipython>=1.0 in c:\\anaconda3\\lib\\site-packages (from ipython-sql) (7.4.0)\n",
      "Requirement already satisfied: prettytable<1 in c:\\anaconda3\\lib\\site-packages (from ipython-sql) (0.7.2)\n",
      "Requirement already satisfied: sqlparse in c:\\anaconda3\\lib\\site-packages (from ipython-sql) (0.4.3)\n",
      "Requirement already satisfied: sqlalchemy>=0.6.7 in c:\\anaconda3\\lib\\site-packages (from ipython-sql) (1.3.24)\n",
      "Requirement already satisfied: ipython-genutils>=0.1.0 in c:\\anaconda3\\lib\\site-packages (from ipython-sql) (0.2.0)\n",
      "Requirement already satisfied: decorator in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (4.4.0)\n",
      "Requirement already satisfied: traitlets>=4.2 in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (4.3.2)\n",
      "Requirement already satisfied: jedi>=0.10 in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (0.13.3)\n",
      "Requirement already satisfied: prompt-toolkit<2.1.0,>=2.0.0 in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (2.0.9)\n",
      "Requirement already satisfied: colorama; sys_platform == \"win32\" in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (0.4.1)\n",
      "Requirement already satisfied: pickleshare in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (0.7.5)\n",
      "Requirement already satisfied: backcall in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (0.1.0)\n",
      "Requirement already satisfied: pygments in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (2.3.1)\n",
      "Requirement already satisfied: setuptools>=18.5 in c:\\anaconda3\\lib\\site-packages (from ipython>=1.0->ipython-sql) (40.8.0)\n",
      "Requirement already satisfied: parso>=0.3.0 in c:\\anaconda3\\lib\\site-packages (from jedi>=0.10->ipython>=1.0->ipython-sql) (0.3.4)\n",
      "Requirement already satisfied: wcwidth in c:\\anaconda3\\lib\\site-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython>=1.0->ipython-sql) (0.1.7)\n",
      "Requirement already satisfied: pandas in c:\\anaconda3\\lib\\site-packages (1.3.3)\n",
      "Requirement already satisfied: numpy>=1.17.3 in c:\\anaconda3\\lib\\site-packages (from pandas) (1.21.6)\n",
      "Requirement already satisfied: pytz>=2017.3 in c:\\anaconda3\\lib\\site-packages (from pandas) (2018.9)\n",
      "Requirement already satisfied: python-dateutil>=2.7.3 in c:\\anaconda3\\lib\\site-packages (from pandas) (2.8.0)\n",
      "Requirement already satisfied: six>=1.5 in c:\\anaconda3\\lib\\site-packages (from python-dateutil>=2.7.3->pandas) (1.12.0)\n"
     ]
    }
   ],
   "source": [
    "# These libraries are pre-installed in SN Labs. If running in another environment please uncomment lines below to install them:\n",
    "!pip install --force-reinstall ibm_db==3.1.0 ibm_db_sa==0.3.3\n",
    "# Ensure we don't load_ext with sqlalchemy>=1.4 (incompadible)\n",
    "!pip uninstall sqlalchemy==1.4 -y && pip install sqlalchemy==1.3.24\n",
    "!pip install ipython-sql\n",
    "!pip install pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The sql extension is already loaded. To reload it, use:\n",
      "  %reload_ext sql\n"
     ]
    }
   ],
   "source": [
    "%load_ext sql"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next cell enter your db2 connection string. Recall you created Service Credentials for your Db2 instance in first lab in Week 3. From your Db2 service credentials copy everything after db2:// (except the double quote at the end) and paste it in the cell below after ibm_db_sa://\n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork/labs/FinalModule_Coursera_V5/images/details.png\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remember the connection string is of the format:\n",
    "# %sql ibm_db_sa://my-username:my-password@my-hostname:my-port/my-db-name?security=SSL\n",
    "# Enter the connection string for your Db2 on Cloud database instance below\n",
    "%sql ibm_db_sa://CQP79093:RCGVI2LNdrfD6nXn@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb?security=SSL"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problems\n",
    "\n",
    "Now write and execute SQL queries to solve assignment problems\n",
    "\n",
    "### Problem 1\n",
    "\n",
    "##### Find the total number of crimes recorded in the CRIME table.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>total_number_of_crimes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>533</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[(533,)]"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select count(PRIMARY_TYPE) as Total_Number_of_Crimes from CHICAGO_CRIME_DATA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 2\n",
    "\n",
    "##### List community areas with per capita income less than 11000.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>community_area_name</th>\n",
       "        <th>per_capita_income</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>West Garfield Park</td>\n",
       "        <td>10934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>South Lawndale</td>\n",
       "        <td>10402</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>Fuller Park</td>\n",
       "        <td>10432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>Riverdale</td>\n",
       "        <td>8201</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('West Garfield Park', 10934),\n",
       " ('South Lawndale', 10402),\n",
       " ('Fuller Park', 10432),\n",
       " ('Riverdale', 8201)]"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select community_area_name, per_capita_income from CHICAGO_CENSUS_DATA where per_capita_income < 11000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 3\n",
    "\n",
    "##### List all case numbers for crimes  involving minors?(children are not considered minors for the purposes of crime analysis)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>case_number</th>\n",
       "        <th>description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>HL266884</td>\n",
       "        <td>SELL/GIVE/DEL LIQUOR TO MINOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>HK238408</td>\n",
       "        <td>ILLEGAL CONSUMPTION BY MINOR</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('HL266884', 'SELL/GIVE/DEL LIQUOR TO MINOR'),\n",
       " ('HK238408', 'ILLEGAL CONSUMPTION BY MINOR')]"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select CASE_NUMBER, DESCRIPTION from CHICAGO_CRIME_DATA \\\n",
    "where DESCRIPTION Like '%MINOR%'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 4\n",
    "\n",
    "##### List all kidnapping crimes involving a child?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>case_number</th>\n",
       "        <th>primary_type</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>HN144152</td>\n",
       "        <td>KIDNAPPING</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('HN144152', 'KIDNAPPING')]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select CASE_NUMBER, PRIMARY_TYPE from CHICAGO_CRIME_DATA \\\n",
    "where PRIMARY_TYPE Like '%KIDNAPPING%' and DESCRIPTION Like '%CHILD%'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 5\n",
    "\n",
    "##### What kinds of crimes were recorded at schools?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>primary_type</th>\n",
       "        <th>location_description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>PUBLIC PEACE VI</td>\n",
       "        <td>SCHOOL, PRIVATE, BUILDING</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>BATTERY</td>\n",
       "        <td>SCHOOL, PUBLIC, BUILDING</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>NARCOTICS</td>\n",
       "        <td>SCHOOL, PUBLIC, BUILDING</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>PUBLIC PEACE VI</td>\n",
       "        <td>SCHOOL, PUBLIC, BUILDING</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>ASSAULT</td>\n",
       "        <td>SCHOOL, PUBLIC, GROUNDS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>BATTERY</td>\n",
       "        <td>SCHOOL, PUBLIC, GROUNDS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>CRIMINAL DAMAGE</td>\n",
       "        <td>SCHOOL, PUBLIC, GROUNDS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>CRIMINAL TRESPA</td>\n",
       "        <td>SCHOOL, PUBLIC, GROUNDS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>NARCOTICS</td>\n",
       "        <td>SCHOOL, PUBLIC, GROUNDS</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('PUBLIC PEACE VI', 'SCHOOL, PRIVATE, BUILDING'),\n",
       " ('BATTERY', 'SCHOOL, PUBLIC, BUILDING'),\n",
       " ('NARCOTICS', 'SCHOOL, PUBLIC, BUILDING'),\n",
       " ('PUBLIC PEACE VI', 'SCHOOL, PUBLIC, BUILDING'),\n",
       " ('ASSAULT', 'SCHOOL, PUBLIC, GROUNDS'),\n",
       " ('BATTERY', 'SCHOOL, PUBLIC, GROUNDS'),\n",
       " ('CRIMINAL DAMAGE', 'SCHOOL, PUBLIC, GROUNDS'),\n",
       " ('CRIMINAL TRESPA', 'SCHOOL, PUBLIC, GROUNDS'),\n",
       " ('NARCOTICS', 'SCHOOL, PUBLIC, GROUNDS')]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select distinct(primary_type), location_description from \\\n",
    "CHICAGO_CRIME_DATA where location_description like '%SCHOOL%'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 6\n",
    "\n",
    "##### List the average safety score for each type of school.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>Elementary, Middle, or High School</th>\n",
       "        <th>average_safety_score</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>ES</td>\n",
       "        <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>HS</td>\n",
       "        <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>MS</td>\n",
       "        <td>48</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('ES', 49), ('HS', 49), ('MS', 48)]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%sql\n",
    "select \"Elementary, Middle, or High School\", AVG(safety_score) as Average_Safety_Score \n",
    "from CHICAGO_PUBLIC_SCHOOLS group by \"Elementary, Middle, or High School\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 7\n",
    "\n",
    "##### List 5 community areas with highest % of households below poverty line \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>community_area_name</th>\n",
       "        <th>highest_percent_households_below_poverty</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>Armour Square</td>\n",
       "        <td>40.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>East Garfield Park</td>\n",
       "        <td>42.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>Englewood</td>\n",
       "        <td>46.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>Fuller Park</td>\n",
       "        <td>51.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>North Lawndale</td>\n",
       "        <td>43.1</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('Armour Square', Decimal('40.1')),\n",
       " ('East Garfield Park', Decimal('42.4')),\n",
       " ('Englewood', Decimal('46.6')),\n",
       " ('Fuller Park', Decimal('51.2')),\n",
       " ('North Lawndale', Decimal('43.1'))]"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select COMMUNITY_AREA_NAME, MAX(PERCENT_HOUSEHOLDS_BELOW_POVERTY) as HIGHEST_PERCENT_HOUSEHOLDS_BELOW_POVERTY \\\n",
    "from CHICAGO_CENSUS_DATA where PERCENT_HOUSEHOLDS_BELOW_POVERTY > 39.2 group by COMMUNITY_AREA_NAME LIMIT 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 8\n",
    "\n",
    "##### Which community area is most crime prone?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>community_area_number</th>\n",
       "        <th>number_of_occurrence</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>25</td>\n",
       "        <td>43</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[(25, 43)]"
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select community_area_number, COUNT(*) AS number_of_occurrence from CHICAGO_CRIME_DATA \\\n",
    "group by community_area_number \\\n",
    "order by number_of_occurrence desc limit 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click **here** for a hint\n",
    "\n",
    "<!--\n",
    "Query for the 'community area number' that is most crime prone.\n",
    "-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 9\n",
    "\n",
    "##### Use a sub-query to find the name of the community area with highest hardship index\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>community_area_name</th>\n",
       "        <th>hardship_index</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>Riverdale</td>\n",
       "        <td>98</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('Riverdale', 98)]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%sql select community_area_name, hardship_index from CHICAGO_CENSUS_DATA \\\n",
    "where hardship_index = (select max(hardship_index) from CHICAGO_CENSUS_DATA)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 10\n",
    "\n",
    "##### Use a sub-query to determine the Community Area Name with most number of crimes?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " * ibm_db_sa://CQP79093:***@125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/bludb\n",
      "Done.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>community_area_name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>Austin</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('Austin',)]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%sql\n",
    "SELECT COMMUNITY_AREA_NAME\n",
    "FROM CHICAGO_CENSUS_DATA\n",
    "WHERE COMMUNITY_AREA_NUMBER IN (SELECT COMMUNITY_AREA_NUMBER from\n",
    "CHICAGO_CRIME_DATA group by COMMUNITY_AREA_NUMBER order by COUNT(*) DESC LIMIT 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright © 2020 [cognitiveclass.ai](cognitiveclass.ai?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkDB0201ENSkillsNetwork22-2022-01-01&cm_mmc=Email_Newsletter-_-Developer_Ed%2BTech-_-WW_WW-_-SkillsNetwork-Courses-IBMDeveloperSkillsNetwork-DB0201EN-SkillsNetwork-20127838&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Author(s)\n",
    "\n",
    "<h4> Hima Vasudevan </h4>\n",
    "<h4> Rav Ahuja </h4>\n",
    "<h4> Ramesh Sannreddy </h4>\n",
    "\n",
    "## Contribtuor(s)\n",
    "\n",
    "<h4> Malika Singla </h4>\n",
    "\n",
    "## Change log\n",
    "\n",
    "| Date       | Version | Changed by    | Change Description                 |\n",
    "| ---------- | ------- | ------------- | ---------------------------------- |\n",
    "| 2021-11-17  | 2.6  | Lakshmi  | Updated library  |\n",
    "| 2021-05-19 | 2.4     | Lakshmi Holla | Updated the question   |\n",
    "| 2021-04-30 | 2.3     | Malika Singla | Updated the libraries   |\n",
    "| 2021-01-15 | 2.2     | Rav Ahuja | Removed problem 11 and fixed changelog |\n",
    "| 2020-11-25 | 2.1     | Ramesh Sannareddy | Updated the problem statements, and datasets |\n",
    "| 2020-09-05 | 2.0     | Malika Singla | Moved lab to course repo in GitLab |\n",
    "| 2018-07-18 | 1.0     | Rav Ahuja | Several updates including loading instructions |\n",
    "| 2018-05-04 | 0.1     | Hima Vasudevan | Created initial version |\n",
    "\n",
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}