Classify Cairns building stock.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from __future__ import division, print_function\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "from itertools import product\n",
    "\n",
    "import warnings\n",
    "sns.set_context(\"poster\")\n",
    "sns.set_style('ticks')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The input file has been extracted from the full Queensland NEXIS dataset, for only Cairns and Yarrabah Regional Councils. It's been augmented by adding a field (`M4`) that contains the maximum local wind multiplier for the address location. This will be used to classify the site and populate the `AS4055_CLASS` field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename = \"C:/WorkSpace/data/exposure/CAIRNS_Residential_Wind_Exposure_201510_M4.csv\"\n",
    "\n",
    "df = pd.read_csv(filename, sep=\",\",header=0, index_col=0, skipinitialspace=True)\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These two functions will allocate a classification based on the M4 value, then assign a vulnerability model based on that classification and the building age. The functions update the existing `pandas.DataFrame` object - no copy is created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "def buildingClass(df):\n",
    "    thresholds = [0.0, 0.8278, 0.973, 1.147]\n",
    "    classes = ['C1', 'C2', 'C3', 'C4']\n",
    "    for thres, cls in zip(thresholds, classes):\n",
    "        idx = np.where(df['M4'] >= thres)[0]\n",
    "        df['AS4055_CLASS'][idx] = cls\n",
    "        \n",
    "    return df\n",
    "\n",
    "def classifyBuildingAttribute(df, field, classesfrom, classesto):\n",
    "    ndf = df.copy()\n",
    "    for cf, ct in zip(classesfrom, classesto):\n",
    "        ndf[field][ndf[field]==cf] = ct\n",
    "    return ndf\n",
    "\n",
    "\n",
    "def vulnCurve(df, default='dw1'):\n",
    "    classes = ['C1', 'C2', 'C3', 'C4']\n",
    "    curves = ['dw3', 'dw4', 'dw5', 'dw6']\n",
    "    # Set all to be default curve to begin with\n",
    "    df['WIND_VULNERABILITY_FUNCTION_ID'] = default\n",
    "    filter = df['YEAR_BUILT'].map(lambda x: x not in ['1982 - 1996', '1997 - present'])\n",
    "    for cls, curve in zip(classes, curves):\n",
    "        idx = np.where(df['AS4055_CLASS'] == cls)[0]\n",
    "        df['WIND_VULNERABILITY_FUNCTION_ID'][idx] = curve\n",
    "\n",
    "    df['WIND_VULNERABILITY_FUNCTION_ID'][filter] = default\n",
    "    return df\n",
    "\n",
    "def vulnCurveCombo(df,  default='dw1'):\n",
    "    walls = ['Brick', 'Cement', 'Timber']\n",
    "    ages = ['Pre-1950', '1950-1964', '1965-1979', '1980-1994', 'Post-1995']\n",
    "    n = 1\n",
    "    for p in product(ages, walls):\n",
    "        df[\"WIND_VULNERABILITY_FUNCTION_ID\"][(df.YEAR_BUILT==p[0]) & (df.WALL_TYPE==p[1])] = 'dw{0}'.format(n)\n",
    "        n+=1\n",
    "    return ndf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First pass - just do a classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "classesfrom = np.unique(df['YEAR_BUILT'])\n",
    "classesto = ['Pre-1950','Pre-1950','Pre-1950', \n",
    "             '1950-1964', '1965-1979', '1980-1994', 'Post-1995']\n",
    "ndf = classifyBuildingAttribute(df, 'YEAR_BUILT', classesfrom, classesto)\n",
    "walltypesin = np.unique(df['WALL_TYPE'])\n",
    "walltypesout = ['Brick', 'Cement', 'Timber', 'Timber']\n",
    "ndf = classifyBuildingAttribute(ndf, 'WALL_TYPE', walltypesin, walltypesout)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "from itertools import product\n",
    "walls = ['Brick', 'Cement', 'Timber']\n",
    "ages = ['Pre-1950', '1950-1964', '1965-1979', '1980-1994', 'Post-1994']\n",
    "\n",
    "ndf = vulnCurveCombo(ndf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputfile = \"C:/WorkSpace/data/exposure/cairns_prepost_classified_iag.csv\"\n",
    "ndf.to_csv(outputfile, header=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "df = buildingClass(df)\n",
    "\n",
    "df = vulnCurve(df, 'dw1')\n",
    "\n",
    "outputfile = \"C:/WorkSpace/data/exposure/cairns_prepost_classified.csv\"\n",
    "#df.to_csv(outputfile, header=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Second pass. Perform a virtual retrofit of all the older (pre-1981) building stock, by assigning a slightly different default vulnerability curve. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = vulnCurve(df, 'dw2')\n",
    "outputfile = \"C:/WorkSpace/data/exposure/cairns_retrofit_classified.csv\"\n",
    "#df.to_csv(outputfile, header=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [],
   "source": [
    "def autolabel(rects, rotation='horizontal'):\n",
    "    # attach some text labels\n",
    "    for rect in rects:\n",
    "        height = rect.get_height()\n",
    "        if np.isnan(height):\n",
    "            height = 0\n",
    "        ax.text(rect.get_x()+rect.get_width()/2., 1.05*height, '%d'%int(height),\n",
    "                ha='center', va='bottom', rotation=rotation)\n",
    "\n",
    "ax = sns.countplot(x='AS4055_CLASS', data=df, palette='RdBu', hue='YEAR_BUILT',\n",
    "                   order=['C1', 'C2', 'C3', 'C4'])\n",
    "autolabel(ax.patches, rotation='vertical')\n",
    "\n",
    "ax.legend(loc=1)\n",
    "ax.set_xlabel('AS4055 Classification')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = sns.countplot(x='WALL_TYPE', data=ndf, palette='RdBu', hue='YEAR_BUILT')\n",
    "autolabel(ax.patches, rotation='vertical')\n",
    "\n",
    "ax.legend(loc=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note this only describes the classification of the site. AS4055 classification is not relevant for buildings prior to 1982, and even then, not widespread until after 1990.\n",
    "\n",
    "We can do some basic analysis of the data. First, group by the AS4055 class and building age:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped = df.groupby(['AS4055_CLASS', 'YEAR_BUILT'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We perform a 'count' on the 'LATITUDE' field. Which field doesn't actually matter - it provides the total number in each of the groupings. We can also sum up different fields (e.g. replacement value)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "100 * grouped.count()['LATITUDE']/len(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(['YEAR_BUILT']).sum()['REPLACEMENT_VALUE']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(['YEAR_BUILT']).count()['LATITUDE']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df['WIND_VULNERABILITY_FUNCTION_ID'] = 'dw3'\n",
    "outputfile = \"C:/WorkSpace/data/exposure/cairns_uniform.csv\"\n",
    "#df.to_csv(outputfile, header=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "ndf.groupby(['YEAR_BUILT', 'WALL_TYPE']).count()['LATITUDE']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.pivot_table(ndf, index='WALL_TYPE', columns='YEAR_BUILT', values='REPLACEMENT_VALUE', aggfunc=len)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}