homework_20230320.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a4ee178e-d0c8-4c95-80ad-007c01ebebee",
   "metadata": {},
   "source": [
    "### Exercise 1: Flow sensors\n",
    "\n",
    "The data file has three recorded values of water flow recorded using three sensors S1, S2, and S3.\n",
    "\n",
    "- Compute the central tendency measures\n",
    "- Compute the spread measures\n",
    "- What can you say about S1, S2, S3 in terms of Means, Variance, and Standard deviation?\n",
    "- Which one of the sensors is different from the others?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0792de6f-7dbb-4036-87a4-65d5313e452c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# read the data\n",
    "df = pd.read_csv('group3.csv', sep = ';')\n",
    "print('Number of rows    = ' + str(df.shape[0]))\n",
    "print('Number of columns = ' + str(df.shape[1]))\n",
    "\n",
    "# compute central tendency and spread measures\n",
    "for col_name, col_data in df.items():\n",
    "    print(f'\\n** Sensor {col_name} **')\n",
    "    print(f'min, max     = {col_data.min():.1f}, {col_data.max():.1f}')\n",
    "    print(f'mean, median = {col_data.mean():.1f}, {col_data.median():.1f}')\n",
    "    print(f'std, var     = {col_data.std():.1f}, {col_data.var():.1f}')\n",
    "    print(f'skew         = {col_data.skew():.1f}')\n",
    "    \n",
    "# create a figure with two subplots\n",
    "fig, axs = plt.subplots(nrows=1, ncols=2, figsize=(10, 4))\n",
    "\n",
    "# boxplot\n",
    "df.plot(kind='box', ax=axs[0])\n",
    "\n",
    "# histograms\n",
    "for col_name in df.columns:\n",
    "    axs[1].hist(df[col_name], bins=20, alpha=0.5, label=col_name)\n",
    "axs[1].legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43f1beeb-0a24-42a1-a12d-eb20c783fdbf",
   "metadata": {},
   "source": [
    "**Question**<br>\n",
    "What can you say about S1, S2, S3 in terms of Means, Variance, and Standard deviation?\n",
    "\n",
    "**Answer**<br>\n",
    "Means: $S1 > S2 \\approx S3$<br>\n",
    "Variance and std: $S1 \\approx S2 < S3$\n",
    "\n",
    "**Question**<br>\n",
    "Which one of the sensors is different from the others?\n",
    "\n",
    "**Answer**<br>\n",
    "- In terms of central tendency, S1 is different from S2 and S3: it gives much higher values. One possible explanation is that the measurements of S1 are biased (e.g. due to a production error)\n",
    "- In terms of spread, S3 is different from S1 and S2: it has much higher variance in its measurements. One possible explanation is that S3 has more measurement noise. \n",
    "\n",
    "**Conclusion**<br>\n",
    "S2 seems to give the best measurements: it has lowest variance (highest precision) and it doesn't have the apparent bias of S1. \n",
    "\n",
    "(Of course, without more information on the background of this dataset, any conclusion is highly speculative)"
   ]
  },
  {
   "attachments": {
    "44c218c2-4b20-4083-a47e-8fbc34fea595.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuAAAABCCAYAAAAWlWPFAAANYklEQVR4Xu2dQXbVNhSGzR5KptAhhz2QZA/AkEOmLIIkXQTT0A6BPSSwh5wMS6ewiPbdl95wo8iWLNmWZH9vBLFkXX33t/xblu1H/+5+HT8IQAACEIAABCAAAQhAYBECjzDgi3CmEQhAAAIQgAAEIAABCOwJYMARAgQgAAEIQAACEIAABBYkgAFfEDZNQQACEIAABCAAAQhAAAOOBiAAAQhAAAIQgAAEILAgAQz4grBpCgIQgAAEIAABCEAAAhhwNAABCEAAAhCAAAQgAIEFCWDAF4RNUxCAAAQgAAEIQAACEMCAowEIQAACEIAABCAAAQgsSAADviBsmoIABCAAAQhAAAIQgAAGPFIDf3392l1eX3cX795F1ihf7NvNTXf66dNdIJenpw+Ckn5dXF11vm0xPcitH2ojd/+59YkvTx/wgx/jS+go6N+eO37l1g9Fnrv/3PrEx/gy1/hy8uFD9/3nz73ETo6OujeHh/fk9seXL93R8+fdi2fPQjLs3V7UgNsOSGfPXr3qnjx+nNyZOStKrE93sblJmLPN3H3HxCxl5Pf+5csH4pILDvvrE6Gvfm7sWp/48kj6+LkXZtLC8W4gcTUgf+/jnxfVr9rEl0dyKD9uns9fv35wsiiRX+0x8YVz32p+rXnRXvomeUrpj/jC2guN///szOnbnW8TX9Q3MVkqvxL7VPGJVk6Ojx+MnTp+pU5eSoxFDfjx+fl+5lVAnX3+XPXscl8S4mRcplRMzH1l5O9iykIXHDFt5PR+KD7fQeG2RXwPBw+ZdZIre2u45Vj0XWCV4Ed88UdMX37kxCcX0PbkIH9zL7JK5FdP7MQXznOL+VXj83F3t1gn1PrOJyX0R3xh3WmJofzoBKrcZe8zoSXyayfvZIY6Nz45N1otW3oxk5xDtIsYcN/V59BVsq8Dvlk8hWQNvcDTn4U4VF/Kh7brQXwnVOcWhd5a0+19M4zxh8L4kkPCCe0tp25o31NsJ750ir5Bo6YlVsSXnls7duXMzORFMFx7ipkj4us3PXOyidm35Pfi8vLehFrshE7M/nPLEF8uwdvZZZ00rfFcPFV86vP6xlKflsbQLWLAJUA54ctPZljHLj9xB3AXkkKRWT413fakHlPfXjX59i/braG3IqzhBBMSzpBIcuqOEV9qWeJLJXdbzzcrUZMBJ778/MbcvcprJb12TWbM1wviS8+tntvtHba513mPjda9w0Z8YwneLk3U9c81GvCp4hMvd7W7k+hbninUcr1IMQNuTbcuRYmVgZS36xld86AHlC1jDbhb372KCW13DYJ7SyvXgPfdIei7DeLjFhLOEGvf7L9veUJsvqYuR3x5RH0Dpm/ZR14r6bWJL52d7/Z6+t6mr0l8eUxr5ye90+VP2tOhNcJ5NNJqE18aN63l+qW5l5mMjXbK+EJLTJoz4LnLT3wddhf6D83mherHbndF4ZpjuwTF9/DTWFGNLR8SztD+5n5wYmxf3PLEl06w7+Kwllk/4kvPrdTMvfjPaz1cm/jCjGImR2pdXiSx+ybIct5UkUfsYW3iyyPqGu5azh3aqynjC11cNLkExS4/GfsqF7fDapityR0SRF99neGN2T60qN+V9tgZC9/s7n5Q63lLRd+hFBLO0CFY2wHlxprTt7yhJ652zfH5Lk5rugVLfHEa6ys1drzJa218beIbz8zWaJFfTRddPn7EF6/JqfxJfIvjSk4dX2h5Te6d4yJLUNz1OWOu5t0Zan3Ism89dp8h1ja1vhr4vv2722OXg+Teohgnv1+lQ8IZ2m9OXd2vHghzzP4TX6oqbm8P29dp+pZrxex9rvwSXwz94TJyfNgLdsmV/Ma8r3au/O4nE4gvK8k18/PNCPa9YWkIwlz6I74s6fU+P+S+VSvUylz57Xt+aGx8En+Md8v1IkUMuK75Tl2nbNdwicEbemDSJwRbX4y0PM1r30Ee2u67ylJDPrQtJMqptscIJzSDNuaiyLevuQ4w9+0z0nbKGsOtxucuAUthJ8zn4kd804wC9u1PY++ezZlf7R3x5eW5Vn7u27+klymTMHONL8SXrru+pb0pD/DPkd8p49Mx0H2bj6U3xVLYIgY8XQIPa6Ykf8r2a9xX6oXNlH2p6bbe0AVC7oXGlMzsvuCXRxZ+8MsjkFcb/cEvj0BebfSXx09qDz1HNxXf5gy4GO6nBwd3t1NTbm/lp6buPdg19ktHqsJMnVmdO17iyyMMP/jlEcirjf7gl0cgrzb6Wzc/27u+Z7lUA7HLkIeINWfAQx/AyZPHOmrb2/gpt5/XQYFeQAACEIAABCAAgTgCrr/0meyxr81elQGPw0gpCEAAAhCAAAQgAAEI1EmguRnwOjESFQQgAAEIQAACEIAABOIIYMDjOFEKAhCAAAQgAAEIQAACkxDAgE+CkZ1AAAIQgAAEIAABCEAgjgAGPI4TpSCwSgLy0Im8B/9i9z58fhBYmgD6W5o47VkC6A89lCSAAS9Jn7YhUJhA7qd0C4dP840TQH+NJ7Dx8NFf4wlsPPyiBtx+kl5enWe/Rtk4V8KHQBMEhj420EQHCLJpAuiv6fQ1Hzz6az6FTXegqAHX9ylyG6hpDRF8wwT6PjbQcJcIvSEC6K+hZK0wVPS3wqQ21KUiBtx+KMZlVeunwRvKKaFCIJqAXARP8UWv6AYpCAFDAP0hh5IE0F9J+rRdxIALdvu5dJafIEQILE+AO0/LM6fFXwTQH2ooSQD9laRP20KgmAG3pnvKT3uSVghAII4ADyDFcaLUPATQ3zxc2WscAfQXx4lS8xFY3ICz/GS+ZLJnCIwhwANIY2hRdmoC6G9qouxvDAH0N4YWZecgsLgBl07Y5Sf2TShzdJB9QgACfgI8gIQyShJAfyXp0zb6QwOlCRQx4NZ0s/yktARof6sEeABpq5mvo9/or448bDUK9LfVzNfT7yIGXE33t5ub7ur6unv/8mU9RIgEAhsgwANIG0hyxV1EfxUnZwOhob8NJLmBLhYx4A1wIUQIrJoADyCtOr3Vdw79VZ+iVQeI/lad3mY6hwFvJlUECoHpCPAA0nQs2dN4AuhvPDNqTEcA/U3Hkj2lE8CAp7OjJgSaJcADSM2mbhWBo79VpLHZTqC/ZlO3qsAx4KtKJ52BAAQgAAEIQAACEKidAAa89gwRHwQgAAEIQAACEIDAqghgwFeVTjoDAQhAAAIQgAAEIFA7AQx47RkiPghAAAIQgAAEIACBVRHAgK8qnXQGAhCAAAQgAAEIQKB2Ahjw2jNEfBCAAAQgAAEIQAACqyKAAV9VOukMBKYhIF+rld/J0VH35vDw9t8fPnS/Hxzw5dppELOXAQLoD3mUJID+StLfTttFDbi8DP/o+fPuxbNn+5P72atX3ZPHj7dDn55CYEYCehJxjbT8X469y+vre61/fPduf/zpcfnkt9+6P79+3RtuPlwxY6JWuOshfWl3Zcz//vPn/r/Hu/OA6Ey1KecF9LdCYSzQJfnM/Nudttzf093YdrEb4xj/FkgCTUQRKGrAxSBcnp52csCcff58d3BERU4hCECgl4C9oNUTkhrsvSEfuOCVbXKiks8160/Mup68wA6BEIHQhIqM/e7dlZPj47vJGPQXIsz2MQREb+evX+/1xfg3hhxl5yRQxIDbmQ+3c2LI+UEAAtMScE9A8n9ryG1rYrwvrq72f5KT1umnT/sLZX4QiCUQ0pfMfOuM97ebm73GdBYc/cVSplwMAbkbIz/Vm/w7pE/GvxiylMklUMSAS9A6uybrS0OzJbmdpD4EtkxADY6aaJ0RD5lq38z5ljnS9zgCIX3Z8V6fK9A9W5OE/uJ4U6qfgDv2ScmQPnVv6A9lzU2gmAG3g7AuRZm7s+wfAlsjoCcRe/tVT0rKwq6NtHzkGNVlAVvjRn/TCQzpy+pRZr31LoxoTWbA9YFfaR39peeAmrcE3KVO8jfGP9RRC4HFDTjLT2pJPXGsnYCeaKz59vXZd5LShy5lqYA+rBmaMV87T/qXRsDqy11uont0lwSgvzTW1PpFQDT0948fwWdXGP9QTSkCixtw6ahdfmLfhFIKAu1CYG0EdB1t3zpv21939lHqivGWN1Fc7R6+lGUB8renu1cQ6kNMa+NFf+YjYPXlWxIg2rIP+aK/+XKxlT2rzhj/tpLxNvtZxIBb083ykzaFQ9T1EtC7TDEz1vq6OC0rJ66Ly8t7b0HhOY16c117ZK6+3CVRrlFCf7VntI34fLPavsgZ/9rI51qjLGLA1XTLYKszbGsFTL8gsCQB+waJe7Pc/39Qx13/aN+/rHXdhzVlP/aVcUv2h7baIjCkL+2JW0ZnKdFfW7muNVqZgJCf77WpjH+1Zm2bcRUx4NtETa8hAAEIQAACEIAABCDQdRhwVAABCEAAAhCAAAQgAIEFCWDAF4RNUxCAAAQgAAEIQAACEMCAowEIQAACEIAABCAAAQgsSAADviBsmoIABCAAAQhAAAIQgAAGHA1AAAIQgAAEIAABCEBgQQIY8AVh0xQEIAABCEAAAhCAAAQw4GgAAhCAAAQgAAEIQAACCxLAgC8Im6YgAAEIQAACEIAABCDwH7wdH/5kg2qtAAAAAElFTkSuQmCC"
    }
   },
   "cell_type": "markdown",
   "id": "7ce7ed31-2a40-4f64-88f5-209cdda58a7a",
   "metadata": {},
   "source": [
    "### Exercise 2: Categorical quartiles\n",
    "Is it possible to divide a categorical dataset using quartiles?\n",
    "\n",
    "**Answer**<br>\n",
    "It depends on the nature of the categorical data. In order to compute quartiles, we need to be able to sort the data. Since *nominal* ordinal data by definition has no ordering, it is not possible to compute quartiles on such data. With *ordinal* categorical data, however, it is in principle possible to compute quartiles, but issues may arise when having very few levels (many repeating values) or when a quartile falls in between two categories.<br>\n",
    "<br>\n",
    "Example: \n",
    "\n",
    "![image.png](attachment:44c218c2-4b20-4083-a47e-8fbc34fea595.png)\n",
    "\n",
    "Conclusion: it's possible, but somewhat ugly and not without issues.<br>\n",
    "\n",
    "**Question**<br>\n",
    "Suppose we have list of set of colors\n",
    "Colors = [Red, Green, Blue, Blue, Green, Red, Yellow, Orange, Purple, Blue, Green, Red, Blue, Yellow, Orange, Green, Blue, Red, Yellow, Green]\n",
    "\n",
    "How can we make Quartiles in such a case?\n",
    "\n",
    "**Answer**<br>\n",
    "There is no natural way to compute quartiles on nominal data, because they cannot be ordered."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}