2015-10-28-Velocity-Amsterdam/Episode 5 -- Forecasting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Episode 5 -- Forecasting\n",
    "\n",
    "Problem:\n",
    "* Given the history of a metric, predict future values.\n",
    "* Inherently impossible.\n",
    "* Need strong assumptions about regularities (i.e. model).\n",
    "* Measure the goodness of fit\n",
    "\n",
    "Application: Capacity planning.\n",
    "\n",
    "* How much ressources will I need tomorrow?\n",
    "* Usually covered by simple regression methods\n",
    "  - linear growth\n",
    "  - exponential growth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regression Method\n",
    "\n",
    "* Parametric Ansatz for $f(t) = f(\\theta; t)$\n",
    "\n",
    "* Define residuals $e_i$ by:\n",
    "\n",
    "$$x_i = f(\\theta; t_i) + e_i$$\n",
    "\n",
    "* Loss function:\n",
    "\n",
    "$$ Loss(\\theta) = \\sum_i e_i^2 = \\sum_i (y_i - f(\\theta;t_i))^2 $$ \n",
    "\n",
    "* Chosse $\\hat{\\theta}$ by minimizing $Loss(\\theta)$.\n",
    "\n",
    "* A _goodness of fit_ measure is the minimal loss $MinLoss=Loss(\\hat{\\theta})$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Warmup: Constant Model\n",
    "\n",
    "* Parametric Ansatz for f: $f(x) = a$, constant.\n",
    "* Quadratic loss function:\n",
    "\n",
    "$$ Loss(a) = \\sum_i (y_i - a)^2 $$\n",
    "\n",
    "* Minimize loss function (_using calculus!_) gives __mean value__:\n",
    "\n",
    "$$ \\hat{a} = \\frac{1}{n} \\sum_i y_i = \\mu_Y $$\n",
    "\n",
    "\n",
    "\n",
    "$$ MinLoss(X,Y) = \\sum_i (y_i - \\mu_Y)^2 = n \\cdot Var(Y) $$\n",
    "\n",
    "* $Var(Y)$ measures goodness of constant fit!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple Linear Regression \n",
    "\n",
    "* Parametric Ansatz for f:\n",
    "  $$f(a,b; t) = bt + a, \\quad \\theta=(a,b)$$\n",
    "\n",
    "* Quadratic loss function:\n",
    "\n",
    "$$ Loss(a,b) = \\sum_i e_i^2 = \\sum_i (y_i - f(t_i))^2 =  \\sum_i (y_i - b t_i - a)^2 $$ \n",
    "\n",
    "$$ = a^2 A + ab \\cdot B + b^2 \\cdot C + D$$\n",
    "\n",
    "for some variables $A,B,C,D$ depending on $X,Y$.\n",
    "\n",
    "* Regression: Minimize $Loss(a,b)$ with respect to parameters $(a,b)$.\n",
    "\n",
    "* Since $Loss(a,b)$ is quadratic, it has a unique minimum which is easy to compute:\n",
    "\n",
    "$$\\hat{b} =  \\frac{\\sum_i (x_i - \\mu_x)(y_i - \\mu_y)}{\\sum_i(x_i - \\mu_x)^2} = Cov(X,Y) / Var(X) = \\rho_{X,Y} \\frac{\\sigma_Y}{\\sigma_X}$$\n",
    "\n",
    "$$\\hat{a} = \\mu_Y - \\hat{b} \\mu_X $$\n",
    "\n",
    "Remarks:\n",
    "\n",
    "* See (http://en.wikipedia.org/wiki/Simple_linear_regression) for a derivation.\n",
    "\n",
    "* Works also for more complex functions (e.g. polynomials)\n",
    "\n",
    "* Name \"linear\" regression comes from \"quadratic\" (!) loss function.\n",
    "\n",
    "* Note that Pearson Correlation appears in formula for $\\hat{b}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import math\n",
    "import random\n",
    "import numpy as np\n",
    "import matplotlib as mpl\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "mpl.rcParams['figure.figsize'] = (15,5)\n",
    "%matplotlib inline\n",
    "X = np.loadtxt(\"DataSets/RequestRates.csv\", delimiter=\",\")[:,1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model = 0.126045234879 + 0.678158488815 * x\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fcb53a2ded0>]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEDCAYAAADQunSaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VGXWwPHfpEBCRxAQEZH2iEgRuyggSNUxu/paXlGw\n7Qpr33F17UrRtcS+wrriiqhr21fGKFUUaSqgVJUHUEAE6U1KSDKZ948zs5mEQJKZSe7cyfl+Pnxg\n5t65ORnHc595ynk8wWAQpZRSySnF6QCUUkpVHk3ySimVxDTJK6VUEtMkr5RSSUyTvFJKJTFN8kop\nlcTSYnmxMSYTWA6MsNaOj3j+AmA0EAAmWWtHxRSlUkqpqMTakn8A2A6UnGz/PHAJ0B3oZ4zpEOPP\nUUopFYWok7wx5kTgROATwBPxfGtgh7V2g7U2CEwC+sQaqFJKqYqLpSX/FHBnKc83A7ZGPN4CHBPD\nz1FKKRWlqJK8MWYIMMta+zMRrfiQkl03JY8rpZSqItEOvA4CWhtjLgFaAAeNMeuttZ8BvyKt+bAW\nwIYyrpcL1IwyFqWUqq7KbER7Yi1QZox5GFhjrX0j4rnlwIVIcp8HXGWtXX2EywTLE2wC0/id5eb4\n3Rw7aPwJL6YplBE8xpihwG5r7URgOPDv0LF3ykjwSimlKknMLfk4cfvdVON3lpvjd3PsoPEnPF3x\nqpRSSUyTvFJKJTFN8koplcQ0ySulVBLTJK+UUklMk7xSSiUxTfJKKZXENMkrpVQS0ySvlFJJTJO8\nUkolMU3ySimVxDTJK6VUEtMkr5RSSUyTvFJKJTFN8koplcQ0ySulVBKLamcoY0wt4HWgCZABjLTW\nfhJxfC3wMxAIPTXYWrsxlkCVUkpVXLTb/10EzLfWPm2MaQlMBz6JOB4EBlhr98caoFJKqehF1V1j\nrX3PWvt06GFLYH0ppyX1llpKKXfz+vzNf922z+kwKl1Me7waY+YBxwIXWWuXRTy/BpgDtALmWGvv\nLeNSbt9nUeN3lpvjd3Ps4ML4vT6/B7gBeK7JUbVqj7u/r6vir6hou2sAsNaeY4zpArwJdIk49CAw\nBdgJTDTGXGqt/U8Zl0uIHcVjoPE7y83xuzl2cFH8u/ce5MyOzfj6u03UzkjjBm9HcFH8pSjzBhVV\nS94YcyqwxVq7PvT4O6CntXZbKecOB5paax85wiVd1xooQeN3lpvjd3Ps4KL4vT7/QOBfQFPgC2BI\nTnbWOlwSf7SibcmfBxwP3GmMaQrUAbYDGGPqAx8hA68HgB7AB3GIVSmlKszr89cCngRuBvKBu4Fn\ncrKzAkd8YZKItiWfAYwDjgMygUeAxsBua+1EY8xtwHXAXmCRtfa2Mi7pmtbAYWj8znJz/G6OHRI8\nfq/P3w3pTu4AfA8MzsnOWhxxSkLHHw8xDbzGkdvfaI3fWW6O382xQ4LG7/X5U4G7gJFAOvAC8Nec\n7KwDJU5NyPjjKaaBV6WUSjRen/944A2kq3gTcG1OdtZUZ6NyjiZ5pVRSCE2NvAp4GagHfAj8MSc7\n65AJIdWJ1q5RSrme1+dvCLyN9L+nANcDl1b3BA/akldKuZzX5z8f6Z5pAXwJXJOTnfWjs1ElDk3y\nSilX8vr8NYFRgA8oBB4CHs/JzipwNLAEo0leKeU6Xp//ZOAtoDOwGrg6Jzvra2ejSkya5JVSruH1\n+VOAW4EngJrAP4E/52Rn7XU0sASmSV4p5Qpen785so9FX2AbcEVOdpbf0aBcQGfXKKUSntfnvxRY\nhiT4yUAnTfDloy15pVTC8vr89YDngWuBXKT+zJic7KyEWKrvBprklVIJyevzdwcmACcA3yKDqz84\nG5X7aJJXSiUUr8+fjkyHvA+pK/M48EhOdlaeo4G5lCZ5pVTC8Pr87ZFVq6cD65CFTbOdjcrdNMkr\npRwXqjvzB+BZoBbSTXNrTnbWbkcDSwKa5JVSjvL6/E2AVwEvsAu4Pic7611no0oemuSVUo7x+vwX\nAq8BTYDPgKE52Vm/OBtVcokqyRtjaiGLEpoAGcBIa+0nEccvAEYDAWCStXZU7KEqpZJFaEu+p4Hh\nQB5Sf+a5nOysQkcDS0LRtuQvAuZba582xrQEpgOfRBx/HugHbAS+MMb8x1qrU5+UUnh9/tOQwVUD\nLEe25FvqbFTJK6okb619L+JhS2B9+IExpjWww1q7IfR4EtAH0CSvVDUW2pLvHuBRJPc8C9yXk52V\n62hgSS6mPnljzDzgWKRlH9YM2BrxeAvQJpafo5RyN6/PfwIyY6Y78g1/aE521qfORlU9xJTkrbXn\nGGO6IF+9uoSeLrncuLyb5Lp9mbLG7yw3x+/m2OEI8QeDQT7/Zj2ZNdM4cLCA7p2bc/NlXZrXrVVj\nelUGWAY3v/9l5tdoB15PBbZYa9dba5cYY9KMMY2ttduQu3SziNNbABviEWwCc/uO7xq/c9wcOxwh\nfq/PfxQwFrgM+A24Ze7SjRP+OvT0REqqbn//yxRtS/484HjgTmNMU6AOsB3AWrvOGFPPGHM8ktwv\nRDbXVUpVE16f/wJgPNAcmIusXF1Tntd6PJ4M6D9UHk0dHwwGtc8+Bp5gsOI3VWNMBjAOOA7IBB4B\nGgO7rbUTjTHnIUX9AT6w1j5TxiXdfjfV+J3l5vjdHDuUiN/r82cAjwF3AgVIbniivFvySYJ/YDI8\n1EueGTETRg2sxETv9ve/TFEl+Urg9jda43eWm+N3c+wQEb/X5++MbMl3MrASqRq5oCIX83gG3AQ5\nYyE99EwecPGwYHDKP+IYcyS3v/9l0hWvSqmYhLbkuwOpFlkD6Ye/Kyc7a5+jgSlAk7xSKgbbdh0A\nWQzZG5k6fX1OdtbH0V9x6ngYcSU82Esej5wpz6loaZJXSkXF6/NfXiczHSTBfwzcmJOdtTmWawaD\nwVyPxzMQFujAa5xon3x8aPzOcnP8rovd6/PXB14ErqmRnkpefmAY8IpLt+Rz3ftfUZrk40Pjd5ab\n43dV7F6f/zxk5erxwMIx9/Q+rUWTuq6JvxSuev+jod01SqkyeX3+GsDDwF9DT40CRrRoUle35Etw\nmuSVUkfk9flPREqXnAqsQRY2zXU2KlVemuSVUqUKbck3DMhGFj2+Dtyek521x8m4VMVokldKHcLr\n8zdFdmwaBOwEhuRkZ33gbFQqGprklVLFeH3+i5E9V48GPgWuzcnOKk+RQZWANMkrpQDw+vy1gWeA\nPwIHkVWsL+qWfO6mSV4phdfnPwMZXG0HLEW25FvubFQqHjTJK1WNeX3+NOBeZHpkKrK59gM52VkH\nHQ1MxY0meaWqKa/P3wZZ2HQ2svfDkJzsrM+cjUrFmyZ5paqZ0NTIa4EXkA1/3gOG5WRn7XQyLlU5\nUpwOQClVdbw+fyPgA2R6ZCFwDXClJvjkFXVL3hjzJHBu6BqPW2s/jDi2FvgZCISeGmyt3Rh9mEqp\nWHl9/n7IgqZjgNlI98xaJ2NSlS/ajbzPBzpaa88xxhwFLAI+jDglCAyw1u6PQ4xKqRh4ff5M4G/A\nbciWfPcCT+VkZwWO+EKVFKJtyc8C5of+vRuobYzxWGsjS1omdWU3pdzA6/N3RbbkOwlYgUyN/NbZ\nqFRViirJW2sDQHhrrxuAT0okeICxxphWwBxr7b3Rh6iUqqjQlnw+YDSyYerfgbtzsrP023U1E9Ps\nGmNMFnA90LfEoQeBKUjNi4nGmEuttf8p43IJUdg+Bhq/s9wcf1xj37rzAJ3aNGbZj9toULcmt19x\nCqd1aHozcHM8f04EN7/34O74y+wxiWXgtT/StzfAWvtb5DFr7ZsR500COgFlJXk3d++4feOBahO/\nx+PJgP6JtLVcXN97r89/JbKRdn3Av+u3g384rUPTrfG6fimqzWfHraIdeK0PPAX0ttbuKuXYR0jy\nPwD0QKZsKeUoSfAPTIaHeskzI670eDwDEyDRx8zr8zcAXgIGA/uBPwDjXLoln4qjqLb/M8b8EVkG\nvTLi6c+AZdbaicaY24DrgL3AImvtbWVc0u13U43fWeWK3+MZcBPkjJUuaoA84OJhweCUf1RqdEd2\nSOwV/bbh9fl7Am8ALZEJEVfnZGetqpRoD1UtPjtuFu3A6yvAK0c4/gKymk4pVQGHftt44CqP54L3\nIS2/ZMIPbck3ArgbWdj0KDA6Jzsrv+ojV4lKN/KOD43fWeVsyYcT6IO95JmRM2GU0901xWIv/m0j\nFykMGWrUM2JmOF6vz38SMjWyK/AjsiXfl1UbOlBNPjtuprVrVLURDAZzPR7PQFhQaldI4g3KTkb2\n7ZgG9AEe7IVn4VCvz5+OjIllAOOAO3Oys347/HVUdaYt+fjQ+J0Vc/ylDMrOrKJWfomWfDiOv/SC\nMcCfQ0cmULO2l3OumPxd7Qb1OwLb83P3/mnqyxMaynHHbkrV/rOT6LQlrxQgLfiHehUNyj7YK9Ti\nr/CgbFnfCCKPHzgwkYyMjP8eK/q2Mf9V+HhwOJ5mbS+gc7+ZgRoZ9TsCU3dsXDFs3jt7/wU5veSV\nyTNTSMWXVqFUKo6KWuI5Y+XPA5PludKPDxjwNpHHQRI9eGYDpKYX0LnvIk67eBFpNdIKgVuBgfPe\n+bx/0U0pHbkphW8sShXRJK8UAFPHSxdNHvJn5Ex5rqIivxGUlnyLH//ii6spPTlPHd/4+Oe+7XHN\n57Ts9DN7d+TuPbBn8xk52Vkv6dx3VRHaXaMUZQ/KViWvz5920Z8n/jUYDHaBfcFtP2+YtvnHKZf9\n9G1OxODq1PEw4sriM4WiuSmpZKcDr/Gh8TuryuIvX3/74adpljzes+ebfPHFDZnh416fvy0yb/JM\nZE+GITnZWV9EE0sV0c9OgtMkHx8av7OqJP7yzsCp4MDr2IyMDE9oS74bgOeA2sgc+FtysrOKlQ1J\nQPrZSXCa5OND43dWFSX5SimLEPT6/E2AfwJZyP4Mw3Oys/4dY7hVRT87CU775JVy0MIfNgMsA5oC\nM4GhOdlZPzsZk0ou2pKPD43fWVXcXRN7WQSvz18LeAK4JVhYGNi0es3E1fNfvmbX5tUH4hlzFdDP\nToLTlrxS5RSvGThen78bMrjaIXCggLkf9Erds/WiS2Fno8gFTQkysKpcTlvy8aHxOysh4y+ZpC/6\n88R84C5gJJC+/ZeNn339f9f1LiwIr4Uq6uN3sMxCRSXke18Bbo+/TNqSVypCvFrPJZN07YZPDg0W\nFhZ4UlLOA34Frv3yPf8JcGPv0q8QvzILqnqLZfu/J4FzQ9d43Fr7YcSxC5ANhAPAJGvtqFgDVaqy\nxbJzVMmbQ1GSTuPYE3/h5D5dz/akFAD8H/DHnOys7Z5nfpfRs+eEsbLqFXRBk6oMUZU1MMacD3S0\n1p4DDEDm9kZ6HrgE6A70M8Z0iClKpapEWSUJSldavRrIS0uvmc8pg77hlEHf4vEEWbf0+/HA/+Rk\nZ20H6eOfMuUq4OJh8ieyOyZeZRZUdRdtS34Wss0YyLze2sYYj7U2aIxpDeyw1m6A/27k3Qf4IeZo\nlXLQ4btyDu1aOaHbfUvbnv7JwZq1a9TcsTHIsunfzP9t+yPDlk4vPgiWkZFBafPsE6nMgnK3aLf/\nCwD7Qg9vAD6x1oY/vM2AyN3htwBtoo5QqSpTaj2Yd2QRVEE63H0ZjOohx0rvyklJDWC6r6D1qefe\nShCPnduO1fPbEAwuqHCCDl1b++BVTGIaeDXGZAHXA30jni45XSepR65V8iil9fwOPDCxqI9+PDLM\nlEHxgVC5OdRtdEevUwYtod7Rv5GXe3Db/P/rffSuTUeHrv5wD/hGB05VlYtl4LU/cC8wwFobufXY\nRqQ1H9YC2FCOSybEXM4YaPzOikv84SnFubm5XHfdc2PfeacDRYl9CDAduBCAMWNuHZubmzv2+ec/\n5LNFGyio/wWe1FTO73YMNXZvOXrayxnAJ6Ern8eYMbeOBcZWVuwO0vidU/bextHMkzfG1AdmA72t\ntdtKOb4c+T9hAzAPuMpau/oIl3T7XFWN31lxjb/4LJtc4HHgDKA3MAWoAXy0Cv51bma9R/2dL+hz\n1tGttnFwfyFLp+1k8092Fjx5GTxg4aEGctURu2BUq2AwuLsyY3eAxp/gom3JXwE0At43xoSf+wxY\nZq2dCAwHwgWW3ikjwStVZQ43eFr8+T7pkuADwPvAo6FX/w3JB/cAF7Y7tsNxizr2Oq15jcxtbP6p\nKUundeTg/hnA2T2g3zOS4P87GNsAFlyJdteoKhbtwOsrwCtHOD4bOCfaoJSqDJLI754K54QGT7tc\n5fF4+su/I+fHD1spf88ArqEoUd8DTCatBnTstYzjTu7WPJAfYNmnnVm3tBWQX0W/iVLlpyteVTVy\n/g3QrgcMCj3e2kOeq1FQfArks+3hvtXQvW3JKzRsDl0HzKR2g/3s2lyHJVNnrf1tW/9WkuDfQP5e\nOwum/RmGnQ7e9tLN89RMneeunKBJXlUj6WfDUIqS+RDg/bOR8aUIucDmJrAQ+BZ4AE9KIe3Oeot2\nZzQCz35Wfd2GVV+9M7swMNoLc6+BlO4QCEJwLnz+ZmhWTnu53v0r4anf6Tx35QRN8qoaCXwFDD70\nuRkR8+NzgewD8Pd68C5wBbUbTOaUQXto0Kwx+3dnsHjKAXZseA74/qPQQOpLoT9AeHORyG8Go9rD\nUu2PV47QJK+SSomB1Xeg/5Whf48HXoVHLpM56wCPzoIZrxbNj597A6RfAx+fCdOAIbTstJGTeqWS\nll6P9d8V8t3n51KQ9y7wDHDjXR6P5yVtoatEpqWG40Pjd1YQ8JRSYGwX3N1A5rhLqV6gHvT7CPLT\n5XXpBTDtYmCPvPa0XjCIGplT6dKvEU3bbCEvN51ln3bk15XjkemTJyL97NOAVw/Z/q+Cm4skxXvv\ndBAxcHv8ZdIkHx8av7NCSf6CW2Dyi8X3YA0vXsoD+twBvR6RqY25wGPAacCCApj3Icy4DAI0OeF9\nuvRrRM3a+Wxd14glUzuRu/dt4BfgkdC1nwD+CAwpdY/XCpQsTor33ukgYuD2+MukST4+NH5nBT0e\nTyZcvwTGti+e5Cciv9pM4Ift8G4jqAu8jUyPBClXMIPUtNfo0HMlrbqsJVAQZMWcH1jz7XbgVGAq\n8HugP/LNIA8YvhJe6xJjd43r33s0/oQWValhpRJP/6HwXHt4naLyvOOA74DfIdWwr2ok46N+iua/\npwNDqN+0Iedd/RGtuqxlz7Y6zHl7J2u+vQWoDywC/olMvZyAfAsAWPei9serRKcDryqJZAD1gKeA\nVKAlcCPFp0xOAj5AtjsAPEHanr6a9mf3JSU1hZ++gRVzZlAYuBtp8XdCunvC1xgKfATMXwkzXq2S\nX0upGGiSVwnt0B2XqAn9/i6Pp90cDAZ35+bmIqWA/7AKjm0HNZE6ed8Al5W4YgFyI3iIzHp3cMpA\ny1HH7iR3bwaLp5zKtp+bAAOJLER2qAl74KMztBWv3ECTvEpYHo+nPlw/Hy4OrRq972rIPBVGZ8oZ\nIy70eDymZ89xyIAryB7ZdyKt+leRwdV7Q1d8HVgF9KfFSdDx/Lmk10xl48odLPv0cvJza0f89Hyk\ny+d7ZEuEa0PPvwHseriUQmNKJSQdeI0PjT/OpAX/lyUwOrRq9DVgOTI/PXJg1fu1zGs/3IyaiaHX\nbQfWk56RRacLTqZ5+03kHyxk+WdL2PDDdcgN4cHQNR7dBXNnQJNz4emmUi74mNCxL2fBk/3j2IpP\nuPe+gjT+BKcteZWg+g+VBB9O3k0pvk1BWDBi8kAuUgp4BbLjZPjQdmATjVt2pOuAY8mos4kdG45i\n0eROHNizHBlU/W4NDFoIKQE4+ViYfqm89v6V8OXLUDMAafm6DZ9yG03yykV6I7Nbrg49fiAP9q+F\ncafDVcA7wHVIK/5fyHYG6aSkZnPiuZbWp66lMJDHijkdWL2gHQTzkYHVQcDkE2DhOvjyP/DEi8VL\nElycW9pceKXcQKdQqgQ1dbysUg1Ph5y5Gh7fDf+DzJD5QxAW1YDWl0lJ4D8gCT48LfJaYCl1G3fi\n3MFzaX3qWvbuqMXcf//I6vmtQgl+PDAg9PN+AP7SC1LPqsrfUqnKpi15lZBC9WR+B/NDM2l2L4Kh\nT0uCnwU87ZH57qGJN9xS8gqc0O1aTjzXQ2raXtYuDvL9rNMoLPgKucbXyIBsCjIgezNSkCw3Tbpo\nRoXGAkbO1BLBys2iHng1xnQGPgSesdb+vcSxtcDPyNY6AIOttRuPcDm3D35o/DEqZaokUv/l7l6y\n6dj/5cLTGbLAyQd8jCToNKT/PQ8YA9xJRp0DdB0wjcYta3BwXyFLpp3BljVNkB2eTgBuAPYAw4BL\nke6d8CrWHKAfcMdKWPdiuIBZJf7qjr/3MdL4E1xULXljTC0gG1nrXZogssH3/mgDU9VHKYXFroS5\nH0iCfx9ZnTooQ2bW5CMDrKuBO0JXmIB046zgmPb/oNMFzamRUYNNP3pYOq0veQfqhM57KPSaa4C/\nI3vMX0xR/ztIsq8LjGkPF+frIKtyu2j75A8CFwGbj3BOUt8dVew8Hk+GFBXrN7Oo/no6Ur0x3Sct\n+MjyA3cCbYG/IMk6/PzVpNW4j64DhnHqRS1JSfWwdPpGFvoLyTtQs8RP3YR07RwEzkSmZob7/Z9F\nvhUolTyi3eM1AAQiNvEuzVhjTCtgjrX23iOdqKoXabmffwPcdQc81lb6yEs6+wT4lKKt+sJmIrNm\nihx17A66DriAWvU3sWtTAxZN6sy+Xc8DawCL3BxAFkq1AjJ2w/315bnXkXnwHuD7VZDSThK+9sWr\n5FBZA68PIhOWdwITjTGXWmv/U0k/S7lIUdeM1G2Xlnh/ZKbLkNBZbyD97v9CumjCSfo+pAX+v8Ao\nPCn30v7sVbQ940cIelj5VXtWfWUIFhaErvsAkthvR/rgHwd8e+Dt+jJcNAM4Gknqy2bCG7+Dzf/d\nZES7alQyiGnFqzHmYWBbyYHXEucMB5paax85wqUSYtmtqnxjx37C8OH9kA03+lHUH74Hme3Sj+Ll\nfD9BEvInQBskYWdTu2FHThkUoEHTTPbtqsniyQvZ+evdoWuNAnYBPYBCYAHQDVgKdASyKOrrh6OP\nHsXKlXfSoEGDSv3dlaoEZXaLxzpP/pAfYIypb4z5whgTqi9CD2BZOa/l1j8afzn/DB/+4jD5cX0o\nXhZ4DPDLHjk2g6JyviBdLm2Bm4C3Ob7LMfS4OoMGTTP5eXkus988g52/rkZKCl+PLJbKBFYixcYM\nMjA7AtgP/GlzZF//1q3307DhlcOS/b3X+JM2/iOKqiVvjDkLWQveBCnrtwP5bv2TtXaiMeY2ZGXK\nXmCRtfa2Mi7p9mlMGn85Fd8aLxfp2duBTGPcjuy2BNJN8yNwHPBXAGrWGkmX/qfS5ATIO5DO0k+7\nsmlVY+BWpOfxEaAB8o1gwbfwabdDvzHkAYPeh8mXFX/u4lJ3eKoC+tlxltvjL1O0A69fIevBD3f8\nBeCFaINSyato0+wFQ+FAJpx6L7zWRI6OB/YBc5HW9wGgMRCgaZuddO7bjZq1YOvaxiye2o2D+zKR\nBN0YuRG8j5Q3eAwY9CPSR1OaWTDi6KI9WB9YKZt+K5V8dMWrqnKhRD8e+rwBTzSRFnUu0gp/CBiN\n9MmvIDXdw0k9p3B8Zw+BglSWf25Zu2grcDqS4F9Dvg1kIN0004G+QGFQbhqXU3xQ94HQZh8zJsDG\nUBnjB9tDzYkej+dwm20r5Vqa5FWVkG6aPjdCSncoSIFrukHbtnI0l6I9V7OQxU1X0aDZ9ZwycB61\nG3rYvaUuiybPYe/24cAcpDunLdAISfBh+cjsHE8ArkBKIDRCdnOywMIX5CYzYGjx/WAf7CXfLtBC\nZCqpaJJXlU42/7huIWS1lUqSbyH97b8ATyIzXsIDoeDxXEXbM9+k3VmN8XjgxwWrsfPaUBg4EWnt\n/z505TzgLuTGAPA8kvgvB96eC387Fh7uIcfeAPbMgs/HVf5vrFTi0E1D4qNaxV+yzsyRujjk3LuW\nwmPt5JnXkJa1BW4D7gGOBe4G0qlVfx+nDFxIw+a7OPBbBounLGT7+sFAK2SapdSnEW8gCT4b2JIH\nz9WQVv2js+Cx/nJOnxulsmT+l/D5uHCsxQeAQRY/jXKiu6ZafXYSkNvjL5O25FWFlFZnRqpF9j/M\nIqL+QyXBh/vdUylqeU8AeiHTHCdwXMeedDz/B9JqBNiwohnLZ5xC/sHzgb8hA6tjkBvCTUhVjauR\npD4CuGEdfB66kRQA0vcPvBT6U0zxAeDS4lYqOWhLPj6qTfwez4CbIGds8UVMj26GvzWVxyV3UipI\nl/1X05EFTT2QfnKQwdOXSc9YS+e+gzimXU3yD+axbEZXNq4I5WvykJvCDcjCptlIyaRxSCGx8DmT\nKbp5ODolsqKqzWcnQbk9/jJpS17F6DMkwUfupHTvc3BU6PH6uXDdFujQRFryvyI14HOB5zn6+D/S\npf9SMuocZPsvDVg8uQMHfnsF6b4BaaV3RRY35QPnATWBvwXgnlTIoF2751m1algV/b5KuYu25OOj\n2sR/6GKmO4GxFF9YdAcyCJqL1ItpjST2jwiX9k1J+5gO5x7HCd3WUhjwYOe258dvVkPwIuTbwYvI\ngOxEZHC2AdIHHwhdKwXwvgWe2Tt3vjO2YcPsmQnQvx6NavPZSVBuj79M2pJXUZjjhwtbQbtWMlsm\nch76aOAJJOlPQ8r5hguRyVTHekfv5pSBAeo2Xstv2+uwaNKp7NlaC5lxQ+i8jsAXwP3APKTUQQ/A\nS1FdG8/sUJfMWBil/etKlUKTvCrmSDNn5Nh9U+GR0LTE8cA6ZH/V6Uh3yiaKz1uPvHhvWp/6Hid2\nr0tKKqxZtIUfZvejsCAVaa3fhiTv0cBvyMybl5Fqkr2R1n1K6JwHVkaWAg7F6YY+eKWqlHbXxEdS\nxF/KzJmZkd0ehw665iFdMFORypFbkUJg/0D61HOR6Y3HkVH3MroOWErj43aQuzfIkmmwde1BpH/9\nO6SEwaYADEyVWjMjD8BpmVL+aBXyjeAcZGOPVVvgP+2DweDuyPgr7+2pVG6OHTT+hBdrFUqVVPoP\nPXSHpnA1WE9+AAAUvUlEQVSrHmSmTEnfIK3trNBr6iI7L90JfA7cSXPTiJ7XzKDxcTvYtDrArAl9\n2Lp2W+g1v0N2ejobeDm1aPu9UZlyA/hhFRyYIz9rHpC7qkSCV0odgXbXqMPIRfZ9OXi+x9O3J3g8\n0KGFtMz/HDrnWWQjj3DuH4Ls8pQONCetZl869f6BYzukUJCXypJpi1i/vGXo9d0o6noZT7i2e3Fp\nwOh2MOgFWPGmPKf97UpVhHbXxEdSxF/U535PDyk90ARYiCTyDGQzjp+RJH4c8BNSRiAVGAbUAh4G\nRtCoxa90HfA1mfVqsPPXeiya/CH7d52FLGJ6BhlYXYnMpLk/dP2nkLLBGcCbSEXJ8CyaaTceIbm7\n+f13c+yg8Sc8TfLxkRTxS42ZaxZArXbQBbgxdFgKhknCHYEMtu4ETkSmSILMqNlDSurptD+nPW1O\nW0sw6GHVVxtZ/fVqgsHweX9DZtH8k6JVsFOAFciNYibw9mZ4takk+zeQQmNPzjzCtEg3v/9ujh00\n/oSnffIK+G+NmQUwrp3MYklH5qSnI+UDZiBb6uUDPZEZNY9T1H9/N3WOCtL9f2vQ9vS17N+dwbx3\nC1n11dJQgg+fdw9wDPKtIA/5CH4H/An5JvDNTHjfwGVvySrWq5E++pLjA0qp8oi6T94Y0xn4EHim\n5B6vxpgLkHlwAWCStXZUTFGqKtB/KDzUTua2g1RynAzUQGa+LAdeQbbXG0TReQBBWnVdS4ce3UlN\ng5+XHeC7mR8RyL8Y2eavpDrITeIpZDXrzcjK2ZffgenXhUoBz4ZBg4v6+5VS0YiqJW+MqYWMwE09\nzCnPA5cA3YF+xpgO0YWnKovH48nweAbc5PEMuCk3NxfZpeldZPpiD6Qg2Lehx1lAC4oWPIEk7wnU\nrL2HMy6Zx8m9fyCQn88CfwpLpw8ikN8auRlsQ3aGDO/l+gYyH/7ZIPwyXzb4qAUsmBVO8HL9qeNl\nCmf4dSNnRs6LV0qVT7Qt+YPICNpfSx4wxrQGdlhrN4QeT0Iywg/RBqniq+R8+H79Xgea3yzlAgLI\nNnr9KVqpCpLgP0FqwMsK12Zte9K57zRqZKazZc1RLJk2h4P7TkIGbduFXvu/yEDr9Ugd+HDlyEc9\nMKwhDLgFahSUnDWjVSKVio9o93gNAAFjTGmHmyGrYsK2AG2i+TmqskTOh4fZs68AloR2aZoBXAaE\ne+D6U7SC9XvgD6Smv0TH81+i5cltCBTUZNmMjqxbciyyIfcg5CYAMqj6HjJ7hojnwy5uB2MLDlct\nUlexKhW7ypgnX3K6TnlHrhNimk8MXBP/mDG3Mnx45DOfIYOoE5D+8ncBX+hYeJ/U94CjaHDMOE4Z\n2JHaDTLYvbmARZPPZ++OOsic+duRG8cQZBZNDeTbQcl59OEbwRWMGZM+FqlwFivXvP+lcHPsoPE7\nqcz8WhlJfiPSmg9rAWwox+vcPI3JFdOwiurSFKTDzMth9Hly5CvgQmQYxUfxqpKXA/fiSelOuzN3\n0e7MDuCB1fPBzjufYOGXyIybAxSvWbMYqWNzYYkoXjkABZkyLfKpmcOHjxo4bFjM3TCueP8Pw82x\ng8af8GJN8oe8OdbadcaYesaY45HkfiEyyVo56NC6NHdvBT8y6Hky8DTQASkFHCbdLbUbjKbrwCU0\nPKY2+/cUsHjyuezYcBTFW/nrkQFSgNeRFax9kGGb8Bz5Rw/A5FZQ+Ht4Fe1nV6ryRbUYyhhzFrKa\npQmy19oOZArFT9baicaY85DVMQAfWGufKeOSbr+bJnT8kuD7vQofDy5eXGwy0ud+O/IrhOu45yAD\npJM47uTOdOz1PWk1AvzyfXOWf7aRgrzIHZieQubPH4XcJEA2BhmMJP9ZgAGWbIb3OweDwS2V8Csm\n9PtfBjfHDhp/wtMVr/GRcPEXdc0cyIQz/gTntis+WyYPKeF7HLLrUnid0QjgVmpkzqNz3800a9uM\nvNx0ls/ozEbbhEO32bsMud9nAA8iyb4Nsir2twKY+VdosLeSW+0J9/5XgJtjB40/4WmSj4+Eir94\n10x4N6YA0r1yXeis14ElyDz44sn/6FZj6NK/FRm1YdvPeSye0o/cvZmh1+QjC5nC16iJDKaeQFEN\nmvHAO2thxpmV1HIvKaHe/wpyc+yg8Sc8rUKZlPrcKAk+AHyN/GcuRGa2TkZWr/4xdKxISloBJ/X4\njlZdW1EY8PD9F+356Zt/hM5vg2wMAtJaz0SuvwZp2V9G8Vk072RXUYJXSh2BJvkkI63462+VQdMX\nkEXNvZEiYJ2Q/+Q3I0MmA5DJUOOp1ySLUwYuoW6jfezZVotFk/bx27bHgM5IpUmQrp3TkMHaNORb\nQQrSL19SWn6l/ZJKqXLT7pr4SJj4ZfemCWOlzszdoWf/gbTiHwg9noAk6d+D523anNYYc04KKakp\n/PRtS1bM6UxhQQAZfH2Vov72A8i0yLco3rd/M7JZd3hufZVvpJ0w738U3Bw7aPwJT1vySWkekuDD\npXx/AUZSlJivBq4hs97PdB1wLo1a7CZ3by6Lp/7KtnUDkW6YJ5FvArORfvg9yIKn8IDtyNC1nkda\n8uG6757ZOjVSqcShLfn4SJj4Q901S2Bse0nqE5HQIgdXD3Jshymc3DuV9JoBfl21gaXTh5Kfm48s\nhAog+6q+Ejr/CWQzjwZIkh/yI2x+CVoPh2fbS0u/ylvvkRLm/Y+Cm2MHjT/haUs++dQEuxzuagNP\npUqt9tuQLpqrSa+Zz8l9PubYEzMoyIPFUzrxy/eLkRIEtZHFS3nIQqnpSCt+LdIPn4cUH22+Ed4d\nCzPHwgYtIKZUAtOWfHwkRPyys9Pda6FdAykb8AKSpFsAl9PouOl0HXCQzLq12bHhKBZP6cb+3enI\nAGonivZufYOiapF5wHCkZPAVSOmDFODiYYcrLOaAhHj/o+Tm2EHjT3jakk8q/f4O5zQo6prpCPQh\nJXUCpvtntDktjcLCVFbMXc+P83sSDKYj+7emIAOnHyHFyjIo2mT7TaQvfhJwZejn5KGUcgdN8kmn\nIOLfQeo2+oBTBh1DvaM97N25n8WTL2DXpgbAc8DxSDfMecAypLvmGSTZT0Y+HuG9XaevhEvay3V1\nAw+l3EK7a+IjIeL3eDzHwV/WQXsPXMMJp0zmxPNSSU0rZN0S+P6LfgQKMkNn5yGtc5CW/x1Igs9A\nZtKMAe4MHX90Fjx2MfQPNeUTrv89Id7/KLk5dtD4E5625JNKv8dhtCejzj669P+Ao4+vw8H9+Xzz\ncVe2/PRPJJlH+hp4OPTvAxR10bwDLF0NFy2Ewrkw41XdwEMpd9Ikn2SOafcrnfoup0ZGHTb/lMuS\naR7y9i9D+udHUbRL04PI4qUUZIOPY5Dk/tbXkD8BPh+XYK11pVQUtLsmPhyP3+vz1wsUHHw5Na3m\n4EB+Ct990Zafl76FFBD7K0ULo6YAHwCpSGL/CdnjZRTw1EwH57rHwvH3PwZujh00/oSnST4+HI3f\n6/N3RybCnxAoyF86a8IXJ+3b2SxNVr0GgLeRKZEAo4GdSElhWdzUosX1/PLLtmEJ2NdeXm7+/Lg5\ndtD4E55217iY1+dPBx5C5kECjJ4+dvzGgrzX/l70VAYyQ2YSskn3SGRGzXTCW/M1bdqQ9evf1P52\npZJQ1EneGPMscCZyJ7zdWrsw4tha4GekGQkw2Fq7MfowVUlen789UinsNGRJ6jU52VlzPM+MuUla\n6E8h0yRvR/rdt4WeCy9wyg/9/WjBxx/frzd7pZJUtNv/9QTustZ6jTEnAq9Za8+JOL4G6Git3V/O\nS7r9K1OVxe/1+T1IMfhnkCb5G8BtOdlZuyFyw5AHe0kf/J0r4acxcMbvYWQPucp9P8LiVGArzLgo\nGAxurqr4K4mbPz9ujh00/oQXbQuuN/AhgLV2hTGmoTGmjrV2b8Q5Sf3GOcHr8zcBxgEXIR3r1+Vk\nZ70XeU4wGMz1eDwDYUGxmjIej2csLNE6M0pVM9Em+WbANxGPtyJTNVZFPDfWGNMKmGOtvTfKn6NC\nvD7/RUiCb4J0rl+bk531S2nnljanXee5K1U9xasv1oN87Ql7EJmrtxOYaIy51Fr7nzKukRDTfGJQ\nKfHnHizgtZzvAEhLTWHohSdx8Xmt+6SkeNbH+Ufp++8cN8cOGr+TyuwxiTbJb0Ra82HNgV/DD6y1\nb4b/bYyZhJQ4LCvJu7l7p1L69bw+/+lIhbD2wLKCQOHg3/VssyzePwf390u6OX43xw4af8KLNslP\nAx4FXjHGdAM2WGv3ARhj6iMVrgZYaw8APZDVN6qcvD5/GrKC6WHkv9EzwP052Vnaj66UqpCoF0MZ\nYx5HEngA2eSzG7DbWjvRGHMbUqR8L7DIWntbGZdz+900bvF7ff7WyMKmc4ANwNCc7KwZ8bj2Eej7\n7xw3xw4af8LTFa/xEXP8oamRQ4AXgbrA+8CwnOysHbGHV6Zq//47yM2xg8af8FKcDkCB1+dvBLwH\nvB56aghwRRUleKVUEtOVjg7z+vx9keTeHJiDrFxd62RMSqnkoUneIV6fPwN4HNmtowApNvNkTnZW\n4IgvVEqpCtAk7wCvz98FqTvTEbDA4JzsrG+O/CqllKo4TfJVyOvzpyB76j0G1ABeBv6Sk51V3ho/\nSilVIZrkq4jX5z8OGA+cD2wBrs/JzvrE2aiUUslOZ9dUAa/PfwWwFEnwOUAnTfBKqaqgLflK5PX5\n6wMvIdsy7QduAv6Zk52VEIsTlFLJT5N8JfH6/D2QlastgQXA1TnZWSudjUopVd1oko8zr89fA6nr\ncw+ymm4kMDInOyvf0cCUUtWSJvk48vr8HZCpkacAa5DW+zxno1JKVWc68BoHwWAQr89/M/AtkuD/\nBXTRBK+Ucpq25GPk9fmbnXpiE5AB1h1I672s2vlKKVUlNMnHbuI3K7YATEe25NvocDxKKfVf2l0T\nuzG3Xd4VYIAmeKVUotF68vGh8TvLzfG7OXbQ+BNe1N01xphngTORN+l2a+3CiGMXAKORXaMmWWtH\nxRqoUkqpiouqu8YY0xNoa609B7gBeKHEKc8DlwDdgX7GmA4xRamUUioq0fbJ9wY+BLDWrgAaGmPq\nABhjWgM7rLUbrLVBYBLQJx7BKqWUqphok3wzYFvE462h58LHtkYc2wIcE+XPUUopFYN4TaE80sBF\neQc1EmIEOAYav7PcHL+bYweN30ll5tdok/xGilruIPuT/hr694YSx1qEniuLm0e43T5Cr/E7x82x\ng8af8KLtrpkG/A+AMaYbsMFauw/AWrsOqGeMOd4YkwZcGDpfKaVUFYt6nrwx5nGgBzJN8magG7Db\nWjvRGHMe8ETo1A+stc+UcTm33001fme5OX43xw4af8LTxVDxofE7y83xuzl20PgTnpY1UEqpJKZJ\nXimlkpgmeaWUSmKa5JVSKolpkldKqSSmSV4ppZKYJnmllEpimuSVUiqJaZJXSqkkpkleKaWSmCZ5\npZRKYprklVIqiWmSV0qpJKZJXimlkpgmeaWUSmIV3v7PGJMOvA60RDYMuc5au6bEOfnAnIin+lhr\nC2OIUymlVBSi2eP1KmCHtXawMaYv8DhwZYlzdllrz485OqWUUjGJprumN/Bh6N8zgO7xC0cppVQ8\nRZPkmwFbAUJdMMHQht2RMowxbxlj5hhj7ow1SKWUUtE5YneNMeYG4MYST59Z4nFp+yP6gDdD/55l\njJllrf0muhCVUkpFq8IbeRtj/gX821o7LTQIu8Za2+II5z8B/GCtfT2mSJVSSlVYNAOv04DLQn97\ngc8iDxpjDPAEcAnSyj8HeD+2MJVSSkUjmiT/LtDXGDMbyAWuBTDG3AN8Ya39yhizApgP5AMfWWsX\nxilepZRSFVDh7hqllFLuoStelVIqiWmSV0qpJKZJXimlklg0A6+VwhjTCxnUvd5a+4nD4ZSbMeZZ\nZO1AELjdbYPMxpjOyArmZ6y1f3c6noowxjwJnIt8jh+31n5YxksShjGmFlIDqgmQAYx00+c+zBiT\nCSwHRlhrxzsdT3mF8s37SOwAy6y1tzkXUcUZYwYDfwEKgIestZNKOy8hkrwxpg1wGzDL6VgqwhjT\nE2hrrT3HGHMi8BoyZdQVQokmG5jqdCwVZYw5H+gYeu+PAhZRVG7DDS4C5ltrnzbGtASmA65L8sAD\nwHakkeM2n1trL3c6iGgYYxoBDwHdgLrAo0CpST5Rums2AJcCe50OpIL+W8fHWrsCaGiMqeNsSBVy\nEEk2m50OJAqzgPD/oLuB2saY0lZfJyRr7XvW2qdDD1sC652MJxqhhs2JyM3JNe99BDfGHHYB8Km1\ndp+1dpO19qbDnZgQLXlrbS6ArKNylWZAZLmGrcAxwCpnwqkYa20ACLjwfQ/Hvi/08AbgE2ut61qT\nxph5wLHIzdZtngJuBq5zOpAoBIGTjDF+4CjgUWvtpw7HVBHHA7VC8TcEHrHWflbaiVWe5A9TD+ch\na+30qo6lEnhw59dW1zLGZAHXA32djiUaoe6mLkitpy5Ox1NexpghwCxr7c9u+gYVYRWSGN83xrQG\nPjfGtLHWFjgdWDmlIDen3wOtgM+RxH+IKk/y1tpxwLgjnOKmJLkRac2HNQd+dSiWascY0x+4Fxhg\nrf3N6XgqwhhzKrDFWrveWrvEGJNmjGlsrd3mdGzlNAhobYy5BGgBHDTGrD9cazLRWGs3Eiq3Yq39\nyRizCflGtc7RwMpvE/BlqBLwT8aY3w73+UmI7poIHtzVTzYNGfB4xRjTDdhgrd1XxmsSkZvecwCM\nMfWR7oLe1tpdTscThfOQltedxpimQB0XJXistf/dKMgY8zBSqNAVCR7AGHMV0M5a+6gxpgkyy2mD\nw2FVxDTg9VAByKM4wucnIZK8Meb3wAjkTtrLGPOItfZ0h8Mqk7X2S2PMN8aYuchWiDc7HVNFGGPO\nAv6JfMALjDE3AT2ttTudjaxcrgAaAe9HjCkMsda6ZQBzLDDOGDMLyAT+5HA81c1HwNvGmDlAKjDc\nRV01WGs3GmM+AL4KPXXL4c7V2jVKKZXEEmUKpVJKqUqgSV4ppZKYJnmllEpimuSVUiqJaZJXSqkk\npkleKaWSmCZ5pZRKYprklVIqif0/4EjD/UHe/hYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcb51947150>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Library Function for linear regression:\n",
    "#\n",
    "# stats.linregress?\n",
    "#\n",
    "\n",
    "# Request rates to nodes in a cluster\n",
    "X = np.loadtxt(\"DataSets/ReqMultiNode.csv\", delimiter=',', usecols=(0,1,2,3,4,5))[:1000]\n",
    "T, A, B, C, D, E = X.T\n",
    "\n",
    "from scipy import stats\n",
    "b, a, r_value, p_value, std_err = stats.linregress(A,B)\n",
    "\n",
    "print \"model = {} + {} * x\".format(a,b)\n",
    "\n",
    "grid = [0,1,2,3,4,5]\n",
    "f    = lambda x: a + b*x\n",
    "\n",
    "plt.scatter(A,B)\n",
    "plt.plot(grid, map(f, grid))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Goodness of fit\n",
    "\n",
    "* Natural measure for goodness of fit is the Loss of the ideal fit viewed as a function of $X$ and $Y$:\n",
    "\n",
    "$$ MinLoss(X,Y) = \\sum_i (y_i - \\hat{b} x_i - \\hat{a})^2 $$\n",
    "\n",
    "* Problem: Scales (quadratic) with $Y$: $MinLoss(X,3 Y)=9 MinLoss(Y)$ without the regression getting better or worse. \n",
    "\n",
    "* Use loss of constant regression (ie. mean value):\n",
    "\n",
    "  $$ ConstLoss(Y) = \\sum_i (y_i - \\mu_y)^2  = n \\cdot Var(Y) $$\n",
    "  \n",
    "**Def:** The $R^2-Value$ value is\n",
    "\n",
    "$$ R^2 = 1 - \\frac{MinLoss(X,Y)}{ConstLoss(Y)}$$\n",
    "\n",
    "* Lies in [0,1].\n",
    "* Perfect fit if $R^2=1$\n",
    "\n",
    "* Scaling invariance: Does not change when $X,Y$ is replaced by $a X, b Y$.\n",
    "\n",
    "* Can be viewed as variance ratio $SS_{reg} / SS_{tot}$.\n",
    "  http://en.wikipedia.org/wiki/Coefficient_of_determination\n",
    "\n",
    "**Proposition:**\n",
    "\n",
    "$$ R^2 = \\frac{Cov(X,Y)^2}{Var(X) \\cdot Var(Y)} = (\\frac{Cov(X,Y)}{\\sigma_X \\cdot \\sigma_Y})^2$$\n",
    "\n",
    "By Cauchy-Schwary inequality we have $0\\leq \\rho^2 \\leq 1$.\n",
    "\n",
    "The (Pearson) _correlation_ $\\rho = \\sqrt{R^2}$ is defined as measures how well a linear model fits the plot."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Examples\n",
    "\n",
    "## Linear regression on memory usage\n",
    "\n",
    "* detemine time to run out of ressources\n",
    "* read off: annual growth (negative in this case)\n",
    "* read off: R2 value\n",
    "\n",
    "<figure>\n",
    "<img src=\"img/example-memory-usage.png\">\n",
    "<figcaption>Linear memory forecast.</figcaption>\n",
    "</figure>\n",
    "\n",
    "* share: https://share.circonus.com/shared/graphs/86d2480c-7729-4a97-bcde-f95ed997aedf/ngagXt#no-set  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exponential regression\n",
    "\n",
    "Use a different Model\n",
    "\n",
    "$$f(t) = exp(a \\cdot t + b)$$\n",
    "\n",
    "Trick: Use log to reduce to linear case.\n",
    "\n",
    "* Forecast exponential growth\n",
    "* Compute Compound Annual Growth rate\n",
    "\n",
    "<figure>\n",
    "<img src=\"img/example_exp_forecast.png\">\n",
    "<figcaption>Exponential user-statistics forecast.</figcaption>\n",
    "</figure>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Ineractive Example: Disc space usage\n",
    "    \n",
    "https://circonus-ops.circonus.com/trending/graphs/view/48970eef-364b-4094-83a2-794232cb1abc#no-set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Holt-Winters forecasing\n",
    "\n",
    "* For periodic workloads.\n",
    "* Forecasts are sensitive to change of parameters, and input data.\n",
    "* Non-parametric / baysiean approach\n",
    "* Extends ideas behind exponential smoothing\n",
    "\n",
    "<figure>\n",
    "<img src=\"img/example_holt-winters.png\">\n",
    "<figcaption>Holt-Winters forecasing of web-request rates.</figcaption>\n",
    "</figure>\n",
    "* graph: https://circonus-ops.circonus.com/trending/graphs/view/ac6c7510-e763-4a2f-b8ab-61ef7b111f09#set-YksJYFQq\n",
    "* share: https://share.circonus.com/shared/graphs/ac6c7510-e763-4a2f-b8ab-61ef7b111f09/7UEs8J#no-set"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}