19 - Python 10min a day - Reading errors and exceptions in Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"assets/jeremy-lapak-CVvFVQ_-oUg-unsplash.png\" alt=\"Python Envs\" style=\"display: block; margin: 0 auto\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Learning Python 10 minutes a day #19\n",
    "## Reading errors and exceptions in Python\n",
    "[Medium article link](https://towardsdatascience.com/learning-python-10-minutes-a-day-19-4d688c4101ce)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a [series](https://python-10-minutes-a-day.rocks) of short 10 minute Python articles helping you to get started with Python. I try to post an article each day (no promises), starting from the very basics, going up to more complex idioms. Feel free to contact me on [LinkedIn](https://www.linkedin.com/in/dennisbakhuis/) for questions or requests on particular subjects of Python, you want to know about."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we code it is inevitable to make mistakes. These mistakes make Python generate huge print-outs and these can be somewhat overwhelming. The goal of today is to get a bit more familiar with the error readout and learn that they are actually quite descriptive. Below is a code that has an relatively easy to spot error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sub_function(argument):\n",
    "    \"\"\"\n",
    "    A function with a mistake\n",
    "    \"\"\"\n",
    "    pint(argument)\n",
    "\n",
    "def my_function(argument):\n",
    "    \"\"\"\n",
    "    A function that links to the sub_function\n",
    "    \"\"\"\n",
    "    sub_function(argument)\n",
    "    \n",
    "my_function('I want to print this')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have two functions which are nested into each other and are called sequentially. First we call my_function and provide a string. In my_function we call another function called sub_function. While creating the sub_function we did not have enough coffee yet and made a typo: print -> pint. This will create this large print-out:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"assets/day19-error.png\" alt=\"Python Envs\"  width=\"1200\" style=\"display: block; margin: 0 auto\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a lot of information for one missing letter and for new-comers maybe a bit daunting. In IPython and Jupyter this message, officially called a traceback, is parsed in color. If you would run vanilla Python, this would all be in the same color which looks even more scary. Still, this message is actually quite informative and with the right steps, is really easy to read."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A Python traceback is read from bottom to top. The bottom line tells you the name of the exception and a short description. In many cases, this is all you need to know. In this example, we get an NameError and tells us that the name ‘pint’ is not defined. When I code, I test my code constantly and therefore, remember that I just added a line with a print() function. So, such a small typo is identified rather quickly. If you are not sure where this exception happened, Python gives us additional information, which is divided in levels of code depth. Each time, we call a function, we go one level deeper. The base level is the main code level in which we called the my_function() function. The my_function introduces another level where another function call to sub_function is made. The sub_function creates the third and final level where the error happened. These three levels are visible in the traceback, with the deepest level (where the error actually happened) at the bottom. In each of the levels, an arrow is visible to point out the line in which the error happened. It also prints out line numbers and two lines before and after the line with the error to make the piece better identifiable in the actual code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You might ask why we need the higher code levels if the error happened in the bottom. The reason is that the error happened, because we might supply the wrong type of argument in an earlier step (higher level). Lets create another faulty piece of code and go through the traceback step by step:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sub_function(number_in_string):\n",
    "    \"\"\"\n",
    "    A function with a mistake\n",
    "    \"\"\"\n",
    "    print(int(number_in_string))\n",
    "\n",
    "def my_function(argument):\n",
    "    \"\"\"\n",
    "    A function that links to the sub_function\n",
    "    \"\"\"\n",
    "    sub_function(argument)\n",
    "    \n",
    "my_function('25beer3')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"assets/day19-error2.png\" alt=\"Python Envs\"  width=\"1200\" style=\"display: block; margin: 0 auto\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have to analyze this traceback from bottom to top. First (1) we see that we have a ValueError. A value error is raised when a function or class gets a value it does not expect. In this case, the int() class expected a string representation of a number. The description makes this a bit more clear: invalid literal for int() with base 10: ‘25beer3’. Indeed, it is ambiguous to make an integer from that string. The base 10 is added, because numbers can also be base 2 (binary), base 8 (octal), or base 16 (hexadecimal)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we need to check where this error occurs. In the first block from the bottom (2) we indeed see that we try to convert the variable number_in_string to an integer. Code-wise, there is nothing wrong here, so we go one level higher (3). In this function, there is again nothing wrong so we move on to the top level (4). Here we have found the error: we provided the wrong type of argument."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example it quite easy to fix as we have provided a faulty string ourselves. In real-life situations, this string might be extracted from another string imported from a file, so no that visible. For example, the CSV file we are reading might not have sensible (just plain faulty) values for all rows. The solution is to check if the string is what we expect before the first function call and otherwise substitute the string. Depending on how likely a function can get faulty parameters, tests can be added. Developers have solved this problem using unit tests. Tests to check if the function works as expected. In these unit tests, we can all test the data for validity. My opinion is that you need to use common sense on what to test and what you can assume to be fine. And if shit hits the fan, you always have the trustworthy traceback to guide you."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The method described in this article is a typical workflow to find the mistake in your code. Most of the time, the bottom message is all you need. If you need to inspect the position were the exception occurred, it is because we provided the wrong sort of arguments to a function. The trick is to find the function which caused the problem. The traceback is a perfect tool to find the problematic function. These types of errors are often fixed by quickly inspecting the DocString of the function you want to call."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice for today:\n",
    "Finding errors is a skill that every programmer has to master. Below I have provided an terrible piece of Monday morning code. Use the error dialog to find and squash all the bugs. Try not to directly find them, but really use the traceback to identify the errors to get some practice with its format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy import zeros\n",
    "\n",
    "\n",
    "def make_empty_structure(shape):\n",
    "    \"\"\"\n",
    "    Create empty Numpy array\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    shape : tuple\n",
    "      tuple containing the rows and columns\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    Numpy array\n",
    "    \"\"\"\n",
    "    rows, columns = shape\n",
    "    structure = zeros(rows, columns)\n",
    "    return structure\n",
    "    \n",
    "def combine_structures(structure1, structure2):\n",
    "    \"\"\"\n",
    "    Combine two Numpy arrays\n",
    "    \"\"\"\n",
    "    combined == structure1 + structure2\n",
    "    return combined\n",
    "\n",
    "def add_one_to_structure(structure):\n",
    "    \"\"\"\n",
    "    Add one to Numpy array\n",
    "    \"\"\"\n",
    "    added = structure 1\n",
    "    return added\n",
    "\n",
    "def generate_structures_and_add(shape1, shape2):\n",
    "    \"\"\"\n",
    "    Generate combined structure\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    shape1, shape2 : tuple\n",
    "      tuples of (rows, columns) -> shape1 must be identical to shape2\n",
    "    \"\"\"\n",
    "    structure1 = make_empty_structure(shape1)\n",
    "    structure2 = make_empty_structure(shape2)\n",
    "    added1 = add_one_to_structure(structure1)\n",
    "    added2 = add_one_to_structure(structure2)\n",
    "    combined = combine_structures(added1, added2)\n",
    "    return combined\n",
    "\n",
    "# Main code\n",
    "shape1 = (5, 5, 3)\n",
    "shape2 = (4, 4, 3)\n",
    "generate_structures_and_add(shape1, shape2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have any questions, feel free to contact me through [LinkedIn](https://www.linkedin.com/in/dennisbakhuis/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}