Testing your code - Unit Tests in Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Testing Your Code - Unit Tests in Python\n",
    "This scenario walks you through another key concept in a developer's toolkit: unit tests. As a data scientist, we often don't really think about testing our code, but using unit tests can help you write better code, faster and spend less time debugging.  Using unit tests will also help you keep your code working properly as you modify and improve your code. So let's get started!\n",
    "\n",
    "In this lesson you'll learn:\n",
    "* What are unit tests?\n",
    "* How to write effective unit tests \n",
    "* How to use the `unittest` framework in Python\n",
    "\n",
    "## What Are Unit Tests?\n",
    "A unit test is an automated way of testing small blocks of code (usually functions) by executing that code and compares the behavior with expected results. \n",
    "\n",
    "Let's consider the following scenario: you have some data pipeline that needs an email address as an input. To make sure that the data is valid to begin with, you might create a function called `isValidEmail()` that accepts an email address as an argument and returns `true` or `false` depending on whether the email is valid or not.  This may seem like a simple situation, but how do you know that the function works in all cases?  What happens if the function receives no input?  What happens if the function receives more than one email address? \n",
    "\n",
    "The best way to make sure that your code is going to do what you expect it to do, is to write unit tests to actually test that your code does what you want it to do. \n",
    "\n",
    "### Unit Test Frameworks\n",
    "What I have described seems fairly straightforward, and for simple cases it is, however in every language there are unit test frameworks that can assist you in crafting effective unit tests.  Python has several, but the one that is included with the main python packages is creatively called `unittest`.\n",
    "\n",
    "Most all unit test frameworks have similar core features which are functionalities to:\n",
    "* Setup the environment prior to running tests\n",
    "* Execute code\n",
    "* Compare results with expected results, usually with a collection of `assert()` methods\n",
    "* Clean up environment after running tests\n",
    "\n",
    "Frameworks will have other features as well, but these are the basic core functionalities, regardless of what framework you choose to use.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Writing a Basic Unit Test\n",
    "Let's say that we have a simple function called `addTwo()` which accepts a number and adds 2 to it. In order to write a unit test to verify that the function does what we expect it to do in all cases.  Here is the function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-07-22T17:45:23.078486Z",
     "start_time": "2020-07-22T17:45:23.075433Z"
    }
   },
   "outputs": [],
   "source": [
    "def addTwo(x):\n",
    "    return x + 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, this is a simple function so let's write a unit test to see if in fact the function works. \n",
    "\n",
    "The first thing we have to do is import the `unittest` module and create a class with the tests. After creating a class, the next step is to create methods usually named `testSomething` which describes what is being tested. \n",
    "\n",
    "A good tip is to name your test something descriptive such as `testValidationFunctionWithInvalidInput` rather than `test32` or something like that so that you will know which test failed and can debug faster.\n",
    "\n",
    "```python\n",
    "import unittest\n",
    "\n",
    "class SimpleUnitTest(unittest.TestCase):\n",
    "    \n",
    "    def testWithInt(self):\n",
    "        self.assertEqual(4, addTwo(2))\n",
    "```\n",
    "\n",
    "Every unit test should have at least one call to an `assert()` method. \n",
    "\n",
    "### Notebook Considerations\n",
    "**Note:  Normally, unit tests are contained in a separate file from your actual code, however, since this in a notebook there are a few modifications to the code so that the test runs in the notebook**\n",
    "\n",
    "In otder to run a unittest in a Jupyter you'll have to add the following code in a notebook cell.  The `exit=False` argument prevents the kernel from exiting when the tests are completing which would crash the Jupyter notebook.  In a self-contained script, this is not necessary.\n",
    "\n",
    "```python\n",
    "if __name__ == '__main__':\n",
    "    unittest.main(argv=['ignored'], exit=False)\n",
    "```\n",
    "\n",
    "Try out the examples below and you should see that Jupyter executes two tests and the results are OK."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-10T14:59:18.907723Z",
     "start_time": "2020-05-10T14:59:18.895528Z"
    }
   },
   "outputs": [],
   "source": [
    "import unittest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-10T14:34:31.858327Z",
     "start_time": "2020-05-10T14:34:31.854902Z"
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "class SimpleUnitTest(unittest.TestCase):\n",
    "    \n",
    "    def testWithInt(self):\n",
    "        self.assertEqual(4, addTwo(2))\n",
    "        \n",
    "    def testWithFloat(self):\n",
    "        self.assertEqual(4.0, addTwo(2.0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-10T14:34:32.255313Z",
     "start_time": "2020-05-10T14:34:32.248863Z"
    }
   },
   "outputs": [],
   "source": [
    "if __name__ == '__main__':\n",
    "    unittest.main(argv=['ignored'], exit=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see what happens when a unit test fails:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-10T14:35:51.456957Z",
     "start_time": "2020-05-10T14:35:51.448858Z"
    }
   },
   "outputs": [],
   "source": [
    "class SimpleFailingUnitTest(unittest.TestCase):\n",
    "    \n",
    "    def testFailingTest(self):\n",
    "        self.assertEqual(5, addTwo(2))\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    unittest.main(argv=['ignored'], exit=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this last example we see that the test failed in the output shown below and we do see the cause of the failure.\n",
    "```\n",
    "======================================================================\n",
    "FAIL: testFailingTest (__main__.SimpleFailingUnitTest)\n",
    "----------------------------------------------------------------------\n",
    "Traceback (most recent call last):\n",
    "  File \"<ipython-input-20-f61fda50b2ef>\", line 4, in testFailingTest\n",
    "    self.assertEqual(5, addTwo(2))\n",
    "AssertionError: 5 != 4\n",
    "\n",
    "----------------------------------------------------------------------`\n",
    "```\n",
    "\n",
    "## Testing Failure Conditions\n",
    "This test is not especially helpful but one thing you can do with the frameworks is test HOW your code fails and make sure that is what you want it to do.  For instance, what happens if you execute the following code:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-11T20:40:16.558996Z",
     "start_time": "2020-05-11T20:40:16.395639Z"
    }
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "can only concatenate str (not \"int\") to str",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-2-37c9957a9b71>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0maddTwo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"2\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-1-2af3894357c0>\u001b[0m in \u001b[0;36maddTwo\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0maddTwo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: can only concatenate str (not \"int\") to str"
     ]
    }
   ],
   "source": [
    "addTwo(\"2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The this code fails with an exception because Python cannot convert a `string` to an `int`.  In a real situation, we might not want the code to halt execution with an exception due to bad input, or we might want to provide a more helpful error message. \n",
    "\n",
    "The `unittest` framework provides a series of `assert` functions which enable you to test a variety of situations including:\n",
    "* `assertEqual(a,b)`: tests that a = b\n",
    "* `assertNotEqual(a,b)`: tests that a != b\n",
    "* `assertTrue(a)`: tests that a is `True`\n",
    "* `assertFalse(a)`: tests that a is `False`\n",
    "* `assertRaises(ErrorType)`: tests that the unit test fails with a specific error type\n",
    "\n",
    "For a complete list of `assert` methods look in the documentation here: https://docs.python.org/3.8/library/unittest.html#unittest.TestCase.assertTrue\n",
    "\n",
    "Let's try now writing a unit tests to make sure that the error returned with invalid input is in fact a `TypeError`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-10T14:59:48.993133Z",
     "start_time": "2020-05-10T14:59:48.990155Z"
    }
   },
   "outputs": [],
   "source": [
    "class SimpleExceptionTest(unittest.TestCase):\n",
    "    \n",
    "    def testInvalidInput(self):\n",
    "        with self.assertRaises(TypeError):\n",
    "            addTwo(\"2\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-10T14:59:49.878972Z",
     "start_time": "2020-05-10T14:59:49.873132Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.001s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "if __name__ == '__main__':\n",
    "    unittest.main(argv=['ignored'], exit=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the example above we see that even though the code fails, the unit test passes because the code returns the correct type of error. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing Effective Unit Tests\n",
    "Unit tests are intended for you, the data scientist, to help you make sure your code is doing what you want it to do, so here are a few thoughts on writing effective unit tests:\n",
    "\n",
    "* One assertion per unit test: While you can include multiple assertions in a unit test, it can make it difficult to figure out the cause of the failure, so keep the unit test to one assertion\n",
    "\n",
    "* Keep them short:  Unit tests should not be lengthy demonstrations of code cleverness. They should be simple, quick and easy to understand. If your tests are too complicated, sometimes you can spend hours trying to fix an issue that in fact was not in your code, but rather in your test.\n",
    "\n",
    "* Write unit tests as you go: Unit tests should not be an after thought. You should write them as you so that you identify bugs early and often. \n",
    "\n",
    "* Name your tests clearly:  As mentioend earlier, you should name your tests clearly so that it is obvious what failed.  \n",
    "\n",
    "* Test failure as well as success: When writing unit tests, think creatively about what could go wrong and write a test for that.  For instance, if you're populating a dataframe with data pulled from an external source, what happens if the source is unreachable?  What happens if the source changes their data format? Carefully considering what can go wrong can prevent disasters later.\n",
    "\n",
    "* Don't forget edge cases:  These are actually some of the most important unit tests. Give some thought to what edge cases exist and make sure you write a unit test for them.\n",
    "\n",
    "Finally, don't ignore the results of your tests.  If they aren't all passing, you're not done. Don't move on until ALL your unit tests pass."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final Thoughts\n",
    "This exercise has demonstrated how to write simple unit tests to debug code. There are a few topics which are beyond the scope of this tutorial which are:\n",
    "\n",
    "* `setUp()` and `teardown()` methods:  These methods can be added to your unittest class for code that needs to be run before or after the tests are executed. [1]\n",
    "* Organizing your code in modules: This scenario demonstrated unit tests in a notebook context. Normally, unit tests are run in a separate script.  You can read more here: [2]\n",
    "* Dynamic tests: All the tests you saw used static input.  It is possible to run your unit test with dynamic input.\n",
    "* Doctest: It is possible to include unit test instructions directly in your code with `doctest`.\n",
    "\n",
    "\n",
    "[1]: https://docs.python.org/3.8/library/unittest.html#unittest.TestCase.setUp\n",
    "[2]: https://docs.python.org/3.8/library/unittest.html#organizing-tests\n",
    "\n",
    "## Conclusion\n",
    "In this short scenario, you learned how to use unit tests to test and debug your code. You learned how to use the `unittest` framework to craft these tests and test all kinds of conditions.  For further reading on the topic here are a few tutorials and references:\n",
    "\n",
    "* https://realpython.com/python-testing/\n",
    "* https://docs.python.org/3.8/library/unittest.html#module-unittest\n",
    "* https://docs.python-guide.org/writing/tests/\n",
    "* https://jeffknupp.com/blog/2013/12/09/improve-your-python-understanding-unit-testing/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}