Testing - pytest.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Testing\n",
    "\n",
    "Purpose: when developping, make sure all the parts of your code are still working fine"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Testing and Debugging\n",
    "\n",
    "`assert`: making sure horrible things don't happen in your code without you knowing about it.<br>\n",
    "`ipdb`: because print statements can only get you so far"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`assert` causes an error when a condition fails, so it's a good way to make sure that if something breaks in your code you know about it right away, before the error propagates to somewhere else. Try out the example below:\n",
    "\n",
    "    def make_neuron_array(array_size):\n",
    "        neuron_array = [0] * array_size\n",
    "        assert len(neuron_array) == array_size\n",
    "        return neuron_array\n",
    "        \n",
    "For info, check out [this article](https://wiki.python.org/moin/UsingAssertionsEffectively)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`ipdb.set_trace()` can be used to open an iPython shell wherever in your code you have a problem. You can also step through the code using `n`. For example, after you've installed `ipdb` using `pip` try running this code and stepping through it:\n",
    "\n",
    "    import ipdb\n",
    "\n",
    "    print(\"start\")\n",
    "    vals = [3, 1, 4]\n",
    "    ipdb.set_trace()\n",
    "    vals[0] += 5\n",
    "    print(\"done\")\n",
    "\n",
    "You can also use ipdb to open a debug shell whenever an exception occurs using the following code, taken from [a StackOverflow question](http://stackoverflow.com/q/4234612/1079075):\n",
    "\n",
    "    import sys\n",
    "    from IPython.core import ultratb\n",
    "    sys.excepthook = ultratb.FormattedTB(mode='Verbose',\n",
    "         color_scheme='Linux', call_pdb=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unit Testing\n",
    "\n",
    "Forcing your code into little functions (that do one thing) so you can test in an automated manner.\n",
    "\n",
    "**Why?**\n",
    "- Forces you to write modular code\n",
    "- Your tests double as documentation for examples on how the code should work\n",
    "- Save time double checking little pieces of your code aren't breaking when you change something"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unit Testing in Python with PyTest\n",
    "\n",
    "Python includes a unit testing module by default, but it has a lot boilerplate. Instead use [PyTest](http://pytest.org/latest/)!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic Testing with PyTest\n",
    "\n",
    "The following examples use the file structure defined in `Installation - setuptools.ipynb`, repeated below:\n",
    "\n",
    "    |- README.md\n",
    "    |- killer\n",
    "    |   |-- __init__.py\n",
    "    |   |-- kill.py\n",
    "    |   | -- tests\n",
    "    |   |   |-- test_kill.py\n",
    "\n",
    "In `test_kill.py` let's make the most basic test possible.\n",
    "\n",
    "    import pytest\n",
    "\n",
    "    import killer.kill as kk\n",
    "\n",
    "    def test_spray():\n",
    "        kk.spray()\n",
    "\n",
    "Note that all test functions have to start with the word `test`. Now run it with `py.test test_kill.py` and watch the test pass."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fixtures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Basic Fixtures**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "In `kill.py` two more functions that take the same argument, but do different stuff with it.\n",
    "\n",
    "    def throw(thing):\n",
    "        print(\"Throwing %s!\" %(thing,))\n",
    "\n",
    "    def kick(thing):\n",
    "        print(\"Kicking %s!\" %(thing,))\n",
    "\n",
    "To test these functions, we don't want to define the argument twice, so instead we use a fixture. Add the following code to `test_kill.py`.\n",
    "\n",
    "    @pytest.fixture\n",
    "    def thing():\n",
    "       return \"socks\"\n",
    "\n",
    "    def test_kick(thing):\n",
    "        kk.kick(thing)\n",
    "\n",
    "    def test_kick(thing):\n",
    "        kk.throw(thing)\n",
    "        \n",
    "PyTest suppresses any print statements, so to convince yourself the argument is really being passed, comment out the fixture and notice that the tests fail."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Nengo Fixtures**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Nengo has some useful fixtures built in for plotting the results of tests, setting random seeds, as well as logging the outputs and speed of test execution. To use them, add `nengo.conftest import *`. To see them in action, just run the tests in Nengo and look at the outputs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classes\n",
    "\n",
    "You can also group tests in a class like so:\n",
    "\n",
    "    class TestBasic(object):\n",
    "\n",
    "        def test_spray(self):\n",
    "            kk.spray()\n",
    "\n",
    "        def test_shoot(self):\n",
    "            kk.shoot()\n",
    "\n",
    "This also means that you can inherit classes for testing. For a good example of how and why to do this, check out [this pull request by Jan](https://github.com/nengo/nengo/blob/465d6a9e7b01caece896adee1916793911f9e5d2/nengo/tests/test_copy.py) where he tests copying Nengo objects with minimal repeating code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using `conftest.py` So You \"Don't Repeat Yourself\"\n",
    "\n",
    "If you need fixtures or classes that should be shared accross various testfiles, you can put them in `conftest.py`. For example, if you needed the `thing` fixture for other testfiles, you could move it to `conftest.py` and it would be found automatically when you ran tests."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Things not covered in this talk, but possible with PyTest\n",
    "\n",
    "- Logging how long tests take\n",
    "- Using command-line options to run tests in specific manners\n",
    "- Running multiple tests at once\n",
    "- Testing for errors and warning"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}