11_python_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "# Data science in Python\n",
    "\n",
    "- Course GitHub repo: https://github.com/pycam/python-data-science\n",
    "- Python website: https://www.python.org/ \n",
    "\n",
    "## Session 1.1: Starting with data and Python\n",
    "- [Jupyter notebook](#Jupyter-notebook)\n",
    "  - [Exercise 1.1.1](#Exercise-1.1.1)\n",
    "- [Shell commands](#Shell-commands)\n",
    "  - [Exercise 1.1.2](#Exercise-1.1.2)\n",
    "- [Basic Python](#Basic-Python)\n",
    "  - [Exercise 1.1.3](#Exercise-1.1.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mind map\n",
    "\n",
    "<img src=\"img/mind_maps/mind_maps.001.jpeg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Jupyter notebook\n",
    "\n",
    "<img src=\"http://jupyter.org/assets/nav_logo.svg\">\n",
    "\n",
    "- The [Jupyter Notebook](http://jupyter.org/) is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. \n",
    "\n",
    "- Jupyter provides a rich architecture for interactive data science and scientific computing with: \n",
    "    - Over 40 programming languages such as Python, R, Julia and Scala.\n",
    "    - A browser-based notebook with support for code, rich text, math expressions, plots and other rich media.\n",
    "    - Support for interactive data visualization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to install Jupyter on your own computer?\n",
    "\n",
    "- We recommend using a virtual environment after having installed [Python 3](https://www.python.org/) on your computer\n",
    "```bash\n",
    "python3 -m venv venv\n",
    "source venv/bin/activate # activate your virtual environment\n",
    "```\n",
    "- Install Jupyter:\n",
    "```\n",
    "pip install jupyter\n",
    "```\n",
    "- Start the notebook server from the command line:\n",
    "```\n",
    "jupyter notebook\n",
    "```\n",
    "- You should see the notebook home page open in your web browser."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to run Python in a Jupyter notebook?\n",
    "\n",
    "- See [Jupyter Notebook Basics](http://nbviewer.jupyter.org/github/jupyter/notebook/blob/master/docs/source/examples/Notebook/Notebook%20Basics.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1.1.1\n",
    "- Create a new Jupyter notebook with one Markdown cell and one Python Code cell. Run the code.\n",
    "- Download the python file associated with the notebook newly created"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Shell commands \n",
    "\n",
    "- Three commands\n",
    "  - `pwd` to print working directory\n",
    "  - `ls` to list content of a directory\n",
    "  - `cd` to change directory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run the Python interpreter\n",
    "\n",
    "On a Mac or Linux machine you first have to open a terminal and then type the command `python3`.\n",
    "<center><img src=\"img/python_shell.png\"></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run Python code from a file\n",
    "\n",
    "For running Python code from a file, open a Terminal window and type the command `python3` or just `python` followed by the name of the script or file that contains Python code.\n",
    "\n",
    "```\n",
    "python3 scripts/hello.py\n",
    "```\n",
    "\n",
    "Please, make sure that you are running the version 3 of Python:\n",
    "```\n",
    "python --version\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Exercise 1.1.2\n",
    "- Find in a terminal window the Python script downloaded from Jupyter notebook and execute it.\n",
    "```\n",
    "python3 my-script.py\n",
    "```\n",
    "- List all data files in `data/` folder from the course materials and find our first data file `gapminder.csv`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Python\n",
    "\n",
    "### Cheat Sheet\n",
    "\n",
    "- [Cheat Sheet](cheat_sheet_basic_python.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### For loops\n",
    "\n",
    "The **`for` loop** in Python iterates over each item in a collection (such as a list) in the order that they appear in the collection. What this means is that a variable (`colour` in the below example) is set to each item from the collection of values in turn, and each time this happens the indented block of code is executed again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_colours = ['red', 'blue', 'green']\n",
    "for colour in all_colours:\n",
    "    print(colour)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Files\n",
    "\n",
    "To read from a file, your program needs to open the file and then read the contents of the file. You can read the entire contents of the file at once, or read the file line by line. The **`with`** statement makes sure the file is closed properly when the program has finished accessing the file.\n",
    "\n",
    "\n",
    "Passing the `'w'` argument to `open()` tells Python you want to write to the file. Be careful; this will erase the contents of the file if it already exists. Passing the `'a'` argument tells Python you want to append to the end of an existing file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# reading from file\n",
    "with open(\"data/genes.txt\") as f:\n",
    "    for line in f:\n",
    "        print(line.strip())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# printing only the gene name and the chromosome columns\n",
    "with open(\"data/genes.txt\") as f:\n",
    "    for line in f:\n",
    "        data = line.strip().split()\n",
    "        print(data[0], data[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conditional execution\n",
    "\n",
    "A conditional **`if/elif`** statement is used to specify that some block of code should only be executed if a conditional expression evaluates to `True`, there can be a final **`else`** statement to do something if all of the conditions are `False`.\n",
    "Python uses **indentation** to show which statements are in a block of code. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# printing only the gene name and its position for chromosome 6\n",
    "with open(\"data/genes.txt\") as f:\n",
    "    for line in f:\n",
    "        data = line.strip().split()\n",
    "        if data[1] == '6':\n",
    "            print(data[0], data[2], data[3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting help\n",
    "\n",
    "[The Python 3 Standard Library](https://docs.python.org/3/library/index.html) is the reference documentation of all libraries included in Python as well as built-in functions and data types.\n",
    "\n",
    "For example, to get help for the `split()` function, you can look at the [Python documentation]((https://docs.python.org/3/library/index.html)) and search for [`str.split()`](https://docs.python.org/3/library/stdtypes.html?highlight=split#str.split)\n",
    "\n",
    "The Basic Python [Cheat Sheet](cheat_sheet_basic_python.ipynb) is a quick summary based on the course ['Introduction to solving biological problems with Python'](http://pycam.github.io/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every Python object from the standard library should have documentation (a *docstring*) available that can be viewed in an interactive Python session. Well developed 3rd party packages will have this too, but it depends on the developers.\n",
    "\n",
    "**Shift+Tab** will pop up a little window giving this information on the object under the cursor. this is good for getting a quick reminder about how a function works, for example.\n",
    "\n",
    "To print the full docstring:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Works in any python instance\n",
    "help(len)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Works in Jupyter only, but has better formatting\n",
    "len?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Any* object may have a docstring."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Methods\n",
    "str.split?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# variables\n",
    "x = 123\n",
    "x?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1.1.3\n",
    "\n",
    "We are going to look at a [Gapminder](https://www.gapminder.org/) dataset, made famous by Hans Rosling from his Ted presentation [‘The best stats you’ve ever seen’](http://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen).\n",
    "\n",
    "- Read data from the file `data/gapminder.csv`.\n",
    "- Find which European countries have the largest population in 1957 and 2007."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "## Next session\n",
    "\n",
    "Go to our next notebook: [Session 1.2: Using existing python modules to explore data in files](12_python_data.ipynb)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}