17 - Python 10min a day - List comprehensions and map - an amazing shortcut.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"assets/jeremy-lapak-CVvFVQ_-oUg-unsplash.png\" alt=\"Python Envs\" style=\"display: block; margin: 0 auto\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Learning Python 10 minutes a day #17\n",
    "## List comprehensions and map: an amazing shortcut\n",
    "[Medium article link](https://towardsdatascience.com/learning-python-10-minutes-a-day-17-10fd0642a202)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a [series](https://python-10-minutes-a-day.rocks) of short 10 minute Python articles helping you to get started with Python. I try to post an article each day (no promises), starting from the very basics, going up to more complex idioms. Feel free to contact me on [LinkedIn](https://www.linkedin.com/in/dennisbakhuis/) for questions or requests on particular subjects of Python, you want to know about."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the key elements of Python is its readability. The language forces you to apply structure using indentation which is in my opinion very pleasing to the eye. The key ideas of Python can also be found in [the zen of Python](https://en.wikipedia.org/wiki/Zen_of_Python). It was included as one of many Easter eggs in the interpreter (just type ‘import this’). One of the idioms that are pretty cool (and very Pythonic) are comprehensions. Comprehensions, or more often called list comprehensions as they are most often used with lists, is a somewhat abstract way to create lists. They combine some sort of processing step, together with a loop, to populate a new list. To make this more clear, lets compare the traditional step to the cool kid on the block: a list comprehension:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# traditional\n",
    "squared_numbers = []\n",
    "for number in range(10):\n",
    "    squared_numbers.append(number * number)\n",
    "print(squared_numbers)\n",
    "\n",
    "# comprehension magic\n",
    "more_squares = [number * number for number in range(10)]\n",
    "print(more_squares)\n",
    "\n",
    "# using your custom functions\n",
    "def add_one(value):\n",
    "    return value + 1\n",
    "\n",
    "one_larger = [add_one(value) for value in range(2, 22, 2)]  # range starts from 2 until (but not including) 22 using step 2\n",
    "print(one_larger)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the traditional loop, you first need to define an empty list. Next is a for-loop to iterate over all calculations, and appending this all to the fresh new list. These three lines can be combined into a single line using a list comprehension, but without sacrificing readability. Inside the list comprehension, i.e. inside the definition of the list identified by brackets, there is a for-loop. This for-loop defines a variable used for the iteration which we can freely choose. Often we are lazy and we choose short letters such as x, y, and z. Still, using descriptive names is a better practice. The for-loop acts identical to a regular for-loop and iterates over the full iterable. We can zip multiple iterables or use enumerate but we need to unpack all the additional variables as well (or use them as a tuple). A good thing to remember is that a list comprehension creates a new list, even if we copy each item and have identical lists. Comprehensions work almost anywhere and can be used in functions. They also work with dictionaries and sets, but of course, the definition must be correct for each data type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# list comprehensions always generates new lists\n",
    "list_a = [x for x in range(10)]\n",
    "list_b = [x for x in list_a]  # exact copy\n",
    "print('id list_a:', id(list_a))\n",
    "print('id list_b:', id(list_b))\n",
    "\n",
    "# iterate over multiple values\n",
    "list_c = [(x, y) for x, y in enumerate(range(10, 0, -1))]\n",
    "print(list_c)\n",
    "\n",
    "# comprehension using dictionaries\n",
    "dict_a = {key: value for key, value in list_c}  # we unpack both values from the previous tuple\n",
    "print(dict_a)\n",
    "\n",
    "# works with any data type\n",
    "sentence = ' '.join([x.upper() for x in ['hi', 'everybody', '!!!']])\n",
    "print(sentence)\n",
    "\n",
    "letters = 'abcdefabcdef'\n",
    "unique = {letter for letter in letters}  # does not make sense as set(letters) would do the same\n",
    "print(unique)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A comprehension in general has the syntax: expression for item in iterable. As the result of the comprehension is again an iterable, we can also nest multiple comprehension into one. While this is perfectly fine, keep in mind that readability might go down. Therefore, changing your mega-one-liner into two lines is not a bad habit."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the same fashion as inline for-loops, we can have inline if-statements. I think for the general usage, inline if-statements add clutter to your code and make it less readable. We can however use the if statements in a comprehension and for that use case, they are occasionally pretty neat."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# nested comprehensions can be useful but I advice to keep it to a minimum\n",
    "my_list = [x for x in [y for y in range(10)]]\n",
    "print(my_list)\n",
    "\n",
    "another_list = [x + 1 for x in range(5) for y in range(3)]\n",
    "print(another_list)\n",
    "\n",
    "# order matters\n",
    "yet_another = [x + 1 for y in range(3) for x in range(5)]\n",
    "print(yet_another)\n",
    "\n",
    "# keep an eye for brackets\n",
    "nested_list = [[x + 1 for x in range(5)] for y in range(3)]\n",
    "print(nested_list)\n",
    "\n",
    "# comprehensions can be used to 'flatten' lists\n",
    "flatted = [item for sublist in nested_list for item in sublist]\n",
    "print(flatted)\n",
    "\n",
    "# inline if-statements\n",
    "raining = True\n",
    "umbrella = True if raining else False  # this is a usage, which I find less readable (my opinion)\n",
    "\n",
    "# inline if-statements in comprehension are occasionally great\n",
    "import random\n",
    "random_numbers = [round(random.random() * 100) for x in range(10)]\n",
    "print(random_numbers)\n",
    "\n",
    "under_thirty = [value for value in random_numbers if value < 30]\n",
    "print(under_thirty)\n",
    "\n",
    "# inline can also change results\n",
    "male_female = ['male' if value < 50 else 'female' for value in random_numbers]\n",
    "print(male_female)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List comprehensions are a great way to shorten your code and generally, without the sacrifice of readability. Together with the inline if-statements, it is a simple way to create a filtered list of items. Another, very similar method is the map() function. This functions maps a function over each item of the list and returns a new list with all the returned values. The same can be achieved with a for-loop or with a comprehension, but the map function can be a bit more readable and therefore, I think it is useful to know about:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def square_number(x):\n",
    "    return x * x\n",
    "\n",
    "result = map(square_number, range(10))\n",
    "print(list(result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The map() function takes a reference to a function as the first argument, and an iterable as the second. The result it returns is however not a list, but a map-object. The map-object is a sort of generator, so it only calculates the value when it is requested. This is in most cases more efficient and consumes less memory. To view the complete list, we convert the map-object into a list. Working with data that is easily mapable as we have done here, is also easier to parallelize (run on multiple cores of your computer in parallel). Python has the multiprocessing package that offers a multi-processing map function for tasks that are CPU-intensive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice for today:\n",
    "Comprehensions are a great way to filter data that are structured in lists or combine similar tasks. Common things I do are for example reading CSV files in a list comprehension, resulting in a list of CSV Pandas DataFrames. Using the concat function in Pandas, I can merge all imported files in a single line to have one combined DataFrame with all data. Another great usage is to selectively put values into a list. This is what we are going to practice today and use it to approximate pi.\n",
    "\n",
    "### Assignment:\n",
    "Today we are going to approximate the value of π using the [Leibniz formula](https://en.wikipedia.org/wiki/Leibniz_formula_for_%CF%80). The Leibniz formula is an alternating series (it changes sign each value) and when summing the series for an infinite amount of values, you get pi. The problem is that we do not have infinite memory and therefore, we truncate the value and approximate pi. The Leibniz formula states that pi over 4 is equal to the alternating series given below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "π / 4 = 1/1 - 1/3 + 1/5 - 1/7 + 1/9 - 1/11 + ....."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To solve this problem we can divide it in a couple of steps:\n",
    "1. create a list comprehension for all odd numbers\n",
    "2. using another list comprehension make each second value negative\n",
    "3. using another list comprehension, convert each value as 1 over x\n",
    "4. sum the complete list and multiply by four to get an approximation of π\n",
    "5. increase the amount of numbers (n) to get a better approximation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hints:\n",
    "1. Use range to get a list of numbers up to a value n (range(n)).\n",
    "2. Use the modulus of 2 to check if it is even/odd\n",
    "3. To get each second number, use enumerate together with the modulus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "actual_pi = math.pin = 100  # amount of numbers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I have posted a [solution](https://gist.github.com/dennisbakhuis/a4fe1d3feac0e46bcb76b6c223ccfabd) on my Github."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have any questions, feel free to contact me through [LinkedIn](https://www.linkedin.com/in/dennisbakhuis/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}