Data_Science_for_Customer_Insights_Python_Introduction_Course.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Data Science for Customer Insights - Python Introduction Course",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/JasperLS/data_science_intros/blob/main/Data_Science_for_Customer_Insights_Python_Introduction_Course.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5fCEDCU_qrC0"
      },
      "source": [
        "#Data Science for Customer Insights - Python Introduction Course\n",
        "\n",
        "Welcome to Data Science for Customer Insights. In the following tasks, you will learn how to do Data Science with Python. In the end, you will be able to: \n",
        "\n",
        "- do basic coding and use Python loops, lists, functions and more\n",
        "- access and apply machine learning algorithms to data sets\n",
        "- understand fundamental machine learning concepts\n",
        "\n",
        "Please go trough all tasks and answer the questions carefully. **This is task-based learning - you will have to Google a lot**. You can use any credible source material, you like, e.g.: \n",
        "\n",
        "- stackoverflow\n",
        "- SoloLearn\n",
        "- DataCamp\n",
        "- udemy\n",
        "\n",
        "Again - you will potentially have to Google everything, this is normal!\n",
        "\n",
        "Also, every time you see those signs \"**...**\" you have to change them to generate the correct code. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UQWo_jfy4H9s"
      },
      "source": [
        "**Before you start: Please provide your information**\n",
        "\n",
        "Name:\n",
        "\n",
        "Matr.:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GJBs_flRovLc"
      },
      "source": [
        "##First Steps\n",
        " "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p4NFErGadias"
      },
      "source": [
        "**Print Function**\n",
        "\n",
        "By using the print statement, we're able to output text. \n",
        "\n",
        "*Please run the following code*:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gJr_9dXGpJ05"
      },
      "source": [
        "print ('Hello World')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L3qFPIIqrc77"
      },
      "source": [
        "<img src=\"https://miro.medium.com/max/1250/1*iNw33pBEYZIAoE9oHdhLLw.jpeg\" width=500 height=330>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2fhs6GZ4qFMx"
      },
      "source": [
        "<strong>Basic Operations</strong>\n",
        "\n",
        "With Python, you can easily do some calculations. \n",
        "\n",
        "*Tap around to perform an addition with a result of 5:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c6i-78HEXGNu"
      },
      "source": [
        "3 "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0DsykyJTXNao"
      },
      "source": [
        "*Try to print a substraction with 7 as a result:*\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B3fhbNoMXLuf"
      },
      "source": [
        "10"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8QlK460xXJqJ"
      },
      "source": [
        "*Use parantheses to determine which operations are performed first - try to get 30:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-gE-Ez1qtyIA"
      },
      "source": [
        "3*4+6"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bX2Xoej0WmJn"
      },
      "source": [
        "*Try to divide by 2 to get a result of 2.00*\n",
        "\n",
        "**<strong><font color='blue'> Side Note:** we call 2.00 a float variable, while 2 is called an integer variable. Despite seemingly being the same, Python sees them as different datatypes which for example leads to differnt memory needs.</font></strong>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4ty9qsqVWVq7"
      },
      "source": [
        "4"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lSrWNr3MuFUS"
      },
      "source": [
        "The minus sign symbolizes a negative number which has no effect on how to perform an operation with them. \n",
        "\n",
        "*Tap around and try to get a positive result of 48!*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WgHrAlz6ZTB-"
      },
      "source": [
        "(-3+-1)*(12)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Wt_Y_ZWA9miH"
      },
      "source": [
        "**Comments**\n",
        "\n",
        "You can comment your code, to explain something for later reference.\n",
        "\n",
        "*Please turn \"This is just a comment\" into a proper comment.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ssxN9tka9-2o"
      },
      "source": [
        "4 + 4\n",
        "This is just a comment"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4i-XuhKW3owN"
      },
      "source": [
        "**Storing Values**\n",
        "\n",
        "You can store values in variables.\n",
        "\n",
        "*Please complete the code to generate 42 as an output.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T29nLva63oG7"
      },
      "source": [
        "... = 21\n",
        "\n",
        "... = ... * ...\n",
        "\n",
        "print(...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jhy8suwbZtde"
      },
      "source": [
        "**ZeroDivisionError**\n",
        "\n",
        "Dividing by zero produces an error, as no answer can be calculated. \n",
        "\n",
        "*You don't believe that? Tap around and divide 3 by zero.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ghD_v1QJaWhM"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8bPzVa96a-zX"
      },
      "source": [
        "**Other numerical Operations**\n",
        "\n",
        "You can also use Python for exponentiation or the quotient and remainder of a division.\n",
        "\n",
        "*Perform an exponentiation with a result of 32:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vHWTZFAsbnrU"
      },
      "source": [
        "2"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e0KghhEMdLLP"
      },
      "source": [
        "*Try to get the remainder of the division 1.25 by 0.5 (should be 0.25):*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cKOpQ-2ebvmM"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OSqSFbg6jRw7"
      },
      "source": [
        "##Data Types"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KA7WEmmTjbRl"
      },
      "source": [
        "If Python handles data, it has to somehow turn it into 1s and 0s. Depending on the type of data, Python uses different internal representations. It is therefore very important, to always make sure your data has the right data type."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fxXZEWORXump"
      },
      "source": [
        "**Integers**\n",
        "\n",
        "Integers are whole numbers like: \n",
        "\n",
        "- 5 \n",
        "- 3\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SpL66XSAV3PW"
      },
      "source": [
        "**Floats**\n",
        "\n",
        "Numbers that aren't integers are represented as floats in Python (\"a floating point number\"). Some examples are: \n",
        "- 0.8\n",
        "- -5.346\n",
        "\n",
        "How to create a float? \n",
        "By entering a number with a decimal point, or by using operations such as division on integers as well as operations on a float and an integer.\n",
        "\n",
        "*Please build a float dividing 12 by 4 and try to convert it back into an integer (should be 3 not 3.0):*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pN81xZjUXB5Z"
      },
      "source": [
        "# create float\n",
        "print()\n",
        "# turn it into an integer\n",
        "print()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VprZLg-EY5s-"
      },
      "source": [
        "**Strings**\n",
        "\n",
        "You have to use a string if you want to use text in Python. It's one of the basic things that store text like: \n",
        "\n",
        "- Python is fun\n",
        "- I love Data Science\n",
        "\n",
        "How to create a string? By entering text between two single or double quotation marks.\n",
        "\n",
        "*Run the code to create a string:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "98w-iUSGZ6ko"
      },
      "source": [
        "\"Python is fun\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JX8ScvvleqQO"
      },
      "source": [
        "*Try to print I love Data Science using single marks.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lt-l5lUJaBfd"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VeQ8nTgbbyur"
      },
      "source": [
        "*What's wrong with the following string? Please correct it so that the type is 'str' and there is no error message!*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zFxpt_tYaiPd"
      },
      "source": [
        "type('Lisa's brother likes good food.')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IgVEuFwsb2hE"
      },
      "source": [
        "*Please change the following sentences to bring the second sentence in the second line:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_TdTOOA1cC7c"
      },
      "source": [
        "print ('Lisa likes good food. In the future, he wants to open a restaurant.')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "We12AcbmdcPU"
      },
      "source": [
        "**Booleans&Comparisons**\n",
        "\n",
        "You've already learned about integers, floats and strings. In Python, another type is the Boolean type. It has two values: True and False. By comparing values (using the equal operator ==), you can create Booleans. \n",
        "\n",
        "*Please define my_boolean as True:*  "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S7bs7aEXgjML"
      },
      "source": [
        "my_boolean = \n",
        "print(my_boolean)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G-Uv7DJigzb9"
      },
      "source": [
        "*Please compare two numbers and see if you get the \"True\" statement:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "As8PeDsQnU3P"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "61liw4eOhy2i"
      },
      "source": [
        "*Is \"hi\" equal to \"hello\"? Try it out!*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0srfVv0vnmYr"
      },
      "source": [
        "\"hi\" ... \"hello\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6c-qfGoloEBC"
      },
      "source": [
        "The not equal operator (!=) is another comparison operator. \n",
        "\n",
        "*Please run the code:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zFYUiLPWoLFX"
      },
      "source": [
        "3!=3"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "siEObqC2iNHa"
      },
      "source": [
        "*Please generate a True statement using the operator:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "owFd0uatoOFn"
      },
      "source": [
        "... != ..."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "crYwcDotoYtw"
      },
      "source": [
        "In Python, you can also use operators (> and <) to determine whether one number (float or integer) is smaller than or greater than another.\n",
        "\n",
        "*Tap around (change the ...) and generate one True as well as one False statement:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RV-FHVego3uL"
      },
      "source": [
        "24 ... 16"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KPY852q_o9rJ"
      },
      "source": [
        "6 ... 6.0"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "agDDzvQlrHxO"
      },
      "source": [
        "**<strong><font color='blue'> Side Note: In the first lessons you've learned about the basic operations in Python. To be better prepared for the following steps, the table below provides a summary: </font></strong>**\n",
        "\n",
        "<img src=\"https://i1.wp.com/makemeanalyst.com/wp-content/uploads/2017/06/Assignment-Operator.png?resize=590%2C426\" width=500 height=300>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UdRyKR44dcNI"
      },
      "source": [
        "##Control Structures\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AX8KEqV7d2jR"
      },
      "source": [
        "**if&else statement**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hdtFz5UXptLZ"
      },
      "source": [
        "<img src=\"https://www.tutorialspoint.com/python/images/if_else_statement.jpg\" width=300 height=300>\n",
        "\n",
        "*Now it's your turn! Change the following code (...) to generate \"else\":*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E6xzkR1ukfO0"
      },
      "source": [
        "if 2 + ... == 4 ...\n",
        "  if 3 * ... == 9 ...\n",
        "    print (\"if\")\n",
        "  else: \n",
        "    print (\"else\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "datsEgDB4iXQ"
      },
      "source": [
        "**<strong><font color='blue'> Side Note:**  Indentations are very important in Python, as they structure your code. </font></strong>\n",
        "\n",
        "<img src=\"https://i.redd.it/wk843smkri441.jpg\" width=500 height=300>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "40mdnbNieJTh"
      },
      "source": [
        "**Boolean Logic**\n",
        "\n",
        "Boolean Logic is used to make more complicated conditions for if statements that rely on more than one condition. The Boolean operators are **and, or and not**. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OdysNNSonWA0"
      },
      "source": [
        "*Tap around to get a false statement:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7htGUSgym9KX"
      },
      "source": [
        "print ( 2 < 3 and 2 ... 6 )"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dHpbmcEBn_34"
      },
      "source": [
        "*Change and correct the following code to print \"Hi\":*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XlAjNuIqoKgs"
      },
      "source": [
        "age = ... \n",
        "money = 400 \n",
        "if age > 20 ... money > 50 ...\n",
        "  ...(\"Hi\"..."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0JOau1ITeOBp"
      },
      "source": [
        "**Lists**\n",
        "\n",
        "Lists are used to store multiple items. By using square brackets with commas separating items, you can create a list. \n",
        "\n",
        "*Try to print the first argument of the list (\"Hello\").*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zjovDwhNlYFn"
      },
      "source": [
        "words =  [\"Hello\", \"world\", \"!\"]\n",
        "print (words...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F8VgN4fgrdDZ"
      },
      "source": [
        "*Change the code to create a list and print its 3rd element:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d5rWZJjOrgwC"
      },
      "source": [
        "the_list = ... 33, 66, 99 ...\n",
        "...(the_list[...])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UEmilTRFecFV"
      },
      "source": [
        "**Loops**\n",
        "\n",
        "A for loop is used to repeat a block of code multiple times. It is common to use the for loop when the number of iterations is fixed (e.g. iterating over a fixed list of items in a shopping list), but there are other loops as well.\n",
        "\n",
        "*Change the code below to print all elements of your list:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hrSVVOz2Uzm9"
      },
      "source": [
        "your_list = ['a','b','c','d']\n",
        "\n",
        "for ... ... ... :\n",
        "  print (...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QOEUQc1SVzqh"
      },
      "source": [
        "*Please change the ... to correct the code so that the loop is broken after the loop reaches the 'c' in your list.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kYHdF1AVlZYy"
      },
      "source": [
        "for item ... your_list: \n",
        "  print(item)\n",
        "  if item ... ...: \n",
        "    print (\"Breaking\")\n",
        "    ... \n",
        "\n",
        "print(\"Finished\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hDqpOb_IXfK0"
      },
      "source": [
        "What's wrong with the following code? \n",
        "*Please find the correct code to count all the 't' letters in the string. ATTENTION, you have to correct multiple errors and may need to change pre-written code.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "if61RplmXxGC"
      },
      "source": [
        "str = \"testing for loops to count 't's\"\n",
        "count = 0 \n",
        "\n",
        "for ... in str: \n",
        "  if ... == 't' :\n",
        "    count ... = 1\n",
        "\n",
        "print (count) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XBlj6_t8efo8"
      },
      "source": [
        "**Range**\n",
        "\n",
        "The range() function returns a sequence of numbers. By default, it starts from 0, increments by 1 and stops before the specified number. \n",
        "\n",
        "*The code below should generate a list containing all of the integers, up to 10 (not incl.). Please change the following code (...):*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jeR4mJGUlad7"
      },
      "source": [
        "numbers = list(...(10))\n",
        "\n",
        "print(...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cLKi_bFvDN8K"
      },
      "source": [
        "*Now change the code below to print every even number from 10 to 19 using range:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TjGtMu2hDOz6"
      },
      "source": [
        "numbers = list(...)\n",
        "\n",
        "print(...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OwuxHmxllTwN"
      },
      "source": [
        "##Functions & Modules"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EMtDBH2Xe6kR"
      },
      "source": [
        "**Functions**\n",
        "\n",
        "In addition to using pre-defined functions like range() or print(), you can create your own functions by using the def statement. You have to define functions before they are called! This is especially useful, if you want to do similar operations again and again.\n",
        "\n",
        "*Try for yourself and build a function \"quattro_hello\" which prints \"hello\" four times.*\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pHnpri3gHXV8"
      },
      "source": [
        "def ...:\n",
        "  # build a proper loop to print your output four times \n",
        "  print (...)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aDR8mvcz-q1e"
      },
      "source": [
        "*Now call your function and see if it prints 'hello' three times:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z8N9zB0V8jq7"
      },
      "source": [
        "quattro_hello()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mzUjE-zrHiu6"
      },
      "source": [
        "It is also possible to define a function that takes an argument. \n",
        "\n",
        "*Please change the following code to build a function that prints a given text and adds a question mark:*\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3qJbedzsHyTE"
      },
      "source": [
        "def hello_function ():\n",
        "  print (... + \"?\")\n",
        "\n",
        "hello_function(\"HI\")\n",
        "hello_function(\"...\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K5e3PBnT9G0o"
      },
      "source": [
        "**<strong><font color='blue'> Side Note:**  Python can take any form of object as input to a function AND (almost) everything in Python is an object !! </font></strong>\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "<img src=\"https://miro.medium.com/max/625/1*mPpNrH_b3DEd0HiljDqMkw.png\" >\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V5kSE42tJNkw"
      },
      "source": [
        "*And 2 arguments (develop a code to get a result of 16):*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tNiEftMNJzDD"
      },
      "source": [
        "def print_sum(...,...):\n",
        "  print(... + ...)\n",
        "\n",
        "print_sum(6,11)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oMgTXZFfK94l"
      },
      "source": [
        "*Fill in the ... to define a function that takes two arguments and prints their multiplication:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8xBfautGKOid"
      },
      "source": [
        "... print_mult(x,y)...\n",
        "  print(x * ... )\n",
        "...(2,2)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-caDGUcZLO9j"
      },
      "source": [
        "*Fill in the ... to define a function that prints whether the first given paramter is greater or equal than the second paramter - or not:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OixS1ZvgAPJk"
      },
      "source": [
        "def compare(x, y):\n",
        "  if ...:\n",
        "    print(...)\n",
        "  else:\n",
        "    print(...)\n",
        "\n",
        "print(compare(...,...))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lks5vCIQgFBR"
      },
      "source": [
        "**Modules & Libraries**\n",
        "\n",
        "Modules refer to a file containing Python functions. A file containing Python code, for example: example.py, is called a module, and its module name would be example. The basic way to use a mdule is to add import *some_module_name* at the top of your code, and then using *some_module_name*.var to access functions and values with the name var in the module. This allows you to access complex functions programmed by other people.\n",
        " \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SV5FevBtl459"
      },
      "source": [
        "*Please import a fitting module and generate 5 random numbers from the interval [0,10):*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R1892agsAnxy"
      },
      "source": [
        "import ...\n",
        "\n",
        "for i in range(5): \n",
        "  value = ...\n",
        "  print(value)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pvTTfdOUm96f"
      },
      "source": [
        "*Please import only the pi constant from the respective module and print pi:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Mfxt3G-Ym-wd"
      },
      "source": [
        "from ... import ...\n",
        "\n",
        "print(...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wFCVUGx_Bqef"
      },
      "source": [
        "<img src =\"https://i.redd.it/37xqk9umycn31.jpg\" width=500 height=400>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SpnHNXNvUVUY"
      },
      "source": [
        "**Pip Install**\n",
        "\n",
        "pip is a package management program for Python packages from the Python Package Index (PyPI). It allows you to install and manage software packages written in Python. Basically, it lets you install different modules.\n",
        "\n",
        "Before you learn how to install and import packages, the first step would be to actually make sure you've the correct Python verison installed in your system. \n",
        "\n",
        "*Fill in the ... to check this and the version installed:* "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UhimRI0A1QJ1"
      },
      "source": [
        "!python ... v"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LZsAOsGNt6t2"
      },
      "source": [
        "*Please correct the following code to install the scenedetect package:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sezVDvVCA9HX"
      },
      "source": [
        "import scenedetect # are you able to import the package?"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ovmkymv7unbc"
      },
      "source": [
        "... scenedetect"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dzrXbT3YBGNV"
      },
      "source": [
        "import scenedetect # you should be able to import it now!"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qs52gjw6uzY8"
      },
      "source": [
        "Uninstalling/removing a package is very easy with pip. \n",
        "\n",
        "*Please uninstall scenedetect again:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1G_MJA3_vNuN"
      },
      "source": [
        "pip ..."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-Rh3-Vt9Nev9"
      },
      "source": [
        "##Exceptions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1RR8qzFXgkJh"
      },
      "source": [
        "**Exception and Exception Handling**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qQamrW33vP7o"
      },
      "source": [
        "Exception is an event that occurs due to incorrect code or input. \n",
        "\n",
        "*Please generate a zero devision error:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dr5Bfn-ev_FC"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7EcpGATAv_-m"
      },
      "source": [
        "*Please create a NameError:* "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VDMdlsJcxItH"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CjSKE_T4xJO-"
      },
      "source": [
        "*Please generate a TypeError:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WNVtpN4mxSPV"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4pz8WKMMxWZf"
      },
      "source": [
        "*Please create a ValueError:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4PdQE07HCnBS"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RjaaEpTMyB-y"
      },
      "source": [
        "To handle exceptions, and to execute further code despite occuring errors, you can use a try/except statement. \n",
        "\n",
        "*Please change the following code to print \"an error occured - but the interpreter continued\":*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N7fSjbYFye3c"
      },
      "source": [
        "try:\n",
        "  num1 = 7\n",
        "  num2 = 0\n",
        "  print(...)\n",
        "  print(\"calculation done\")\n",
        "except:\n",
        "  print (\"an error occured - but the interpreter continued\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SSSAZzFFD3dW"
      },
      "source": [
        "<img src =\"https://img.devrant.com/devrant/rant/r_1200037_6jt6c.jpg\" width=500 height=300>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KJxnVSYyVPOt"
      },
      "source": [
        "##Data Science Modules"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8pFuDtAcVWWE"
      },
      "source": [
        "**Numpy**\n",
        "\n",
        "*Please fill in the ... to learn more about Numpy:*\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q4KPTtl2OtfM"
      },
      "source": [
        "NumPy is a program library for the programming language Python. NumPy extends Python with **...** for scientific computing and **...** calculations. The library enables efficient computing with  vectors, **...** and multidimensional **...**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A1yba1TJbiNI"
      },
      "source": [
        "*Please import Numpy as np:* "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BENNc8B4T2rb"
      },
      "source": [
        "import ... as ... "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rRYIaE6Tb0WF"
      },
      "source": [
        "*Please define a list 'cvalues' with different temperatures:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2eHWb-Dyb9b5"
      },
      "source": [
        " ... = [20.1, 20.8, 21.9, 22.5, 22.7, \n",
        "           21.8, 21.3, 20.9, 20.1]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GB0td9p2w5Au"
      },
      "source": [
        "*From our list 'cvalues' we now create a one-dimensional NumPy array, so please change the following code:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "16eyb_-TcC1J"
      },
      "source": [
        "C = np. ... (cvalues)\n",
        "print (C, type(C))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ti-xqhLVxUk9"
      },
      "source": [
        "*Now let us assume that we need the values in degrees Fahrenheit instead of degrees Celsius.\n",
        "This can be done very easily with a NumPy array. The solution of our problem are simple scalar operations. Let's do this:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XR28FF30cWOL"
      },
      "source": [
        "print(...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qFIvnXHOyEZC"
      },
      "source": [
        "*Please find out what the shape of 'C' is:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5qqmqIrvdYWP"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wL-SPKbVE9fD"
      },
      "source": [
        "*Try to create a numpy array with the shape 4x2 (4 rows, 2 columns), containing only zeros*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ldf8EhxHE8v6"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w_wm9FcxVd8T"
      },
      "source": [
        "**Pandas**\n",
        "\n",
        "pandas is a program library for the Python programming language that provides tools for data management and analysis. In particular, it contains data structures and operators for accessing tables and time series.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cMs0xplyHutE"
      },
      "source": [
        "*Please import pandas as pd and check the version:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EvlvkfJvH8NK"
      },
      "source": [
        "import pandas as pd\n",
        "print(pd.__ ... __)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mRYZ8zxhIvMV"
      },
      "source": [
        "*Create a pandas series from a list with strings, a numpy array with integers and a dictionary with the combination of your list and your numpy array:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pzGh2XV4oIAZ"
      },
      "source": [
        "# Inputs\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "mylist = ...\n",
        "myarr = ...\n",
        "mydict = ...(...(mylist, myarr))\n",
        "\n",
        "# Solution\n",
        "ser1 = pd.Series(mylist)\n",
        "ser2 = pd.Series(myarr)\n",
        "ser3 = pd.Series(mydict)\n",
        "print(ser1.head())\n",
        "print(ser2.head())\n",
        "print(ser3.head())"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LJ3KXj28sUCS"
      },
      "source": [
        "**<strong><font color='blue'> Side Note:** Note that a Pandas Series consists of an index (left column) and values (right column). This is important for accessing values.</font></strong>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4WXvLc8CJJ6O"
      },
      "source": [
        "*Now create a 4x4 DataFrame with random integer values:*\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N8PIWzFpJoLI"
      },
      "source": [
        "# create a numpy 4x4 numpy matrix with random integer values\n",
        "myarr = (np. ... (4,4)*10). ... .astype(int)\n",
        "\n",
        "df = ...(myarr)\n",
        "print(df)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ppVXphb1Le4l"
      },
      "source": [
        "Let's learn something about indexing and slicing. \n",
        "\n",
        " **.loc()** has multiple access methods like:\n",
        "\n",
        "- A single scalar label\n",
        "- A list of labels\n",
        "- A slice object\n",
        "- A Boolean array\n",
        "\n",
        "**loc** takes two single/ list/ range operator separated by ','. The first one indicates the row and the second one indicates columns.\n",
        "\n",
        "*Please complete all the tasks:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JW58HwihvfSt"
      },
      "source": [
        "#import the pandas library as pd\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "# you may reuse myarr from above, otherwise please recreate a 4x4 numy array\n",
        "df = ...(myarr,index = ['a','b','c','d'], columns = ['A', 'B', 'C', 'D'])\n",
        "\n",
        "#select the value in row b, columnd C\n",
        "print(...)\n",
        "\n",
        "#select all values in row b\n",
        "print(...)\n",
        "\n",
        "#select all values in from row b to d, and column A to C\n",
        "print(...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WiHop8xU_bLo"
      },
      "source": [
        "*You can use boolean logic to slice dataframes as well. This is extremely handy in many situations. Please complete the tasks given in the comments. **Do not manually select the correct column (by just stating column X and row y)**:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wgmuagXd_bVw"
      },
      "source": [
        "# select all rows with a positive value in the 'A' column\n",
        "print (...)\n",
        "\n",
        "# select all columns with a positive value in the 'b' row\n",
        "print (...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VrxtswA2Mupp"
      },
      "source": [
        "Now we are working with **.iloc()**. Pandas provide various methods in order to get purely integer based indexing. Like python and numpy, these are 0-based indexing.\n",
        "\n",
        "The various access methods are as follows:\n",
        "- An Integer\n",
        "- A list of integers\n",
        "- A range of values\n",
        "\n",
        "*Tap around and change the ... to complete the tasks in the comments:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GeSB-zAiMwGt"
      },
      "source": [
        "# you may reuse df, otherwise, please create a new 4 by 4 dataframe\n",
        "\n",
        "# print the first 3 columns using 'i' and the loop\n",
        "for i in range(3):\n",
        "  print(...)\n",
        "\n",
        "# print every second row using range\n",
        "...[range( ... ) ... ]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l5yZGuq6RDy4"
      },
      "source": [
        "You can read existing tables with pandas, to work with them in Python. The most common table format is csv. \n",
        "\n",
        "*Please download the following file (https://drive.google.com/file/d/1rabckY6MNXi3kGEDJ2KIXGB1rsmdwbEr/view?usp=sharing), upload it to your Colab environment and change the path to read it:*\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5Ow2FxtgQers"
      },
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv ('...')\n",
        "print(df) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7B-Izk3RMSk3"
      },
      "source": [
        "*Let's change the Data a bit:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1nSbRxA1UetU"
      },
      "source": [
        "# Delete the 'fuel consumption' column\n",
        "df = \n",
        "\n",
        "# Delete all rows, where the price is not available \n",
        "df = \n",
        "\n",
        "# Let's only keep cars, with more than 100 PS\n",
        "df = \n",
        "\n",
        "df # Let's see if the table looks good"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5cPeqmMgNs1r"
      },
      "source": [
        "*Now, please save the dataframe as a new csv file*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fS1ycn55NtFz"
      },
      "source": [
        "name = 'cars_new.csv'\n",
        "\n",
        "... "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mjutWdU7VgJF"
      },
      "source": [
        "**Matplotlib**\n",
        "\n",
        "Matplotlib is a program library for the programming language Python, which allows to plot your data.\n",
        "\n",
        "<img src =\"https://miro.medium.com/max/1000/1*WACiczYwdWTnJ94mnizS4Q.jpeg\" width=500 height=300>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yR-VrJyXpQnx"
      },
      "source": [
        "\n",
        "*Please generate a positive linear function and plot it using matplotlib:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jvNbwu8_yEEe"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt \n",
        "a = np. ... # please create 10 evenly spaced numbes between 0 and 1 \n",
        "b = np. ... # please create 10 evenly spaced numbes between 0 and 10 \n",
        "plt.plot(a,b)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DrJomQ5EREim"
      },
      "source": [
        "*Now, try to plot a scatter plot, where you see each individual point*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DsMTSObFQDNU"
      },
      "source": [
        "...(a,b)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pTAh3IpX6Fng"
      },
      "source": [
        "*Now, we want to build a histogram:* "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yIlOVSBBR1td"
      },
      "source": [
        "# Let's start by creating a list of 100 random numbers between 0 and 90\n",
        "import random\n",
        "random_numbers=[]\n",
        "for i in range(100):\n",
        "  random_numbers. ...\n",
        "\n",
        "# Now let's plot a histogram with 10 bins\n",
        "import ... as plt\n",
        "plt. ... ( ... )\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5F0JvOsjVogd"
      },
      "source": [
        "**Sklearn**\n",
        "\n",
        "scikit-learn is a open source library for data analysis in Python based on other Python libraries: NumPy, SciPy and Matplotlib. It contains a number of implementations for various common machine learning algorithms.\n",
        "\n",
        "*Let's start by creating some data:* "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xTiKJ44odfuQ"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "\n",
        "# create 40 random float numbers\n",
        "X = 10* ...  \n",
        "\n",
        "y = 3 * X + np.random.randn(40)*5\n",
        "\n",
        "# Take a look at your data with a scatter plot\n",
        "...\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rFRG_YiVh6OM"
      },
      "source": [
        "In many cases, it can be useful or necessary to standardize your data, so that both your X and y have a mean of 0 and a standard deviation of 1.\n",
        "\n",
        "*Try to standardize X and y using sklearn methods. You should receive 2 True as a result:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GXwazIichb9i"
      },
      "source": [
        "from sklearn import ...\n",
        "import numpy as np\n",
        "X_scaled = ...\n",
        "y_scaled = ...\n",
        "\n",
        "print(X_scaled.mean().round(10) == 0 and y_scaled.mean().round(10) == 0)\n",
        "print(X_scaled.std().round(10) == 1 and y_scaled.std().round(10) == 1)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G2s_v9ZdjwVJ"
      },
      "source": [
        "**<strong><font color='blue'> Side Note:** We may have to add a 'round(10)' here as Python works numerically - instead of getting an exact 0, you may sometimes get something like 2.2204460492503132e-17, which is practically 0, once we round to the tenth decimal point.</font></strong>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yW99dkVKkk3M"
      },
      "source": [
        "*Let's make sure standardization did not change any proportions, by again producing a scatter plot with the scaled data:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dnbjxif3g1VK"
      },
      "source": [
        "...(X_scaled, y_scaled)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B4z-ea5EIfoC"
      },
      "source": [
        "<img src =\"https://preview.redd.it/phfw135f17u41.jpg?width=960&crop=smart&auto=webp&s=0074296cef9eeeb7a40a03f19e618b847c756426\" width=500 height=300>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ky94Fe9pVu8a"
      },
      "source": [
        "##Regression with Sklearn"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_KADPUVuDC1w"
      },
      "source": [
        "There are **five basic steps** when you’re implementing linear regression:\n",
        "\n",
        "1. Import the packages and classes you need.\n",
        "2. Provide data to work with and eventually do appropriate transformations.\n",
        "3. Create a regression model and fit it with existing data.\n",
        "4. Check the results of model fitting to know whether the model is satisfactory.\n",
        "5. Apply the model for predictions."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3vxgqggvDgEG"
      },
      "source": [
        "**Step 1: Import packages and classes**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-4w2k0VNDzkd"
      },
      "source": [
        "*Please import the package numpy and the class LinearRegression:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QOXsjNrzDqTL"
      },
      "source": [
        "import numpy as np\n",
        "from ... import LinearRegression"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "25zDeGGiEBUw"
      },
      "source": [
        "The fundamental data type of NumPy is the array type called numpy.ndarray. From now on, we're using the term array to refer to instances of the type numpy.ndarray."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0qNh58vEEMWL"
      },
      "source": [
        "**Step 2: Provide data:**\n",
        "\n",
        "The second step is defining data to work with. The inputs (regressors, 𝑥) and output (predictor, 𝑦) should be arrays (the instances of the class numpy.ndarray) or similar objects. Please reuse X_scaled and y_scaled. If this did not work, use the alternative data provided. \n",
        "\n",
        "*If necessary, change the following code to provide data for the regression:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XfhGOPWEuI72"
      },
      "source": [
        "# IF YOU DID NOT MANAGE TO CREATE X_scaled and y_scaled, uncomment and use the following lines\n",
        "# X_scaled = np.array([-0.51,  0.67, -0.47,  0.6 ,  1.71, -0.16,  0.34, -1.02, -0.68,\n",
        "#         2.43, -1.2 ,  0.51, -0.91,  0.66, -0.53, -2.31, -0.65,  1.34,\n",
        "#         0.11,  1.1 ,  1.52, -0.96, -1.29, -0.71, -0.3 ,  1.46, -1.08,\n",
        "#         0.24, -0.  ,  0.28,  0.9 ,  0.75, -0.62, -0.71,  1.56, -0.33,\n",
        "#         0.14, -1.43, -0.66,  0.2 ])\n",
        "# y_scaled = np.array([-0.46,  0.63, -0.3 ,  0.66,  1.62, -0.55,  0.27, -1.03, -0.68,\n",
        "#         2.25, -1.25,  0.47, -0.51,  0.63, -0.62, -2.42, -0.76,  1.44,\n",
        "#        -0.12,  1.38,  1.6 , -1.22, -1.21, -0.63, -0.37,  1.24, -1.18,\n",
        "#         0.37,  0.17,  0.4 ,  0.88,  0.77, -0.49, -0.44,  1.63, -0.37,\n",
        "#         0.02, -1.36, -0.51,  0.07])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sgdxd-cp4VBx"
      },
      "source": [
        "*Please print the variables X_scaled and y_scaled to check if they are correct:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CpgJWagDE-x9"
      },
      "source": [
        "...\n",
        "..."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L6PcWKhwFML5"
      },
      "source": [
        "**Step 3: Create a model and fit it**\n",
        "\n",
        "The next step is to create a linear regression model and fit it using the existing data. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hk6SGwfuFihF"
      },
      "source": [
        "*Please create and fit the model (fitting means, sklearn will estimate your parameters for a given Input using the specified model):*\n",
        "**Attention: Many times, sklearn needs input of a specific shape. If this is the case (as it will be here), you may be required to reshape data, even if is basically the same, e.g. reshaping a (n,) array to a (n,1) array.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X5HYk7zYICma"
      },
      "source": [
        "model = ..."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IYmtW_4dFwfi"
      },
      "source": [
        "**Step 4: Get results**\n",
        "\n",
        "Once you have your model fitted, you can get the results to check whether the model works satisfactorily and interpret it.\n",
        "\n",
        "*Please obtain the coefficient of determination (𝑅²):*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "owg_cgsgF7Uf"
      },
      "source": [
        "r_sq = model. ...\n",
        "print('coefficient of determination:', r_sq)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q7IOBX0JGG9U"
      },
      "source": [
        "*Now, please also take a look at the estimated parameters for the intercept and the slope:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2t08j85rGJiP"
      },
      "source": [
        "print('intercept:', ...)\n",
        "\n",
        "print('slope:', ...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hKQtcJTsGRSO"
      },
      "source": [
        "The code above illustrates how to get 𝑏₀ and 𝑏₁. You may notice that the intercept is a scalar, while the slope is an array, as you can have multiple independent variables (X)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BQbgOa03Gcjq"
      },
      "source": [
        "**Step 5: Predict response**\n",
        "\n",
        "Once there is a satisfactory model, you can use it for predictions with either existing or new data.\n",
        "\n",
        "*To obtain the predicted response for X_scaled, change the following code:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Yn_1GeBTI1xm"
      },
      "source": [
        "y_pred = ...\n",
        "print('predicted response:', y_pred , sep='\\n')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GmP1TirNI7UT"
      },
      "source": [
        "*Let's plot both original data and our predicted values as a line:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X_zYLgzOJCCe"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "plt.scatter(...)\n",
        "plt.plot(...)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nj0pBvAEGyOU"
      },
      "source": [
        "In practice, regression models are often applied for forecasts. \n",
        "\n",
        "*Try to use the model to forecast a y value for a single random X value:*\n",
        "**Attention: Make sure, you provide your data in the correct format expected by your model - this can not be an integer ;)**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g5epV7S-MZsF"
      },
      "source": [
        "X = ...\n",
        "y_new = ...\n",
        "print(y_new)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DmI6Szxaid4z"
      },
      "source": [
        "## Machine Learning Pipeline\n",
        "\n",
        "We will now execute a Machine Learning pipeline including data preparation, model training and performance evaluation using a Random Forest and a Linear Regression as a baseline model. \n",
        "\n",
        "We will use a dataset, where customers rated the quality of wine (For details, see: https://archive.ics.uci.edu/ml/datasets/Wine).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VWmk2uMXYoRB"
      },
      "source": [
        "**Step 1: Data Perparation**\n",
        "\n",
        "\n",
        "*Please load the two datasets properly, and store them in two dataframes:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "COtKM6dDRd5u"
      },
      "source": [
        "import pandas as pd\n",
        "url_white = 'https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv'\n",
        "url_red = 'https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv'\n",
        "\n",
        "df_white = ...\n",
        "df_red = ...\n",
        "\n",
        "print(df_white.shape,df_red.shape) # does this look resonable? Make sure you have the correct number of columns. If necessary, take a look at your data!"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "atoUpOnrMum_"
      },
      "source": [
        "*Now add a column \"color\" with the wine colors white = 0 or red = 1 respectively (to preserve this information) and combine both dataframes into one:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MK9qxcwYMtuP"
      },
      "source": [
        "... # add color column to df_white\n",
        "... # add color column to df_red\n",
        "df_wines = ... # combine the two dataframes in a single long table\n",
        "\n",
        "print(df_wines.shape) # this should be (6497, 13)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bsIP2kuhMA68"
      },
      "source": [
        "Remember, we want to understand **what drives customer's quality perception for wines**. To accomplish that, we want to predict the perceived quality of a wine (our dependent variable, y) using all other available variables (indepentend variables, X).\n",
        "\n",
        "*Let's take a brief look at your dependend variable - how many wines per quality rating are there?*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nv32nSUxVLTR"
      },
      "source": [
        "... # create a table with the number of occurences per quality value (e.g., there should be 193 wines rated 8)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZGgViSRJjy1s"
      },
      "source": [
        "*Now split the data in dependent (y) - quality - and independent(X) variable:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vOIyY8IkMg3R"
      },
      "source": [
        "X,y = ...\n",
        "\n",
        "print(y.head(3))\n",
        "X.head(3) # Does this look good?"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NCcOLK-ZMkUI"
      },
      "source": [
        "**Step 2: Training**\n",
        "\n",
        "*Let's split the dataset so that 80% of the data is used to train the model and 20% is kept for future evaluation:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i1LDvrZ5MorA"
      },
      "source": [
        "from sklearn.model_selection import ...\n",
        "X_train, X_test, y_train, y_test = ...\n",
        "\n",
        "print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9FVKFEvSMtzc"
      },
      "source": [
        "Now we can define and fit our model on the training dataset. Let's do this for both a random forest and a linear regression model. Using two model will allow us to better evaluate our model performance in the end.\n",
        "\n",
        "*Please fit a linear regression and a random forest model:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q31IBY8OMw7k"
      },
      "source": [
        "from sklearn.ensemble import ... # take suitable random forest module\n",
        "from sklearn.linear_model import LinearRegression\n",
        "\n",
        "model_rfo = ... # initialize random rorest model\n",
        "model_lin = ... # initialize linear regression model\n",
        "\n",
        "model_rfo.fit(...)\n",
        "model_lin.fit(...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pTwMheqMM0Kf"
      },
      "source": [
        "**Step 3: Performance Evaluation**\n",
        "\n",
        "*Now use the fitted model to evaluate performance on your 20% holdout test dataset, using the mean squared error (*MSE*) performance metric:*\n",
        "Note, that neither model had 'seen' this data before. This is crucial."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4E6JDqYOM7AR"
      },
      "source": [
        "from sklearn.metrics import ... # please use MSE\n",
        "\n",
        "y_hat_rfo = ... # make predictions using your random forest model\n",
        "y_hat_lin = ... # make predictions using your linear regression model\n",
        "\n",
        "mse_rfo = ... # calculate random forest prediction MSE\n",
        "mse_lin = ... # calculate linear regression prediction MSE\n",
        "\n",
        "print('MSE Random Forest: %.3f' % mse_rfo)\n",
        "print('MSE Linear Regression: %.3f' % mse_lin)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uHDkZUZMWP-B"
      },
      "source": [
        "<img src =\"https://i.ibb.co/Yyk6PBP/laughing-men-testing.jpg\" width=500 height=300>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oyUb34dPpqdk"
      },
      "source": [
        "*It is always a good idea, to get a visual intuition as well, so let's quickly plot ground truth y_test against our predictions y_hat_rfo and y_hat_lin*: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4FtycYu3cdE-"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "... # generate a scatter plot for your random forest predictions\n",
        "plt.show()\n",
        "\n",
        "... # generate a scatter plot for your linear regression predictions\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7CN3utEwql_5"
      },
      "source": [
        "Taking both the MSE and the visual representation - which model performs better?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WK9El9w1qv1v"
      },
      "source": [
        "# write your answer"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QED3CFGxq1Wi"
      },
      "source": [
        "**Step 4: Generating Insights**\n",
        "\n",
        "Let's use our model to find out what is the most important variable for quality. \n",
        "\n",
        "*Please create a dataframe which has all the X variables and their respective performance and sort the dataframe by importance:*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pacpzzrhd1fe"
      },
      "source": [
        "insights = pd.DataFrame(columns=['variable','importance'])\n",
        "\n",
        "insights['variable'] = ...\n",
        "insights['importance'] = ...\n",
        "\n",
        "insights. ... # please print the sorted table - what is most imporant?"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tClVksnmXsNn"
      },
      "source": [
        "## Save & Submit\n",
        "\n",
        "Please make sure you included your name and matrikel at the top of this notebook, save your work **with your name in the file name**, and send **two** files to jasper dot schwenzow at uni dash hamburg dot de:\n",
        "- Ipynb *(Go to File -> Download .ipynb)*\n",
        "- PDF *(Go to File -> Print -> Save as PDF)* (optional)\n",
        "\n"
      ]
    }
  ]
}