Intro_to_Neural_Network.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.6"
    },
    "colab": {
      "name": "Intro to Neural Network.ipynb",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/satyambhatt5/Satyam/blob/main/Intro_to_Neural_Network.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t1tTQE6wbjFH"
      },
      "source": [
        "An Artificial Neural Network (ANN) is a computational model that is inspired by the way biological neural networks in the human brain process information."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nfU0KtplbjFS"
      },
      "source": [
        "### Single Neuron"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xnf7GEW5bjFT"
      },
      "source": [
        "The basic unit of computation in a neural network is the neuron, often called a node or unit. It receives input from some other nodes, or from an external source and computes an output. Each input has an associated weight (w), which is assigned on the basis of its relative importance to other inputs. The node applies a function f (defined below) to the weighted sum of its inputs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-UmkL25ZbjFU"
      },
      "source": [
        "<img src=\"https://ujwlkarn.files.wordpress.com/2016/08/screen-shot-2016-08-09-at-3-42-21-am.png?w=1024&h=547\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cLymlkJXbjFU"
      },
      "source": [
        "The above network takes numerical \n",
        "inputs X1 and X2 and has \n",
        "weights w1 and w2 associated with those inputs. \n",
        "\n",
        "Additionally, there is another input 1 with weight b (called the Bias) associated with it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c6VSjxA8bjFV"
      },
      "source": [
        "The function f is non-linear and is called the Activation Function. \n",
        "The purpose of the activation function is to introduce non-linearity into the output of a neuron. \n",
        "This is important because most real world data is non linear and we want neurons to learn these non linear representations."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fp82qmeDbjFV"
      },
      "source": [
        "Every activation function (or non-linearity) takes a single number and performs a certain fixed mathematical operation on it. There are several activation functions you may encounter in practice:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-J1i0-Q2bjFW"
      },
      "source": [
        "\n",
        "    Sigmoid: takes a real-valued input and squashes it to range between 0 and 1\n",
        "\n",
        "σ(x) = 1 / (1 + exp(−x))\n",
        "\n",
        "    tanh: takes a real-valued input and squashes it to the range [-1, 1]\n",
        "\n",
        "tanh(x) = 2σ(2x) − 1\n",
        "\n",
        "    ReLU: ReLU stands for Rectified Linear Unit. It takes a real-valued input and thresholds it at zero (replaces negative values with zero)\n",
        "\n",
        "f(x) = max(0, x)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0KMPELpJbjFW"
      },
      "source": [
        "<img src =\"https://ujwlkarn.files.wordpress.com/2016/08/screen-shot-2016-08-08-at-11-53-41-am.png?w=1493\">"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UexeBW5dbjFX"
      },
      "source": [
        "# Activation \n",
        "# 1. Bringing data in a certain range (-1, 1) or (0,1)\n",
        "# 2. It is bringing non-linearlity in the data \n",
        "# 3. It is increasing the dimentionality of the data "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FkycB01UbjFY"
      },
      "source": [
        "import math\n",
        "math.e"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PHlW-RdnbjFZ",
        "outputId": "ec8b9869-21be-4d8a-f783-adaa2b037a67"
      },
      "source": [
        "#Implementing Sigmoid \n",
        "# 1/(1+e^-x) \n",
        "\n",
        "\n",
        "def sigmoid(x):\n",
        "    return 1/(1+math.e**(-x))\n",
        "print(f'Sigmoid: {sigmoid(-0.322)}')\n",
        "\n",
        "\n",
        "#tanh - (e^x - e^-x)/(e^x + e^-x)\n",
        "def tanh(x):\n",
        "    num = ((math.e**x) - (math.e**(-x)))\n",
        "    den = ((math.e**x) + (math.e**(-x)))\n",
        "    return num/den\n",
        "print(f\"Tanh: {tanh(0.087)}\")\n",
        "\n",
        "\n",
        "def relu(x):\n",
        "    return max(0,x)\n",
        "print(f\"Relu: {relu(90)}\")"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sigmoid: 0.4201884100030751\n",
            "Tanh: 0.08678116153147497\n",
            "Relu: 90\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "geSwDup8bjFa"
      },
      "source": [
        "import tensorflow"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9kRaBX9vbjFa"
      },
      "source": [
        "from tensorflow.keras.activations import relu"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SZzlO0fobjFa"
      },
      "source": [
        "from tensorflow.keras import backend as K"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oX9PaKoSbjFb"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B41NSNoxbjFb"
      },
      "source": [
        "Importance of Bias: The main function of Bias is to provide every node with a trainable constant value (in addition to the normal inputs that the node receives). "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z50nuCZNbjFb"
      },
      "source": [
        "### Feedforward Neural Network"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hEY9-NwjbjFc"
      },
      "source": [
        "The feedforward neural network was the first and simplest type of artificial neural network devised. It contains multiple neurons (nodes) arranged in layers. Nodes from adjacent layers have connections or edges between them. All these connections have weights associated with them."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pXV99vlzbjFc"
      },
      "source": [
        "<img src =\"https://ujwlkarn.files.wordpress.com/2016/08/screen-shot-2016-08-09-at-4-19-50-am.png?w=768&h=568\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YgAwgPnzbjFc"
      },
      "source": [
        "A feedforward neural network can consist of three types of nodes:\n",
        "\n",
        "    Input Nodes – The Input nodes provide information from the outside world to the network and are together referred to as the “Input Layer”. No computation is performed in any of the Input nodes – they just pass on the information to the hidden nodes.\n",
        "    \n",
        "    Hidden Nodes – The Hidden nodes have no direct connection with the outside world (hence the name “hidden”). They perform computations and transfer information from the input nodes to the output nodes. A collection of hidden nodes forms a “Hidden Layer”. While a feedforward network will only have a single input layer and a single output layer, it can have zero or multiple Hidden Layers.\n",
        "    \n",
        "    Output Nodes – The Output nodes are collectively referred to as the “Output Layer” and are responsible for computations and transferring information from the network to the outside world.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "956UKCEMbjFd"
      },
      "source": [
        "In a feedforward network, the information moves in only one direction – forward – from the input nodes, through the hidden nodes (if any) and to the output nodes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TxTYPaMWbjFd"
      },
      "source": [
        "### Multi Layer Perceptron"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j2JYI0lJbjFd"
      },
      "source": [
        "A Multi Layer Perceptron (MLP) contains one or more hidden layers (apart from one input and one output layer).  While a single layer perceptron can only learn linear functions, a multi layer perceptron can also learn non – linear functions."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3LYGysumbjFe"
      },
      "source": [
        "<img src =\"https://ujwlkarn.files.wordpress.com/2016/08/ds.png?w=1024\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LombaZngbjFe"
      },
      "source": [
        "### Back-Propagation Algorithm"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eJSA9UkkbjFe"
      },
      "source": [
        "often abbreviated as BackProp is one of the several ways in which an artificial neural network (ANN) can be trained. It is a supervised training scheme, which means, it learns from labeled training data (there is a supervisor, to guide its learning).\n",
        "\n",
        "To put in simple terms, BackProp is like “learning from mistakes“. The supervisor corrects the ANN whenever it makes mistakes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NxC0hx2SbjFf"
      },
      "source": [
        "Initially all the edge weights are randomly assigned. For every input in the training dataset, the ANN is activated and its output is observed. This output is compared with the desired output that we already know, and the error is “propagated” back to the previous layer. This error is noted and the weights are “adjusted” accordingly. This process is repeated until the output error is below a predetermined threshold."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OHXl2Zq1bjFf"
      },
      "source": [
        "<img src=\"https://raw.githubusercontent.com/rasbt/python-machine-learning-book/master/faq/visual-backpropagation/nonconvex-cost.png\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3dXT8PmmbjFf"
      },
      "source": [
        "\n",
        "\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/rasbt/python-machine-learning-book/master/faq/visual-backpropagation/backpropagation.png\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pJ9ctD0DbjFf"
      },
      "source": [
        "w1, w2, b\n",
        "E = yp - ya #yp = output predictd #ya = output actual # E= error \n",
        "\n",
        "w1_new = w1 - dE/dw1\n",
        "w2_new = w2 - dE/dw2\n",
        "b_new  = b  - dE/db\n",
        "\n",
        "\n",
        "One cycle with Feed Forward & a Back propogation is called as a epoch "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AsBii42ybjFg"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bbQ_Q8m0bjFg"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cBcLtXdnbjFg"
      },
      "source": [
        "<img src=\"https://ujwlkarn.files.wordpress.com/2016/08/screen-shot-2016-08-09-at-11-53-06-pm.png\">"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FO0yGS2QbjFh"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "C4AGHch2bjFh"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IOA_ohDjbjFi"
      },
      "source": [
        "<img src=\"../Linear to Non  Linear.png\"/>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uqcJcNqYbjFi"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yeQ0EidFbjFi"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bn2tksAibjFi"
      },
      "source": [
        "More Detailed info - http://peterroelants.github.io/posts/neural_network_implementation_part01/"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6RidGMEabjFj"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bv1VdLi3bjFj"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PVzKf9PjbjFj"
      },
      "source": [
        ""
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wDCEKdVKbjFj"
      },
      "source": [
        "1. OpenCV\n",
        "2. skimage\n",
        "3. PIL\n",
        "4. keras"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zATgu72objFk"
      },
      "source": [
        "<img src=\"https://ujwlkarn.files.wordpress.com/2016/08/screen-shot-2016-08-09-at-3-42-21-am.png?w=1024&h=547\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xGFWDIYFbjFk"
      },
      "source": [
        "Error = (ya - yp)  #ya - actual output, yp - output predicted by model ( these are probabilities) \n",
        "Error = (ya - (w1*x1 + w2*x2 +b))\n",
        "      = (ya - w1*x1 - w2*x2 -b)\n",
        "         ya, x1 and x2 are constant things and they cant be changed.\n",
        "         can be modified varibales are - w1, w2, b\n",
        "\n",
        "Error = (CONST - w1*CONST - w2*CONST - b) \n",
        "\n",
        "w_1 = w1 - dE/dw1\n",
        "w_2 = w2 - dE/dw2\n",
        "\n",
        "w_1 = w1 - alpha * dE/dw1   # alpha - Learning Rate      alpha - 0.001\n",
        "w_2 = w2 - alpha * dE/dw2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xMdxrbvbbjFk"
      },
      "source": [
        "Error = (ya - yp)\n",
        "to reduce the Error and make it ZERO "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xngSBZDpbjFl"
      },
      "source": [
        "E = ya - yp \n",
        "E = (w1*x1 + b1 - yp)\n",
        "Wn = Wo -alpha*(dE/dW)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h-0avNr_bjFl"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1U0IagZ3bjFl"
      },
      "source": [
        "### How does NN works on Data ? "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rDlq_mBcbjFl"
      },
      "source": [
        "# What is the data for us ?"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mOX8HHOebjFm"
      },
      "source": [
        "<img src=\"https://miro.medium.com/max/726/1*zY1qFB9aFfZz66YxxoI2aw.gif\"/>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TspyHElfbjFm"
      },
      "source": [
        "from PIL import Image"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oCU0pcMfbjFm"
      },
      "source": [
        "#image = Image.open(\"../HD_image.jpg\")"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n3nXE6VYbjFm"
      },
      "source": [
        "#image.height"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YQdF3AGSbjFn"
      },
      "source": [
        "#image.width"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K-OMXyQxbjFo"
      },
      "source": [
        "import cv2\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Acm_z9OebjFo"
      },
      "source": [
        "#rgb_image = Image.open(\"../RGB-color-model.jpg\")"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nUWhOxQBbjFo"
      },
      "source": [
        "#rgb_image"
      ],
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YHXi5Si3bjFp"
      },
      "source": [
        "#plt.imshow(np.array(rgb_image)[:,:,:], cmap=\"gray\")"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "shpL2XJAbjFp"
      },
      "source": [
        "plt.imshow(np.array(rgb_image)[:,:,0], cmap=\"gray\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dshvaTkZbjFp"
      },
      "source": [
        "plt.imshow(np.array(rgb_image)[:,:,1], cmap=\"gray\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EPBh2c-objFq"
      },
      "source": [
        "plt.imshow(np.array(rgb_image)[:,:,2], cmap=\"gray\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8nMs31OFbjFq"
      },
      "source": [
        "image = cv2.imread(\"../HD_image.jpg\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gTbH6wUObjFq"
      },
      "source": [
        "image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rNGqvwJDbjFr"
      },
      "source": [
        "image.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1N9GNtc7bjFr"
      },
      "source": [
        "#1. Greyscale images     - 1 Channel Images \n",
        "#2. Colored Images       - 3 Channel Images (R,G,B)\n",
        "#3. Masked/Transparent   - 4 Channel Images (R,G,B,aplha) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a9lxQm4hbjFr"
      },
      "source": [
        "plt.imshow(image)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Co2WIG_tbjFs"
      },
      "source": [
        "plt.imshow(image[:,:,0])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HdRFxrBLbjFs"
      },
      "source": [
        "plt.imshow(image[:,:,1])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5l8TCe_RbjFs"
      },
      "source": [
        "plt.imshow(image[:,:,2])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "03n4TM-MbjFt"
      },
      "source": [
        "# Machines see images as Matrics with values ranging from 0 - 255 "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jrAA-PW3bjFt"
      },
      "source": [
        "# ! pip install opencv-python"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xL9BnlOfbjFt"
      },
      "source": [
        "from PIL import Image"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ESpbH62tbjFt"
      },
      "source": [
        "image = Image.open(\"../data/dog_cat_small_data/cat.100.jpg\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j1ZacEAjbjFu"
      },
      "source": [
        "image"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T_wsb6cfbjFu"
      },
      "source": [
        "image.height, image.width"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TgogqHcDbjFz"
      },
      "source": [
        "import numpy as np \n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sbg_6EIgbjF0"
      },
      "source": [
        "image_arr = np.array(image)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dmBpRj7zbjF0"
      },
      "source": [
        "image_arr.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KtF4kMxvbjF0"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aTGVf33ebjF1"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RkJTiwm7bjF1"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ipdiPvuRbjF1"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E-smHR9ZbjF1"
      },
      "source": [
        "image_arr[:, :, 1]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-3_1A6K4bjF2"
      },
      "source": [
        "image_arr.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ErcNlWMnbjF2"
      },
      "source": [
        "plt.imshow(image_arr[:,:,0])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6543IgikbjF2"
      },
      "source": [
        "plt.imshow(image_arr[:,:,1])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NcbJ4O27bjF3"
      },
      "source": [
        "plt.imshow(image_arr[:,:,2])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nstWXicxbjF3"
      },
      "source": [
        "# image arr"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tn-esgldbjF3"
      },
      "source": [
        "image_arr.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GLkY6s6EbjF4"
      },
      "source": [
        "image_arr2 = image_arr.copy()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cvgmRGlGbjF4"
      },
      "source": [
        "image_arr2[:,:,0] = 255\n",
        "image_arr2[:,:,1] = 0\n",
        "image_arr2[:,:,2] = 255\n",
        "plt.imshow(image_arr2)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j0yftbiubjF4"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "egad1IpSbjF5"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hgOG9P0VbjF5"
      },
      "source": [
        "image = cv2.imread(\"../Watermark-photos.png\", cv2.IMREAD_UNCHANGED)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kqbEQSFAbjF5"
      },
      "source": [
        "image.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "26Kg0Z6BbjF5"
      },
      "source": [
        "Image.open(\"../Watermark-photos.png\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_XSwjXosbjF6"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "usbPpx0-bjF6"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6sZ5DqtRbjF6"
      },
      "source": [
        "### How dataset is made in DL  (Classification) ? "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oiTDOjrDbjF6"
      },
      "source": [
        "You will need to collect a lot of cat and dog images \n",
        "You will also need to label them \n",
        "\n",
        "#Some things to keep in mind\n",
        "1. Class balancing - You need to make sure that both cat and dog have same number of images \n",
        "2. Maintaing a consistency -  \n",
        "3. Make data hetro - "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "id": "lgSPGShGbjF7"
      },
      "source": [
        "# ! dir \"../data/dog_cat_small_data/\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6aO0yFo_bjF7"
      },
      "source": [
        "# You will makes sure that all the images have same height and same width and same channel\n",
        "# NOTE - Only for FCNN\n",
        "# for ex - \n",
        "# H - 499 \n",
        "# W- 403 \n",
        "# C - 3"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Aptmuo5NbjF7"
      },
      "source": [
        "# 0 - 255  --- 256 --- 2^8  --- \n",
        "# 1, 3, 4 - no of channels\n",
        "# 1 - greyscales \n",
        "# 3 - RGB \n",
        "# 4 - RGB,alpha ( transparent layer)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tek2NDj2bjF7"
      },
      "source": [
        "# 499 * 403 * 3 - number of cells/ pixels  \n",
        "# each cell/pixels will have values rangning from 0 - 255 "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F-efWVnrbjF8"
      },
      "source": [
        "# we have collected 100 images of cat and 100 images of dogs - We makes sure height and widht are sam "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DNXGrOhabjF8"
      },
      "source": [
        "# 499(H) * 403(W) * 3(C) * 200(# of images) - total number of cells/ pixel in whole dataset"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vPKQnLaQbjF8"
      },
      "source": [
        "### How do we feed the data to NN ?? "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yZaMfPMJbjF8"
      },
      "source": [
        "# The Input layer will have 499 * 403 * 3 neurons \n",
        "# How is the ordering decided ? Which pixel will go first "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qzyFM9pbbjF9"
      },
      "source": [
        "<img src = \"https://sds-platform-private.s3-us-east-2.amazonaws.com/uploads/73_blog_image_1.png\">"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uy27e28kbjF9"
      },
      "source": [
        "print(499*403*3) # Number of neurons in Input Layer to accomates a image of shape (499, 403)\n",
        "# print(1080*1920*3)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bp23QybrbjF9"
      },
      "source": [
        "# this is how the images is flattened and goes in the NN !!!"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2K2IYta8bjF9"
      },
      "source": [
        "<img src=\"https://miro.medium.com/max/1281/1*IWUxuBpqn2VuV-7Ubr01ng.png\"/>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AXWKVF7nbjF-"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZXP6JyBgbjF-"
      },
      "source": [
        "<img src=\"https://mk0analyticsindf35n9.kinstacdn.com/wp-content/uploads/2018/12/nural-network-banner.gif\"/>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jKdJmjtHbjF-"
      },
      "source": [
        "# http://neuralnetworksanddeeplearning.com/chap1.html"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GEijOXOabjF-"
      },
      "source": [
        "# 201 - total number of images\n",
        "# 10 -- batch size of 10 \n",
        "# How of number of batches are there ??? \n",
        "# 21"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jwJA_SQ4bjF_"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Mi3_rXKDbjF_"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V7WBbiDEbjF_"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZdIjVTEhbjF_"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "57oDclExbjF_"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wRp57YAVbjGA"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ujeURueybjGA"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b-8EZT98bjGA"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xZRhSY0gbjGA"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i5bi6dcxbjGB"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qPLRgXpkbjGB"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}