tutorial116_upsampling_vs_conv2D_transpose.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "tutorial116_upsampling vs conv2D transpose.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyO+qr5oWUksJRZL7xYooSla",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/bnsreenu/python_for_image_processing_APEER/blob/master/tutorial116_upsampling_vs_conv2D_transpose.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4sQKW8MlsH1U"
      },
      "source": [
        "\"\"\"\n",
        "@author: Dr. Sreenivas Bhattiprolu\n",
        "Difference between UpSampling2D and Conv2DTranspose\n",
        "These are the two common types of layers that can be used to increase the dimensions of arrays.\n",
        "UpSampling2D is like the opposite of pooling where it repeats rows and columns of the input.\n",
        "Conv2DTranspose performs up-sampling and convolution. \n",
        "Conv2DTranspose has been reported to result in Checkerboard artifacts but \n",
        "unfortunately not much information on the comparison of UpSampling2D vs Conv2DTranspose.\n",
        "\"\"\"\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EZvpElZ8scnj"
      },
      "source": [
        "**Example to demonstrate UpSampling2D**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0zpJyPUqsJmj"
      },
      "source": [
        "from numpy import asarray\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import UpSampling2D, Input"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rWUmcuMSsMdS",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5c2b341a-40a2-4275-97dc-2cea7e2ae0c4"
      },
      "source": [
        "# define simple 3x3 input for this exercise\n",
        "X = asarray([[1, 2, 3],\n",
        "             [4, 5, 6],\n",
        "             [7, 8, 9]])\n",
        "\n",
        "# reshape input data te get it ready for the model (N, x, y, channels)\n",
        "X = X.reshape((1, X.shape[0], X.shape[1], 1))\n",
        "print(X.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(1, 3, 3, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WhYWeynOsQoq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "da3e642e-6743-418d-c61c-ba4694763522"
      },
      "source": [
        "\"\"\"\n",
        "Let us define a model. \n",
        "#Upsampling size: for size=(2,2) output would be 6,6 since our array size is 3x3 \n",
        "#and for size=(3,3) output would be 9x9\n",
        "\"\"\"\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Input(shape=(X.shape[1], X.shape[2], 1)))\n",
        "model.add(UpSampling2D(size = (2,2)))\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Please add `keras.layers.InputLayer` instead of `keras.Input` to Sequential model. `keras.Input` is intended to be used by Functional model.\n",
            "Model: \"sequential_4\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "up_sampling2d_1 (UpSampling2 (None, 6, 6, 1)           0         \n",
            "=================================================================\n",
            "Total params: 0\n",
            "Trainable params: 0\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XYf7VhtesTFz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3aabd635-36d8-483d-a1e4-6aa2f13661fc"
      },
      "source": [
        "# Apply the model to our input data\n",
        "upsampled_X = model.predict(X)\n",
        "\n",
        "# reshape to just get our x and y\n",
        "upsampled_X = upsampled_X.reshape((upsampled_X.shape[1],upsampled_X.shape[2]))\n",
        "\n",
        "print(\"Original input: \\n\", X[0,:,:,0])\n",
        "print(\"___________________\")\n",
        "print(\"Upsampled : \\n\", upsampled_X)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original input: \n",
            " [[1 2 3]\n",
            " [4 5 6]\n",
            " [7 8 9]]\n",
            "___________________\n",
            "Upsampled : \n",
            " [[1. 1. 2. 2. 3. 3.]\n",
            " [1. 1. 2. 2. 3. 3.]\n",
            " [4. 4. 5. 5. 6. 6.]\n",
            " [4. 4. 5. 5. 6. 6.]\n",
            " [7. 7. 8. 8. 9. 9.]\n",
            " [7. 7. 8. 8. 9. 9.]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evlS1ra_skG7"
      },
      "source": [
        "**Example to demonstrate Conv2DTranspose**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fFJtyHc_spAs"
      },
      "source": [
        "# example of using the transpose convolutional layer\n",
        "from numpy import asarray\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Conv2DTranspose, Input\n",
        "\n",
        "# define simple 3x3 input for this exercise\n",
        "X = asarray([[1, 2, 3],\n",
        "             [4, 5, 6],\n",
        "             [7, 8, 9]])\n",
        "\n",
        "# reshape input data te get it ready for the model (N, x, y, channels)\n",
        "X = X.reshape((1, X.shape[0], X.shape[1], 1))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SjvmBcPLr-UO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "26c13029-d4fd-4739-c71a-92f21ad7eae5"
      },
      "source": [
        "\"\"\"\n",
        "The Conv2DTranspose upsamples and also performs a convolution. \n",
        "Since this includes convolution we need to specify both the size and \n",
        "number of filters. We also need to specify stride that gets used for upsampling. \n",
        "With a stride of (2,2) the rows and columns of value 0 are inserted to get this stride. \n",
        "In this example:\n",
        "    num_features = 1 (normally we have 64 or 128 or 256 .. etc. )\n",
        "    kernel size = 1×1\n",
        "    stride = 2x2 (so a 3x3 image will result as 6x6). 1x1 stride will return the input\n",
        "Also try stride 3x3 (result will be 9x9 as out input is 3x3 size)\n",
        "We will initialize kernel by using weights =1. Otherwise it assigns random weights \n",
        "and output will not make sense. \n",
        "\"\"\"\n",
        "\n",
        "model1 = Sequential()\n",
        "model1.add(Input(shape=(X.shape[1], X.shape[2], 1)))\n",
        "model1.add(Conv2DTranspose(1, (1,1), strides=(2,2), kernel_initializer='ones'))\n",
        "model1.summary()\n",
        "\n",
        "#Trainable parameters would be 2 for (1,1) kernel. 1 weight and 1 bias.\n",
        "#Trainable parameters would be 5 for (2,2) kernel. 4 weights and 1 bias.\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Please add `keras.layers.InputLayer` instead of `keras.Input` to Sequential model. `keras.Input` is intended to be used by Functional model.\n",
            "Model: \"sequential_13\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv2d_transpose_11 (Conv2DT (None, 6, 6, 1)           2         \n",
            "=================================================================\n",
            "Total params: 2\n",
            "Trainable params: 2\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tgQkT1BTstxU",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6a6eda57-5e5e-4221-9a36-27b7c8824c6f"
      },
      "source": [
        "# Apply the model to our input data\n",
        "transposed_X = model1.predict(X)\n",
        "transposed_X = transposed_X.reshape((transposed_X.shape[1],transposed_X.shape[2]))\n",
        "print(\"Original input: \\n\", X[0,:,:,0])\n",
        "print(\"___________________\")\n",
        "print(\"Transposed : \\n\", transposed_X)\n",
        "print(\"___________________\")\n",
        "print(\"Upsampled : \\n\", upsampled_X)\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original input: \n",
            " [[1 2 3]\n",
            " [4 5 6]\n",
            " [7 8 9]]\n",
            "___________________\n",
            "Transposed : \n",
            " [[1. 0. 2. 0. 3. 0.]\n",
            " [0. 0. 0. 0. 0. 0.]\n",
            " [4. 0. 5. 0. 6. 0.]\n",
            " [0. 0. 0. 0. 0. 0.]\n",
            " [7. 0. 8. 0. 9. 0.]\n",
            " [0. 0. 0. 0. 0. 0.]]\n",
            "___________________\n",
            "Upsampled : \n",
            " [[1. 1. 2. 2. 3. 3.]\n",
            " [1. 1. 2. 2. 3. 3.]\n",
            " [4. 4. 5. 5. 6. 6.]\n",
            " [4. 4. 5. 5. 6. 6.]\n",
            " [7. 7. 8. 8. 9. 9.]\n",
            " [7. 7. 8. 8. 9. 9.]]\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}