t81_558_class_07_4_gan_semi_supervised.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/t81_558_deep_learning/blob/master/t81_558_class_07_4_gan_semi_supervised.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# T81-558: Applications of Deep Neural Networks\n",
    "**Module 7: Generative Adversarial Networks**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 7 Material\n",
    "\n",
    "* Part 7.1: Introduction to GANS for Image and Data Generation [[Video]](https://www.youtube.com/watch?v=0QnCH6tlZgc&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_1_gan_intro.ipynb)\n",
    "* Part 7.2: Implementing a GAN in Keras [[Video]](https://www.youtube.com/watch?v=T-MCludVNn4&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_2_Keras_gan.ipynb)\n",
    "* Part 7.3: Face Generation with StyleGAN and Python [[Video]](https://www.youtube.com/watch?v=Wwwyr7cOBlU&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_3_style_gan.ipynb)\n",
    "* **Part 7.4: GANS for Semi-Supervised Learning in Keras** [[Video]](https://www.youtube.com/watch?v=ZPewmEu7644&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_4_gan_semi_supervised.ipynb)\n",
    "* Part 7.5: An Overview of GAN Research [[Video]](https://www.youtube.com/watch?v=cvCvZKvlvq4&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_5_gan_research.ipynb)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 7.4: GANS for Semi-Supervised Training in Keras\n",
    "\n",
    "GANs can also be used to implement semi-supervised learning/training.  Normally GANs implement un-supervised training.  This is because there are no y's (expected outcomes) provided in the dataset.  The y-values are usually called labels.  For the face generating GANs, there is typically no y-value, only images.  This is unsupervised training.  Supervised training occurs when we are training a model to predict specified y-values.  These techniques are summarized in Figure 7.TTYPE.\n",
    "\n",
    "**Figure 7.TTYPE: Supervised, UnSupervised and Semi-Supervised**\n",
    "![GAN](https://raw.githubusercontent.com/jeffheaton/t81_558_deep_learning/master/images/gan-training.png \"GAN\")\n",
    "\n",
    "This module will look at how to implement semi-supervised training with a GAN. [[Cite:odena2016semi]](https://arxiv.org/abs/1606.01583)\n",
    "\n",
    "As you can see, supervised learning is where all data have labels.  Supervised learning attempts to learn the labels from the training data to predict these labels for new data.  Un-supervised learning has no labels and usually simply clusters the data or in the case of a GAN, learns to produce new data that resembles the training data.  Semi-supervised training has a small number of labels for mostly unlabeled data.  Semi-supervised learning is usually similar to supervised learning in that the goal is ultimately to predict labels for new data.\n",
    "\n",
    "Traditionally, unlabeled data would simply be discarded if the overall goal was to create a supervised model.  However, the unlabeled data is not without value.  Semi-supervised training attempts to use this unlabeled data to help learn additional insights about what labels we do have.  There are limits, however.  Even semi-supervised training cannot learn entirely new labels that were not in the training set.  This would include new classes for classification or learning to predict values outside of the range of the y-values.\n",
    "\n",
    "Semi-supervised GANs can perform either classification or regression.  Previously, we made use of the generator and discarded the discriminator.  We simply wanted to create new photo-realistic faces, so we just needed the generator.  Semi-supervised learning flips this, as we now discard the generator and make use of the discriminator as our final model.\n",
    "\n",
    "### Semi-Supervised Classification Training\n",
    "\n",
    "Figure 7.GAN-SEMI shows how to apply GANs for semi-supervised classification training.\n",
    "\n",
    "**Figure 7.GAN-SEMI: GAN for Semisupervised Training**\n",
    "![GAN for Semisupervised Training](https://raw.githubusercontent.com/jeffheaton/t81_558_deep_learning/master/images/gan-semi-class.png \"GAN for Semisupervised Training\")\n",
    "\n",
    "Semi-supervised classification training is laid exactly the same as a regular GAN.  The only differences is that it is not a simple true/false classifier as was the case for image GANs that simply classified if the generated image was a real or fake.  The additional classes are also added.  Later in this module I will provide a link to an example of [The Street View House Numbers (SVHN) Dataset](http://ufldl.stanford.edu/housenumbers/).  This dataset contains house numbers, as seen in the following image.\n",
    "\n",
    "**Figure 7.SEMI-SVHN: Semi-Supervised Training for the SVHN Data Set**\n",
    "![GAN](https://raw.githubusercontent.com/jeffheaton/t81_558_deep_learning/master/images/svhn_examples_new.jpg \"GAN\")\n",
    "\n",
    "Perhaps all of the digits are not labeled.  The GAN is setup to classify a real or fake digit, just as we did with the faces.  However, we also expand upon the real digits to include classes 0-9.  The GAN discriminator is classifying between the 0-9 digits and also fake digits.  A semi-supervised GAN classifier always classifies to the number of classes plus one. The additional class indicates a fake classification. \n",
    "\n",
    "### Semi-Supervised Regression Training\n",
    "\n",
    "Figure 7.SS-STRUCT shows how to apply GANs for semi-supervised regression training.\n",
    "\n",
    "**Figure 7.SS-STRUCT: Semisupervised Structure.**\n",
    "![GAN](https://raw.githubusercontent.com/jeffheaton/t81_558_deep_learning/master/images/gan-semi-reg.png \"GAN\")\n",
    "\n",
    "Neural networks can perform both classification and regression simultaneously, it is simply a matter of how the output neurons are mapped.  A hybrid classification-regression neural network simply maps groups of output neurons to be each of the groups of classes to be predicted, along with individual neurons to perform any regression predictions needed.\n",
    "\n",
    "A regression semi-supervised GAN is one such hybrid.  The discriminator has two output neurons.  The first output neuron performs the requested regression prediction.  The second predicts the probability that the input was fake.\n",
    "\n",
    "### Application of Semi-Supervised Regression\n",
    "\n",
    "An example of using Keras for Semi-Supervised classification is provided here.\n",
    "\n",
    "* [Semi-supervised learning with Generative Adversarial Networks (GANs)](https://towardsdatascience.com/semi-supervised-learning-with-gans-9f3cb128c5e)\n",
    "* [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://arxiv.org/abs/1511.06434)\n",
    "* [The Street View House Numbers (SVHN) Dataset](http://ufldl.stanford.edu/housenumbers/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3.6 (tensorflow)",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}