Traffic_Light_Classifier-zh.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 交通信号灯分类器\n",
    "---\n",
    "\n",
    "在这个项目中，你将使用计算机视觉技术的知识为交通信号灯图像建立一个分类器！你会获得一个交通信号灯图像数据集，里面包含三种交通信号灯，即红灯、黄灯或绿灯，其中，三分之一的灯是亮起的。\n",
    "\n",
    "在该 notebook 中，你将对这些图像进行预处理，提取有助于我们区分不同类型图像的特征，并使用这些特征将交通信号灯图像分为三类：红灯、黄灯或绿灯。所有这些任务将会分解成几个部分：\n",
    "\n",
    "1. **加载和可视化数据**。\n",
    "      任何一种分类任务的第一步都是熟悉你的数据；在这里，你需要加载交通信号灯的图像并将其可视化！\n",
    "\n",
    "2. **预处理**。\n",
    "    输入的图像和输出的标签都需要是标准化的。通过这种方式，你可以使用相同的分类器分析所有输入的图像，并且知道在最终对一个*新*图像进行分类时需要输出什么样的图像。\n",
    "\n",
    "3. **特征提取**。\n",
    "    接下来，你需要从每幅图像中提取一些特征，从而帮助区分这些图像，并最终对这些图像进行分类。\n",
    "\n",
    "4. **进行分类，并将错误可视化**。\n",
    "     最后，你需要编写一个利用你提取的特征对*任何一个*交通信号灯图像进行分类的函数。该函数将输入一个图像，然后输出一个标签。你还会获得相关代码，用于确定分类模型的准确度。   \n",
    "\n",
    "5. **评估你的模型**。\n",
    "    要通过此项目，你的分类器的准确度必须 > 90％，并且不能将任何一个红灯归类为绿灯；你很可能需要通过更改现有特征或添加新特征来提高分类器的准确度。我也鼓励你多进行尝试，让准确度尽可能接近100％！\n",
    "\n",
    "以下是数据集中的一些示例图像（从左到右：红灯、绿灯与黄灯）：\n",
    "<img src=\"images/all_lights.png\" width=\"50%\" height=\"50%\">\n",
    "\n",
    "---\n",
    "### *以下是完成该项目所需了解的内容：*\n",
    "\n",
    "我们已经为你提供了一些模板代码，但你需要执行其他代码步骤才能成功完成此项目。任何一个通过此项目所需的代码的标题中都标有**'(IMPLEMENTATION)'**。在完成此项目时，你还需要回答一些关于你的思路的问题，这些问题在标题中都标有**'(QUESTION)'**。请务必回答所有问题，并根据 [项目审阅标准](https://review.udacity.com/#!/rubrics/1213/view) 检查你的代码，确保你完成了必要的几个分类步骤！\n",
    "\n",
    "你提交的项目将根据你提供的代码运行情况以及两个主要分类标准进行评估。\n",
    "提交的完整交通信号灯分类器应满足下面两个标准：\n",
    "1. **准确度超过90％**\n",
    "2. ***不要* 将任何一个红灯分类为绿灯**\n",
    "\n",
    "\n",
    "# 1. 加载交通信号灯数据集，并将其可视化\n",
    "\n",
    "这个交通信号灯数据集由3个类别1484个彩色图像组成，即红灯、黄灯和绿灯。与大多数人工获取的数据一样，该数据在这些类型之间的分布并不均匀，具体为：\n",
    "* 904个红色交通灯图像\n",
    "* 536个绿色交通灯图像\n",
    "* 44个黄色交通灯图像\n",
    "\n",
    "*注：所有图像都来自[麻省理工学院的无人驾驶课程](https://selfdrivingcars.mit.edu/)，并符合 [知识共享署名许可协议-相同方式共享 4.0 国际许可协议](https://creativecommons.org/licenses/by-sa/4.0/) 。*\n",
    "\n",
    "### 导入资源\n",
    "\n",
    "在开始编写项目代码之前，请导入你需要的库和资源。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2 # computer vision library\n",
    "import helpers # helper functions\n",
    "\n",
    "import random\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mpimg # for loading in images\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练并测试数据\n",
    "\n",
    "所有1484个交通信号灯图像被将分成训练数据集和测试数据集。\n",
    "\n",
    "* 这些图像中的80％是训练图像，供你在创建分类器时使用。\n",
    "* 其余20％是测试图像，用于测试分类器的准确度。\n",
    "* 所有图像都是含有3种信号灯的交通灯图像，其中有一盏灯是亮起的。\n",
    "\n",
    "## 定义图像目录\n",
    "\n",
    "首先，我们设置一些变量来跟踪这些图像的存储位置："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "IMAGE_DIR_TRAINING: the directory where our training image data is stored\n",
    "IMAGE_DIR_TEST: the directory where our test image data is stored\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Image data directories\n",
    "IMAGE_DIR_TRAINING = \"traffic_light_images/training/\"\n",
    "IMAGE_DIR_TEST = \"traffic_light_images/test/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 加载数据集\n",
    "\n",
    "这些前几行代码将对训练交通信号灯图像进行加载，并将其全部存储在变量`IMAGE_LIST`中。该列表包含图像及其相关标签（“红灯”，“黄灯”，“绿灯”）。\n",
    "\n",
    "我们建议你查看helpers.py文件中的`load_dataset`函数。这个函数可以为你提供一个很好的思路，即如何使用[glob 库](https://pymotw.com/2/glob/)从一个目录中读取大量图像文件。 `load_dataset`函数可以输入图像目录的名称，然后返回图像及其相关标签的列表。\n",
    "\n",
    "例如，`IMAGE_LIST`中的第一个图像标签对可以通过索引访问："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "attributes": {
     "classes": [
      "IMAGE_LIST[0][:]```"
     ],
     "id": ""
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "\n",
    "```python\n",
    "# Using the load_dataset function in helpers.py\n",
    "# Load training data\n",
    "IMAGE_LIST = helpers.load_dataset(IMAGE_DIR_TRAINING)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 可视化数据\n",
    "\n",
    "分析任何一个数据集的前两个步骤都是：1.加载数据；2.查看数据。看一看这些数据，你就会知道要在图像中寻找什么，需要处理什么样的噪音或不一致性等。这可以帮助你了解图像数据集，并且明白**了解数据集是对数据进行预测的一部分**。\n",
    "\n",
    "---\n",
    "### 将输入图像可视化\n",
    "\n",
    "将这些图像可视化，并对这些图像数据进行探究！编写代码，使其可以在`IMAGE_LIST`中显示图像：\n",
    "* 显示图像\n",
    "* 输出图像的形状\n",
    "* 输出相应的标签\n",
    "\n",
    "看一看是否可以显示出每种交通信号灯图像中的至少一种，即红灯，绿灯或黄灯，并查看它们的相似之处与差异之处。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## TODO: Write code to display an image in IMAGE_LIST (try finding a yellow traffic light!)\n",
    "## TODO: Print out 1. The shape of the image and 2. The image's label\n",
    "\n",
    "# The first image in IMAGE_LIST is displayed below (without information about shape or label)\n",
    "selected_image = IMAGE_LIST[0][0]\n",
    "plt.imshow(selected_image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 预处理数据\n",
    "\n",
    "每个图像中加载完成后，你必须使输入和输出标准化！\n",
    "\n",
    "### 输入\n",
    "\n",
    "这意味着每个输入图像都应采用相同的格式，大小相同等。我们将通过对每个图像执行相同的分析来创建特征，对于这样的分类任务，重要的是**类似的图像创建类似的特征**！\n",
    "\n",
    "### 输出\n",
    "\n",
    "我们还需要将输出结果作为易于阅读且易于与其他标签进行比较的标签。将“红灯”和“绿灯”等分类数据转换为数字数据，这是一种不错的做法。\n",
    "\n",
    "一个非常常见的分类输出是一维列表，它是指分类数的长度：在红灯、黄灯和绿灯的情况下为3； 值为0或1表示某个图像是某一类。例如，由于我们有三个类（红灯、黄灯和绿灯），则可以按顺序列出一个列表：[红灯值，黄灯值，绿灯值]。一般来说，顺序无关紧要，在这种情况下，我们选择的顺序是[红灯值，黄灯值，绿灯值]，便于反映垂直顺序中每个灯的位置。\n",
    "\n",
    "红灯的标签应该是：[1,0,0]，黄灯应该是：[0，1，0]，绿灯应该是：[0，0，1]。这些标签被称为**独热编码标签**。\n",
    "\n",
    "*（注：使用 [机器学习算法 ](https://machinelearningmastery.com/how-to-one-hot-encode-sequence-data-in-python/)时，独热编码将尤为重要）。*\n",
    "\n",
    "<img src=\"images/processing_steps.png\" width=\"80%\" height=\"80%\">\n",
    "\n",
    "---\n",
    "<a id='task2'></a>\n",
    "### （实现）：使输入图像标准化\n",
    "\n",
    "* 将每个图像调整为所需的输入尺寸：32x32px。\n",
    "* （可选）你也可以选择在此步骤中对图像进行裁剪、移动或旋转。\n",
    "\n",
    "一种很常见的尺寸是可旋转的方形输入尺寸（且保持相同尺寸），并且以较小的方块进行分析。将所有图像制作成相同的尺寸，便于它们可以通过相同的分类器步骤进行发送，这一点也很重要！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This function should take in an RGB image and return a new, standardized version\n",
    "def standardize_input(image):\n",
    "    \n",
    "    ## TODO: Resize image and pre-process so that all \"standard\" images are the same size  \n",
    "    standard_im = np.copy(image)\n",
    "    \n",
    "    return standard_im\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 标准化输出\n",
    "\n",
    "对于每个加载的图像，我们还指定了预期的输出。为此，我们使用**独热编码**。\n",
    "\n",
    "* 对标签进行独热编码。为此，请创建一个代表每类交通信号灯（红灯、黄灯、绿灯）的零数组，并将预期类别号的索引设置为1。\n",
    "\n",
    "由于我们有三类（红灯、黄灯和绿灯）交通信号灯图像，我们已经强制使用了一个顺序：[红灯值，黄灯值，绿灯值]。对于独热编码，例如黄灯，我们首先将数组初始化为[0,0,0]，然后将中间值（黄灯值）更改为1：[0,1,0]。\n",
    "\n",
    "---\n",
    "<a id='task3'></a>\n",
    "### （实现）：实现独热编码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## TODO: One hot encode an image label\n",
    "## Given a label - \"red\", \"green\", or \"yellow\" - return a one-hot encoded label\n",
    "\n",
    "# Examples: \n",
    "# one_hot_encode(\"red\") should return: [1, 0, 0]\n",
    "# one_hot_encode(\"yellow\") should return: [0, 1, 0]\n",
    "# one_hot_encode(\"green\") should return: [0, 0, 1]\n",
    "\n",
    "def one_hot_encode(label):\n",
    "    \n",
    "    ## TODO: Create a one-hot encoded label that works for all classes of traffic lights\n",
    "    one_hot_encoded = [] \n",
    "    \n",
    "    return one_hot_encoded\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 测试你的代码\n",
    "\n",
    "在对这样的函数完成编程之后，对它进行测试并查看它是否能够产生预期的输出。**一般来说，编写代码之后，最好在小的功能块中测试这些代码**。这样，不仅可以确保你的代码在继续构建分类器时是正确的，而且你也可以尽早地发现错误，以免它们混合。\n",
    "\n",
    "所有的测试代码都可以在文件`test_functions.py`中找到。如果你觉得有用的话，我们建议你查看该代码并添加自己的测试代码！\n",
    "\n",
    "你会在上述文件夹中看到一个测试函数：`test_one_hot(self, one_hot_function)`，它会输入一个参数，一个one_hot_encode函数，并测试该函数的函数性。如果你的one_hot_label代码无法按预期工作，则此测试将输出一条错误消息，告诉你一些关于代码失败的原因。你的代码开始运行后，应该输出TEST PASSED。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importing the tests\n",
    "import test_functions\n",
    "tests = test_functions.Tests()\n",
    "\n",
    "# Test for one_hot_encode function\n",
    "tests.test_one_hot(one_hot_encode)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 构建一个输入图像并输出标签的`STANDARDIZED_LIST` 函数。\n",
    "\n",
    "该函数会输入一个图像标签对列表，并输出一个已调整过大小的图像和独热编码标签的**标准化**列表。\n",
    "\n",
    "这需要使用你在上面定义的那个函数来使输入和输出标准化，所以这些函数必须完整才能使完成这次标准化工作！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def standardize(image_list):\n",
    "    \n",
    "    # Empty image data array\n",
    "    standard_list = []\n",
    "\n",
    "    # Iterate through all the image-label pairs\n",
    "    for item in image_list:\n",
    "        image = item[0]\n",
    "        label = item[1]\n",
    "\n",
    "        # Standardize the image\n",
    "        standardized_im = standardize_input(image)\n",
    "\n",
    "        # One-hot encode the label\n",
    "        one_hot_label = one_hot_encode(label)    \n",
    "\n",
    "        # Append the image, and it's one hot encoded label to the full, processed list of image data \n",
    "        standard_list.append((standardized_im, one_hot_label))\n",
    "        \n",
    "    return standard_list\n",
    "\n",
    "# Standardize all training images\n",
    "STANDARDIZED_LIST = standardize(IMAGE_LIST)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 将标准化数据可视化\n",
    "\n",
    "显示来自STANDARDIZED_LIST的标准化图像，并将其与来自IMAGE_LIST的非标准化图像进行比较。请注意，它们的尺寸和外观是不同的！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## TODO: Display a standardized image and its label\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. 特征提取\n",
    "\n",
    "现在，你需要根据颜色空间、形状分析和特征构建方面的现有知识来创建特征，帮助区分和分类三种类型的交通信号灯图像。\n",
    "\n",
    "你需要创建至少**一项特征**（可选择创建更多特征）。必需的特征是**使用HSV颜色空间创建的亮度特征**：\n",
    "\n",
    "1. 亮度特征。\n",
    "    - 使用HSV颜色空间，创建一个特征，可用于帮你识别3种不同类别的交通信号灯。\n",
    "    - 稍后，你需要回答一个问题，即你刚刚试图用哪些方法来识别交通信号灯。所以，当你浏览这个notebook 时，需要一直思考你要使用的方法：什么方法有效，什么方法不可行？\n",
    "\n",
    "2. （可选）：创建更多特征！\n",
    "\n",
    "是否要创建更多特征，这取决于你，但是你创建的特征应该能提高交通信号灯分类算法的准确度！有一点需要注意的是，要通过这个项目，**绝不能将任何一个红灯归类为绿灯**，因为这会对无人驾驶汽车造成严重的安全风险。为了避免这种错误分类，你可以考虑添加另一个特别区分红灯和绿灯的特征。\n",
    "\n",
    "这些特征将会被合并在这个 notebook 的末尾处，从而形成一个完整的分类算法。\n",
    "\n",
    "## 创建亮度特征 \n",
    "\n",
    "创建亮度特征的方法有很多。亮度特征可以帮助你表示交通信号灯的图像特征。决定完成亮度特征创建的最佳步骤将由你来定。你应该随时查看并测试你的代码。\n",
    "\n",
    "下图是一个用于创建亮度特征（从左到右：标准化图像、HSV彩色图像、裁剪图像、亮度特征）的示例步骤：\n",
    "\n",
    "<img src=\"images/feature_ext_steps.png\" width=\"70%\" height=\"70%\">\n",
    "\n",
    "\n",
    "## RGB到HSV的转换\n",
    "\n",
    "下面，将测试图像从RGB转换为HSV颜色空间，并将每个组件显示在图像中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert and image to HSV colorspace\n",
    "# Visualize the individual color channels\n",
    "\n",
    "image_num = 0\n",
    "test_im = STANDARDIZED_LIST[image_num][0]\n",
    "test_label = STANDARDIZED_LIST[image_num][1]\n",
    "\n",
    "# Convert to HSV\n",
    "hsv = cv2.cvtColor(test_im, cv2.COLOR_RGB2HSV)\n",
    "\n",
    "# Print image label\n",
    "print('Label [red, yellow, green]: ' + str(test_label))\n",
    "\n",
    "# HSV channels\n",
    "h = hsv[:,:,0]\n",
    "s = hsv[:,:,1]\n",
    "v = hsv[:,:,2]\n",
    "\n",
    "# Plot the original image and the three channels\n",
    "f, (ax1, ax2, ax3, ax4) = plt.subplots(1, 4, figsize=(20,10))\n",
    "ax1.set_title('Standardized image')\n",
    "ax1.imshow(test_im)\n",
    "ax2.set_title('H channel')\n",
    "ax2.imshow(h, cmap='gray')\n",
    "ax3.set_title('S channel')\n",
    "ax3.imshow(s, cmap='gray')\n",
    "ax4.set_title('V channel')\n",
    "ax4.imshow(v, cmap='gray')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<a id='task7'></a>\n",
    "### （实现）：创建一个使用HSV颜色空间的亮度特征\n",
    "\n",
    "编写一个函数，它会输入一张RGB图像并返回一个一维特征矢量和/或单值，这有助于对交通信号灯图像进行分类。唯一的要求是该函数应该应用HSV颜色空间转换，其余的由你来决定。\n",
    "\n",
    "通过此函数，你应该能够判断一张图像的标签，并将其分类为红灯、绿灯或黄灯。如果辅助函数能够简化你的代码，你也可以定义几个辅助函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## TODO: Create a brightness feature that takes in an RGB image and outputs a feature vector and/or value\n",
    "## This feature should use HSV colorspace values\n",
    "def create_feature(rgb_image):\n",
    "    \n",
    "    ## TODO: Convert image to HSV color space\n",
    "    \n",
    "    ## TODO: Create and return a feature value and/or vector\n",
    "    feature = []\n",
    "    \n",
    "    return feature\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## （可选）创建更多特征，用于帮助准确标记交通信号灯图像"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# (Optional) Add more image analysis and create more features\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## （问题1）：你创建的特征如何帮你区分这3类交通信号灯图像？\n",
    "\n",
    "**答案：**\n",
    "在这个单元格中写下你的答案。\n",
    "\n",
    "# 4. 进行分类，并将错误可视化\n",
    "\n",
    "使用你创建的所有特征，编写一个可以输入一张RGB图像的函数，并使用提取的特征，输出一个灯光是红色、绿色或者黄色，并将该结果作为独热编码标签。这种分类函数应该能够对任何一张交通信灯图像进行分类！\n",
    "\n",
    "我们建议你编写可能会用到的任何一种辅助函数或可视化代码，但为了测试准确度，请确保此`estimate_label`函数返回一个独热编码标签。\n",
    "\n",
    "---\n",
    "<a id='task8'></a>\n",
    "### （实现）：构建一个完整的分类器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This function should take in RGB image input\n",
    "# Analyze that image using your feature creation code and output a one-hot encoded label\n",
    "def estimate_label(rgb_image):\n",
    "    \n",
    "    ## TODO: Extract feature(s) from the RGB image and use those features to\n",
    "    ## classify the image and output a one-hot encoded label\n",
    "    predicted_label = []\n",
    "    \n",
    "    return predicted_label   \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试分类器\n",
    "\n",
    "这里，我们使用这个 notebook 开始部分的测试数据集，对你的分类算法进行测试！编写完了一个“好”的分类器后，这个项目就算是完整的。\n",
    "\n",
    "在这种案例中，一个“好”的分类器应符合以下标准（只要符合，就可以随时提交你的项目）：\n",
    "1. 分类准确度达到90％以上。\n",
    "2. 切勿将任何一个红灯归类为绿灯。\n",
    "\n",
    "### 测试数据集\n",
    "\n",
    "在下面，我们首先加载测试数据集，然后使用上面定义的`standardize`函数对其进行标准化，然后对其进行**混洗**；这样可以确保顺序不会对准确度测试造成影响。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using the load_dataset function in helpers.py\n",
    "# Load test data\n",
    "TEST_IMAGE_LIST = helpers.load_dataset(IMAGE_DIR_TEST)\n",
    "\n",
    "# Standardize the test data\n",
    "STANDARDIZED_TEST_LIST = standardize(TEST_IMAGE_LIST)\n",
    "\n",
    "# Shuffle the standardized test data\n",
    "random.shuffle(STANDARDIZED_TEST_LIST)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 确定准确度\n",
    "\n",
    "将你的分类算法（也称为“模型”）的输出与真实标签进行比较并确定准确度。\n",
    "\n",
    "此代码将所有错误分类的图像、其预测标签以及它们的真实标签存储在名为`MISCLASSIFIED`的列表中。此代码用于测试，*不应更改*。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Constructs a list of misclassified images given a list of test images and their labels\n",
    "# This will throw an AssertionError if labels are not standardized (one-hot encoded)\n",
    "\n",
    "def get_misclassified_images(test_images):\n",
    "    # Track misclassified images by placing them into a list\n",
    "    misclassified_images_labels = []\n",
    "\n",
    "    # Iterate through all the test images\n",
    "    # Classify each image and compare to the true label\n",
    "    for image in test_images:\n",
    "\n",
    "        # Get true data\n",
    "        im = image[0]\n",
    "        true_label = image[1]\n",
    "        assert(len(true_label) == 3), \"The true_label is not the expected length (3).\"\n",
    "\n",
    "        # Get predicted label from your classifier\n",
    "        predicted_label = estimate_label(im)\n",
    "        assert(len(predicted_label) == 3), \"The predicted_label is not the expected length (3).\"\n",
    "\n",
    "        # Compare true and predicted labels \n",
    "        if(predicted_label != true_label):\n",
    "            # If these labels are not equal, the image has been misclassified\n",
    "            misclassified_images_labels.append((im, predicted_label, true_label))\n",
    "            \n",
    "    # Return the list of misclassified [image, predicted_label, true_label] values\n",
    "    return misclassified_images_labels\n",
    "\n",
    "\n",
    "# Find all misclassified images in a given test set\n",
    "MISCLASSIFIED = get_misclassified_images(STANDARDIZED_TEST_LIST)\n",
    "\n",
    "# Accuracy calculations\n",
    "total = len(STANDARDIZED_TEST_LIST)\n",
    "num_correct = total - len(MISCLASSIFIED)\n",
    "accuracy = num_correct/total\n",
    "\n",
    "print('Accuracy: ' + str(accuracy))\n",
    "print(\"Number of misclassified images = \" + str(len(MISCLASSIFIED)) +' out of '+ str(total))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<a id='task9'></a>\n",
    "### 将错误分类的图像可视化\n",
    "\n",
    "将分类错误的一些图像（在`MISCLASSIFIED`列表中）可视化，并注意那些难以使它们归类的任何一个特征。这样做可以帮助你识别分类算法中的缺陷。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize misclassified example(s)\n",
    "## TODO: Display an image in the `MISCLASSIFIED` list \n",
    "## TODO: Print out its predicted label - to see what the image *was* incorrectly classified as\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<a id='question2'></a>\n",
    "## （问题2）：在将这些误分类可视化之后，你认为你的分类算法存在哪些缺陷？请至少发现两个缺陷。\n",
    "\n",
    "**答案：** 在这个单元格中写下你的答案。\n",
    "\n",
    "## 测试你是否将某一个红灯归类为绿灯\n",
    "\n",
    "**要通过这个项目，你不能将任何一个红灯归类为绿灯！**将红灯分类为绿灯会将导致汽车出现闯红灯行驶的情况，因此这种“把红灯误看做是绿灯”的错误在现实世界中非常危险。\n",
    "\n",
    "下面的代码可以让你测试是否将测试集中的任何一个红灯错误归类为绿灯。**这个测试假定 `MISCLASSIFIED` 是一个元组列表，其顺序是： [misclassified_image, predicted_label, true_label]** 。\n",
    "\n",
    "注：这不是一个全面的测试，但它是一个很好的指标。如果你通过了这个测试，说明你已经掌握了这部分知识！这个测试会遍历错误分类的例子列表，并检查是否存在任何一个红灯被错误标记为[0,1,0]（绿灯）的情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importing the tests\n",
    "import test_functions\n",
    "tests = test_functions.Tests()\n",
    "\n",
    "if(len(MISCLASSIFIED) > 0):\n",
    "    # Test code for one_hot_encode function\n",
    "    tests.test_red_as_green(MISCLASSIFIED)\n",
    "else:\n",
    "    print(\"MISCLASSIFIED may not have been populated with images.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. 改进你的算法！\n",
    "\n",
    "**完成所有任务并且符合以下两个标准，并回答完所有问题后，请提交你的项目：**\n",
    "1. 分类准确度超过90％；\n",
    "2. 没有将任何一个红灯归类为绿灯。\n",
    "\n",
    "如果不符合这两个要求（这在第一次尝试中很常见！），请重新查看你的算法并对其进行改进，改善光线识别 - 这可能意味着更改亮度特征，执行一些背景减除或添加其他特征！\n",
    "\n",
    "---\n",
    "\n",
    "### 接受进一步挑战（可选性挑战项目）\n",
    "\n",
    "如果你发现这个挑战项目太过容易，我建议你去挑战一些更有难度的项目！ 这里有几个**可选** 项目（意思是说，执行下面这些项目后，你不需要进行提交，也不需要必须通过这些项目）：\n",
    "* （可选）尽可能使分类准确度> 95％。\n",
    "* （可选）图像中的一些灯是带形状的，eg：箭头形状；尝试在区分灯的颜色以外，进一步将灯光的形状建模得到分类结果，圆形或箭头形。\n",
    "* （可选）添加另一个特征，尽可能使准确度接近100％！"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}