EXC_Linear_Regression_with_a_real_dataset.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "private_outputs": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jmbanda/CSC4850-Machine-Learning/blob/main/EXC_Linear_Regression_with_a_real_dataset.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TL5y5fY9Jy_x"
      },
      "source": [
        "# Linear Regression with a Real Dataset\n",
        "\n",
        "This Colab uses a real dataset to predict the prices of houses in California.   \n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h8wtceyJj2uX"
      },
      "source": [
        "## Learning Objectives:\n",
        "\n",
        "After doing this Colab, you'll know how to do the following:\n",
        "\n",
        "  * Read a .csv file into a [pandas](https://developers.google.com/machine-learning/glossary/#pandas) DataFrame.\n",
        "  * Examine a [dataset](https://developers.google.com/machine-learning/glossary/#data_set). \n",
        "  * Experiment with different [features](https://developers.google.com/machine-learning/glossary/#feature) in building a model.\n",
        "  * Tune the model's [hyperparameters](https://developers.google.com/machine-learning/glossary/#hyperparameter)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JJZEgJQSjyK4"
      },
      "source": [
        "## The Dataset\n",
        "  \n",
        "The [dataset for this exercise](https://developers.google.com/machine-learning/crash-course/california-housing-data-description) is based on 1990 census data from California. The dataset is old but still provides a great opportunity to learn about machine learning programming."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xchnxAsaKKqO"
      },
      "source": [
        "## Import relevant modules\n",
        "\n",
        "The following hidden code cell imports the necessary code to run the code in the rest of this Colaboratory."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9n9_cTveKmse"
      },
      "source": [
        "import pandas as pd\n",
        "import tensorflow as tf\n",
        "from matplotlib import pyplot as plt\n",
        "\n",
        "# The following lines adjust the granularity of reporting. \n",
        "pd.options.display.max_rows = 10\n",
        "pd.options.display.float_format = \"{:.1f}\".format"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X_TaJhU4KcuY"
      },
      "source": [
        "## The dataset\n",
        "\n",
        "Datasets are often stored on disk or at a URL in [.csv format](https://wikipedia.org/wiki/Comma-separated_values). \n",
        "\n",
        "A well-formed .csv file contains column names in the first row, followed by many rows of data.  A comma divides each value in each row. For example, here are the first five rows of the .csv file holding the California Housing Dataset:\n",
        "\n",
        "```\n",
        "\"longitude\",\"latitude\",\"housing_median_age\",\"total_rooms\",\"total_bedrooms\",\"population\",\"households\",\"median_income\",\"median_house_value\"\n",
        "-114.310000,34.190000,15.000000,5612.000000,1283.000000,1015.000000,472.000000,1.493600,66900.000000\n",
        "-114.470000,34.400000,19.000000,7650.000000,1901.000000,1129.000000,463.000000,1.820000,80100.000000\n",
        "-114.560000,33.690000,17.000000,720.000000,174.000000,333.000000,117.000000,1.650900,85700.000000\n",
        "-114.570000,33.640000,14.000000,1501.000000,337.000000,515.000000,226.000000,3.191700,73400.000000\n",
        "```\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sSFQkzNlj-l6"
      },
      "source": [
        "### Load the .csv file into a pandas DataFrame\n",
        "\n",
        "This Colab, like many machine learning programs, gathers the .csv file and stores the data in memory as a pandas Dataframe.  Pandas is an open source Python library.  The primary datatype in pandas is a DataFrame.  You can imagine a pandas DataFrame as a spreadsheet in which each row is identified by a number and each column by a name. Pandas is itself built on another open source Python library called NumPy. If you aren't familiar with these technologies, please view these two quick tutorials:\n",
        "\n",
        "*   [NumPy](https://colab.research.google.com/github/google/eng-edu/blob/main/ml/cc/exercises/numpy_ultraquick_tutorial.ipynb?utm_source=linearregressionreal-colab&utm_medium=colab&utm_campaign=colab-external&utm_content=numpy_tf2-colab&hl=en)\n",
        "*   [Pandas DataFrames](https://colab.research.google.com/github/google/eng-edu/blob/main/ml/cc/exercises/pandas_dataframe_ultraquick_tutorial.ipynb?utm_source=linearregressionreal-colab&utm_medium=colab&utm_campaign=colab-external&utm_content=pandas_tf2-colab&hl=en)\n",
        "\n",
        "The following code cell imports the .csv file into a pandas DataFrame and scales the values in the label (`median_house_value`):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JZlvdpyYKx7V"
      },
      "source": [
        "# Import the dataset.\n",
        "training_df = pd.read_csv(filepath_or_buffer=\"https://download.mlcc.google.com/mledu-datasets/california_housing_train.csv\")\n",
        "\n",
        "# Scale the label.\n",
        "training_df[\"median_house_value\"] /= 1000.0\n",
        "\n",
        "# Print the first rows of the pandas DataFrame.\n",
        "training_df.head()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5inxx49n4U9u"
      },
      "source": [
        "Scaling `median_house_value` puts the value of each house in units of thousands. Scaling will keep loss values and learning rates in a friendlier range.  \n",
        "\n",
        "Although scaling a label is usually *not* essential, scaling features in a multi-feature model usually *is* essential."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yMysi6-3IAbu"
      },
      "source": [
        "## Examine the dataset\n",
        "\n",
        "A large part of most machine learning projects is getting to know your data. The pandas API provides a `describe` function that outputs the following statistics about every column in the DataFrame:\n",
        "\n",
        "* `count`, which is the number of rows in that column. Ideally, `count` contains the same value for every column. \n",
        "\n",
        "* `mean` and `std`, which contain the mean and standard deviation of the values in each column. \n",
        "\n",
        "* `min` and `max`, which contain the lowest and highest values in each column.\n",
        "\n",
        "* `25%`, `50%`, `75%`, which contain various [quantiles](https://developers.google.com/machine-learning/glossary/#quantile)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rnUSYKw4LUuh"
      },
      "source": [
        "# Get statistics on the dataset.\n",
        "training_df.describe()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f9pcW_Yjtoo8"
      },
      "source": [
        "### Task 1: Identify anomalies in the dataset\n",
        "\n",
        "Do you see any anomalies (strange values) in the data? "
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "### Code for your analysis"
      ],
      "metadata": {
        "id": "aIaX52Gz-Rc7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "TODO: Explain your findings"
      ],
      "metadata": {
        "id": "ZmlVAnH5-RoC"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3014ezH3C7jT"
      },
      "source": [
        "## Define functions that build and train a model\n",
        "\n",
        "The following code defines two functions:\n",
        "\n",
        "  * `build_model(my_learning_rate)`, which builds a randomly-initialized model.\n",
        "  * `train_model(model, feature, label, epochs)`, which trains the model from the examples (feature and label) you pass. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pedD5GhlDC-y"
      },
      "source": [
        "#@title Define the functions that build and train a model\n",
        "def build_model(my_learning_rate):\n",
        "  \"\"\"Create and compile a simple linear regression model.\"\"\"\n",
        "  # Most simple tf.keras models are sequential.\n",
        "  model = tf.keras.models.Sequential()\n",
        "\n",
        "  # Describe the topography of the model.\n",
        "  # The topography of a simple linear regression model\n",
        "  # is a single node in a single layer.\n",
        "  model.add(tf.keras.layers.Dense(units=1, \n",
        "                                  input_shape=(1,)))\n",
        "\n",
        "  # Compile the model topography into code that TensorFlow can efficiently\n",
        "  # execute. Configure training to minimize the model's mean squared error. \n",
        "  model.compile(optimizer=tf.keras.optimizers.experimental.RMSprop(learning_rate=my_learning_rate),\n",
        "                loss=\"mean_squared_error\",\n",
        "                metrics=[tf.keras.metrics.RootMeanSquaredError()])\n",
        "\n",
        "  return model        \n",
        "\n",
        "\n",
        "def train_model(model, df, feature, label, epochs, batch_size):\n",
        "  \"\"\"Train the model by feeding it data.\"\"\"\n",
        "\n",
        "  # Feed the model the feature and the label.\n",
        "  # The model will train for the specified number of epochs. \n",
        "  history = model.fit(x=df[feature],\n",
        "                      y=df[label],\n",
        "                      batch_size=batch_size,\n",
        "                      epochs=epochs)\n",
        "\n",
        "  # Gather the trained model's weight and bias.\n",
        "  trained_weight = model.get_weights()[0]\n",
        "  trained_bias = model.get_weights()[1]\n",
        "\n",
        "  # The list of epochs is stored separately from the rest of history.\n",
        "  epochs = history.epoch\n",
        "  \n",
        "  # Isolate the error for each epoch.\n",
        "  hist = pd.DataFrame(history.history)\n",
        "\n",
        "  # To track the progression of training, we're going to take a snapshot\n",
        "  # of the model's root mean squared error at each epoch. \n",
        "  rmse = hist[\"root_mean_squared_error\"]\n",
        "\n",
        "  return trained_weight, trained_bias, epochs, rmse\n",
        "\n",
        "print(\"Defined the build_model and train_model functions.\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ak_TMAzGOIFq"
      },
      "source": [
        "## Define plotting functions\n",
        "\n",
        "The following [matplotlib](https://developers.google.com/machine-learning/glossary/#matplotlib) functions create the following plots:\n",
        "\n",
        "*  a scatter plot of the feature vs. the label, and a line showing the output of the trained model\n",
        "*  a loss curve\n",
        "\n",
        "You may optionally double-click the headline to see the matplotlib code, but note that writing matplotlib code is not an important part of learning ML programming."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QF0BFRXTOeR3"
      },
      "source": [
        "#@title Define the plotting functions\n",
        "def plot_the_model(trained_weight, trained_bias, feature, label):\n",
        "  \"\"\"Plot the trained model against 200 random training examples.\"\"\"\n",
        "\n",
        "  # Label the axes.\n",
        "  plt.xlabel(feature)\n",
        "  plt.ylabel(label)\n",
        "\n",
        "  # Create a scatter plot from 200 random points of the dataset.\n",
        "  random_examples = training_df.sample(n=200)\n",
        "  plt.scatter(random_examples[feature], random_examples[label])\n",
        "\n",
        "  # Create a red line representing the model. The red line starts\n",
        "  # at coordinates (x0, y0) and ends at coordinates (x1, y1).\n",
        "  x0 = 0\n",
        "  y0 = trained_bias\n",
        "  x1 = random_examples[feature].max()\n",
        "  y1 = trained_bias + (trained_weight * x1)\n",
        "  plt.plot([x0, x1], [y0, y1], c='r')\n",
        "\n",
        "  # Render the scatter plot and the red line.\n",
        "  plt.show()\n",
        "\n",
        "\n",
        "def plot_the_loss_curve(epochs, rmse):\n",
        "  \"\"\"Plot a curve of loss vs. epoch.\"\"\"\n",
        "\n",
        "  plt.figure()\n",
        "  plt.xlabel(\"Epoch\")\n",
        "  plt.ylabel(\"Root Mean Squared Error\")\n",
        "\n",
        "  plt.plot(epochs, rmse, label=\"Loss\")\n",
        "  plt.legend()\n",
        "  plt.ylim([rmse.min()*0.97, rmse.max()])\n",
        "  plt.show()  \n",
        "\n",
        "print(\"Defined the plot_the_model and plot_the_loss_curve functions.\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D-IXYVfvM4gD"
      },
      "source": [
        "## Call the model functions\n",
        "\n",
        "An important part of machine learning is determining which [features](https://developers.google.com/machine-learning/glossary/#feature) correlate with the [label](https://developers.google.com/machine-learning/glossary/#label). For example, real-life home-value prediction models typically rely on hundreds of features and synthetic features. However, this model relies on only one feature. For now, you'll arbitrarily use `total_rooms` as that feature. \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nj3v5EKQFY8s",
        "cellView": "both"
      },
      "source": [
        "# The following variables are the hyperparameters.\n",
        "learning_rate = 0.01\n",
        "epochs = 30\n",
        "batch_size = 30\n",
        "\n",
        "# Specify the feature and the label.\n",
        "my_feature = \"total_rooms\"  # the total number of rooms on a specific city block.\n",
        "my_label=\"median_house_value\" # the median value of a house on a specific city block.\n",
        "# That is, you're going to create a model that predicts house value based \n",
        "# solely on total_rooms.  \n",
        "\n",
        "# Discard any pre-existing version of the model.\n",
        "my_model = None\n",
        "\n",
        "# Invoke the functions.\n",
        "my_model = build_model(learning_rate)\n",
        "weight, bias, epochs, rmse = train_model(my_model, training_df, \n",
        "                                         my_feature, my_label,\n",
        "                                         epochs, batch_size)\n",
        "\n",
        "print(\"\\nThe learned weight for your model is %.4f\" % weight)\n",
        "print(\"The learned bias for your model is %.4f\\n\" % bias )\n",
        "\n",
        "plot_the_model(weight, bias, my_feature, my_label)\n",
        "plot_the_loss_curve(epochs, rmse)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Btp8zUNbYOcd"
      },
      "source": [
        "A certain amount of randomness plays into training a model. Consequently, you'll get different results each time you train the model. That said, given the dataset and the hyperparameters, the trained model will generally do a poor job describing the feature's relation to the label."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1xNqWWos_zyk"
      },
      "source": [
        "## Use the model to make predictions\n",
        "\n",
        "You can use the trained model to make predictions. In practice, [you should make predictions on examples that are not used in training](https://developers.google.com/machine-learning/crash-course/training-and-test-sets/splitting-data). However, for this exercise, you'll just work with a subset of the same training dataset. A later Colab exercise will explore ways to make predictions on examples not used in training.\n",
        "\n",
        "First, run the following code to define the house prediction function:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nH63BmncAcab"
      },
      "source": [
        "def predict_house_values(n, feature, label):\n",
        "  \"\"\"Predict house values based on a feature.\"\"\"\n",
        "\n",
        "  batch = training_df[feature][10000:10000 + n]\n",
        "  predicted_values = my_model.predict_on_batch(x=batch)\n",
        "\n",
        "  print(\"feature   label          predicted\")\n",
        "  print(\"  value   value          value\")\n",
        "  print(\"          in thousand$   in thousand$\")\n",
        "  print(\"--------------------------------------\")\n",
        "  for i in range(n):\n",
        "    print (\"%5.0f %6.0f %15.0f\" % (training_df[feature][10000 + i],\n",
        "                                   training_df[label][10000 + i],\n",
        "                                   predicted_values[i][0] ))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NbBNQujU5WjK"
      },
      "source": [
        "Now, invoke the house prediction function on 10 examples:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y_0DGBt0Kz_N"
      },
      "source": [
        "predict_house_values(10, my_feature, my_label)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-gGaqArcpqY3"
      },
      "source": [
        "### Task 2: Judge the predictive power of the model\n",
        "\n",
        "Look at the preceding table. How close is the predicted value to the label value?  In other words, does your model accurately predict house values?  "
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "### Code for your analysis"
      ],
      "metadata": {
        "id": "MxjjCeUm-nVQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "TODO: Explanation of your analysis"
      ],
      "metadata": {
        "id": "VmQJzEGe-nfA"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wLoqis3IUPSd"
      },
      "source": [
        "## Task 3: Try a different feature\n",
        "\n",
        "The `total_rooms` feature had only a little predictive power. Would a different feature have greater predictive power?  Try using `population` as the feature instead of `total_rooms`. \n",
        "\n",
        "Note: When you change features, you might also need to change the hyperparameters."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H0ab6HD4ZO75"
      },
      "source": [
        "my_feature = \"?\"   # Replace the ? with population or possibly\n",
        "                   # a different column name.\n",
        "\n",
        "# Experiment with the hyperparameters.\n",
        "learning_rate = 2\n",
        "epochs = 3\n",
        "batch_size = 120\n",
        "\n",
        "# Don't change anything below this line.\n",
        "my_model = build_model(learning_rate)\n",
        "weight, bias, epochs, rmse = train_model(my_model, training_df, \n",
        "                                         my_feature, my_label,\n",
        "                                         epochs, batch_size)\n",
        "plot_the_model(weight, bias, my_feature, my_label)\n",
        "plot_the_loss_curve(epochs, rmse)\n",
        "\n",
        "predict_house_values(15, my_feature, my_label)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Nd_rHJ59AUtk"
      },
      "source": [
        "Did `population` produce better predictions than `total_rooms`?"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "TODO: Write-up your answer"
      ],
      "metadata": {
        "id": "aSzt5M8Q_AdR"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C8uYpyGacsIg"
      },
      "source": [
        "## Task 4: Define a synthetic feature\n",
        "\n",
        "You have determined that `total_rooms` and `population` were not useful features.  That is, neither the total number of rooms in a neighborhood nor the neighborhood's population successfully predicted the median house price of that neighborhood. Perhaps though, the *ratio* of `total_rooms` to `population` might have some predictive power. That is, perhaps block density relates to median house value.\n",
        "\n",
        "To explore this hypothesis, do the following: \n",
        "\n",
        "1. Create a [synthetic feature](https://developers.google.com/machine-learning/glossary/#synthetic_feature) that's a ratio of `total_rooms` to `population`. (If you are new to pandas DataFrames, please study the [Pandas DataFrame Ultraquick Tutorial](https://colab.research.google.com/github/google/eng-edu/blob/main/ml/cc/exercises/pandas_dataframe_ultraquick_tutorial.ipynb?utm_source=linearregressionreal-colab&utm_medium=colab&utm_campaign=colab-external&utm_content=pandas_tf2-colab&hl=en).)\n",
        "2. Tune the three hyperparameters.\n",
        "3. Determine whether this synthetic feature produces \n",
        "   a lower loss value than any of the single features you \n",
        "   tried earlier in this exercise."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4Kx2xHSgdcpg"
      },
      "source": [
        "# Define a synthetic feature named rooms_per_person\n",
        "training_df[\"rooms_per_person\"] = ? # write your code here.\n",
        "\n",
        "# Don't change the next line.\n",
        "my_feature = \"rooms_per_person\"\n",
        "\n",
        "# Assign values to these three hyperparameters.\n",
        "learning_rate = ?\n",
        "epochs = ?\n",
        "batch_size = ?\n",
        "\n",
        "# Don't change anything below this line.\n",
        "my_model = build_model(learning_rate)\n",
        "weight, bias, epochs, rmse = train_model(my_model, training_df,\n",
        "                                         my_feature, my_label,\n",
        "                                         epochs, batch_size)\n",
        "\n",
        "plot_the_loss_curve(epochs, rmse)\n",
        "predict_house_values(15, my_feature, my_label)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HBiDWursB1Wi"
      },
      "source": [
        "QUESTION: Based on the loss values, how does the synthetic feature behave?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XEG_9oU9O54u"
      },
      "source": [
        "## Task 5. Find feature(s) whose raw values correlate with the label\n",
        "\n",
        "So far, we've relied on trial-and-error to identify possible features for the model.  Let's rely on statistics instead.\n",
        "\n",
        "A **correlation matrix** indicates how each attribute's raw values relate to the other attributes' raw values. Correlation values have the following meanings:\n",
        "\n",
        "  * `1.0`: perfect positive correlation; that is, when one attribute rises, the other attribute rises.\n",
        "  * `-1.0`: perfect negative correlation; that is, when one attribute rises, the other attribute falls. \n",
        "  * `0.0`: no correlation; the two columns [are not linearly related](https://en.wikipedia.org/wiki/Correlation_and_dependence#/media/File:Correlation_examples2.svg).\n",
        "\n",
        "In general, the higher the absolute value of a correlation value, the greater its predictive power. For example, a correlation value of -0.8 implies far more predictive power than a correlation of -0.2.\n",
        "\n",
        "The following code cell generates the correlation matrix for attributes of the California Housing Dataset:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zFGKL45LO8Tt"
      },
      "source": [
        "# Generate a correlation matrix.\n",
        "training_df.corr()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hp0r3NAVPEdt"
      },
      "source": [
        "The correlation matrix shows nine potential features (including a synthetic\n",
        "feature) and one label (`median_house_value`).  A strong negative correlation or strong positive correlation with the label suggests a potentially good feature.  \n",
        "\n",
        "**Your Task:** Determine which of the nine potential features appears to be the best candidate for a feature?"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#### Place your code here"
      ],
      "metadata": {
        "id": "Ljoo7moR_XuC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8RqvEbaVSlRt"
      },
      "source": [
        "Correlation matrices don't tell the entire story. In later exercises, you'll find additional ways to unlock predictive power from potential features.\n",
        "\n",
        "**Note:** Using `median_income` as a feature may raise some ethical and fairness\n",
        "issues. Towards the end of the course, we'll explore ethical and fairness issues."
      ]
    }
  ]
}