Unit 3 Exercise 1

[lorenzo-delsignore]

Hello, I have these questions:

1. How can you plot a dataset with more features? I applied the principal component analysis and plot the dataset with two features and dispaly it, is it a correct approach? Seems no, since the plot is displayed very bad.
2. In this case, the PCA will be only used for display and not for the rest of the pipeline(e.g dataloaders, dataset, traning etc)?
3. How can you display the learned boundary?

Thanks!

[rasbt]

Hey @lorenzo-delsignore,

using PCA here is actually good thinking! That would be one approach to reduce the dataset into 2D for plotting. But here, I would look at the original feature values because that's what goes into the classifier (you can of course also train a classifier on PCA features, which can sometimes work well too, especially to fit the curse of dimensionality for certain models).

Above, I am recommending plotting the features, but what you say above is also right, we can't do that in a 2D scatterplot. Personally, what I like to do is looking at histograms and pairwise scatterplot. For that, I am often using a scatterplot matrix. It's not necessary but I implemented it here if it helps: https://rasbt.github.io/mlxtend/user_guide/plotting/scatterplotmatrix/

How can you display the learned boundary?

That's a good point. This becomes impossible as well. One could use filler-values for all feature values except a selection of 2 but that is not ideal (I have an implementation of that here: https://rasbt.github.io/mlxtend/user_guide/plotting/plot_decision_regions/#example-7-decision-regions-with-more-than-two-training-features.

Tbh I would not recommend that. In that case, we have to accept that it's not possible to visualize decision boundaries, unfortunately.

[lorenzo-delsignore]

Thanks, so you like to plot one feature against the other features. I tried also with Linear Discriminant Analysis (LDA) in which maximize the variance between classes, but in this case it's a 1D plot since the number of components are num_classes - 1. The displayed plot is more near the accuracy that you get with the model:

What do you think about this approach?

[rasbt]

This would be similar to PCA, but except that you take the class information into account. I would say it still has the same shortcomings as PCA -- it's a linear transformation of the data, not the original data. Not saying it's not useful, but it has its shortcomings.