Iris Species Classification.ipynb

{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Welcome!\n\n* Studied about Machine Learning but don't know where to start using it from?<br>\n* Or you know the concepts but don't know how to implement them?<br>\n* Or are you a beginner in the field of Machine Learning and code tutorials or books are of too high a level for you that they start demotivating you?<br>\nIf so, you are at the right place, becuase I have written this notebook keeping in view all the difficulties I have faced as a beginner (Python is not even my primary language, still I could make it ;) so why can't you?)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"This tutorial is meant for beginners who have atleast some basic theoritical knowledge of fundamental Machine Learning concepts and are looking to implement them practically. We will use the famous Iris Dataset to predict the specie of an Iris flower, given its petal length, petal width, sepal length and sepal width.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We will import 2 Python libraries: Numpy and Pandas as they will be used everywhere in our code\n\n**Note:** One thing I want to make clear at the start of this tutorial is - you **DO NOT** have to remember the code (which initially I used to think is necessary) to proceed further; understanding what is happening and how, is sufficient. Just look what libraries we will use and the parameters in the functions. This will come automatically to you as you keep practicing (Believe me, by the time you will write code for your 4th model, you would be knowing most of the things already)","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Loading Data","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Scikit Learn includes Iris Dataset by default, so we do not need to download the dataset explicitly","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"from sklearn.datasets import load_iris\n\n#Make an object of the dataset for further use\niris_data = load_iris()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see what this dataset contains...","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"iris_data","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Well, it doesn't seems to be much readable, but atleast we got to know that the dataset is in the form of a Python dictionary.\n\nLet's see the keys (or column names, if you can imagine it in the form of a table, with keys as column names and values as column data corresponding to each key)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"iris_data.keys()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So, basically the dataset contains 6 fields.<br>First, let's see what the DESCR (description) contains.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"iris_data.DESCR","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That's some messed up text and it's very difficult to read and understand.<br>Why not try it with print() function?","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(iris_data.DESCR)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"Now it makes much more sense (Read the description once, in case you thought it would be fine skipping it:))\n\nWe should now look at the features and target","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Feature names: \",iris_data.feature_names)\nprint(\"Target names: \",iris_data.target_names)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's look at the actual data itself!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"iris_data.data","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It is more convenient to work on **Pandas Dataframe** than on numpy arrays, so we will convert the data and target arrays to a Pandas Dataframe","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df = pd.DataFrame(data=np.c_[iris_data.data, iris_data.target],\n                 columns= iris_data.feature_names + ['target'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Done that! But how do we know if it was  actually converted or not?\n\nWe can check that by printing some rows of the dataframe.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df.head(5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This looks good and well organised.\n\n**Note:** in target column (or Series, as to say in terms of Pandas), a value of 0 represents 'setosa'. Similarly, a value of 1 represents 'versicolor' and value of 2 represent 'viginica'\n\n\nLet's look at the shape of this dataframe. It should have a total of 150 rows and 5 columns as it was stated in the Description","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Visualizing the Data\n\nBefore starting to train our model, we should have a clear understanding of our data. We should compare and contrast different independent variables (called as features) with the dependent variable (called as target variable) in order to find relationship between them.\n\nOne way to know about the relationships among different variables in our dataset is to plot graphs and compare them visually. We will now plot matrix of graphs that can help us understand the data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#these libraries will help us in plotting graph\nimport matplotlib as plt\n%matplotlib inline\nimport seaborn as sns","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's first define our feature and target variables.<br>\nFor the sake of simplicity, currently I am taking all 4 independent variables as features.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"features = ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']\ntarget = 'target'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.pairplot(df, hue=\"target\")\n# help(sns.pairplot)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From these graphs, it can be clearly observed Iris flowers with small sepals and petals are Iris-Setosa (with target = 0 and in blue color).\n\nWe can also infer that Iris-Virginica's (target = 2 and in green color) petal length and width are greater than those of Iris-Versicolor (target = 1 and in orange color)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Train-Test Data Split\n\nBefore we finally move on to train our model and start predicting, we need to ensure that we should have some data through which we can validate our model. A model, if trained and tested on the same data, may give very high accuracy for the in-sample data (data on which it was trained), since it *knows* what the output should be. But this model may (and surely it will, in most cases) give very low accuracy for out of sample data.\n\nIt is usually good to have 70-80% of data for training and rest for testing.\n\nWe will use train_test_split from scikit learn to split our data into training and testing sets.<br>\nBy default, it splits data in the ratio of 3:1 (75% training and 25% test).","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"X_data = df[features]\ny_data = df[target]\nprint(X_data)\nprint(y_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nX_train, X_val, y_train, y_val = train_test_split(X_data, y_data, random_state=0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's check the number of entries in each set.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(X_train.shape)\nprint(X_val.shape)\nprint(y_train.shape)\nprint(y_val.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train the Model\nOk, we are finally here. We will now train our Model!!\n\nBut wait, how should we choose which model will be good?\n\n* We have some data with corresponding result to look at and we are going to train model which will learn from this data, so this is a Supervised Learning problem.\n* Also, we know that we need to predict the specie of Iris flower, which can be either 0 (Setosa), 1 (Versicolor) or 2 (Virginica), i.e. we need to classify the flower in one of the 3 categories. So this is a Classification problem.\n* For classification of data, we can use either Decision Tree or Random Forest or KNN (K-Nearest Neighbors) or Naive Bayes Model. Which one should we pick here?\n* One way is that that we can try each one of these and finally choose the one which gives the best result. We can do that here since we have a very small dataset, but this method may take a lot of time for training and validation when we have a huge dataset and that might make it practically impossible to implement.\n* Therefore, we will meticulously choose one of the above models. In the graphs plotted earlier, we can clearly observe that the species are clustered in some range of values. So, if we have a point on graph representing the properties of a flower, we can look around for other points with similar properties and corresponding specie and can predict the specie of this point based on the results of our observations.\n* This is exactly what a KNN algorithm does.\n\nHence, we will use KNN model (KNeighborsClassifier) to train our data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.neighbors import KNeighborsClassifier\n\nknn = KNeighborsClassifier(n_neighbors=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since, we know the clusters of species are well separated from each other, we can just look at the nearest neighbor and predict the specie. But this won't be the case with most of the practical problems. We will have to find the optimum number of neighbors(k) we should look before making any prediction.\n\nTo find the best value of the number of neighbours(k), we can try running a loop for all values of k and select that value for which we get the highest accuracy. But, when we have huge dataset, we cannot practically train a model, predict and measure accuracy for each and every possible value of k as this might take a very very long time to complete. If the target values occur in cluster, it is sensible to observe only some of its neighbors to predict its value. Therefore we can run a loop from k=1 to say k=20 approx. (higher limit depends on the distribution of data) and do as stated above.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"You can try uncommenting and running the below code to see how accuracy changes with change in value of k. Notice the fluctuations in accuracy for k=27 to k=33.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# from sklearn.metrics import accuracy_score\n# best_k, max_acc=0,0\n# for k in range(1, 112):\n#     knn = KNeighborsClassifier(n_neighbors=k)\n#     knn.fit(X_train, y_train)\n#     pred = knn.predict(X_val)\n#     acc = accuracy_score(y_val, pred)\n#     print(k, acc)\n#     if acc > max_acc:\n#         max_acc = acc\n#         best_k = k\n# knn = KNeighborsClassifier(n_neighbors=best_k)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We should now fit our model and predict the target value for X_val.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"knn.fit(X_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pred = knn.predict(X_val)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can measure the accuracy of our model using the accracy_score from sklearn.metrics ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.metrics import accuracy_score\naccuracy_score(y_val, pred)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Wohoo! We got an accuracy of 97%. That's really good!\n\n**Note:** We don't have any explicit test dataset (i.e. for which we do not already have target values) here to use our model and predict target value. Otherwise, after performing the validation and getting a good accuracy, we must train our data on whole dataset, i.e. on X_data (X_train+X_val) and y_data (y_train+y_val), else we will lose a part of our precious data in the form of validation data.\n\nUncomment the following code and run to train our model on whole available data.\nYou can try testing it on some hypothetical data of your own.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# knn.fit(X_data, y_data)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I hope you enjoyed and learned something<br>\nKeep practicing!","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}