Amazon_logisticregression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yI8hHeMFzjQp"
   },
   "source": [
    "  APPLYING LOGISTIC REGRESSION FOR AMAZON DATASET\n",
    "  ====================================\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7ppnhwuLmXQ6"
   },
   "source": [
    "To apply Logistic regression and do hyperparameter tunning by Grid-search and Random-search for both L1 an L2  regularization.\n",
    "-----------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "O_Joi4dRzTjP"
   },
   "outputs": [],
   "source": [
    "import pandas as pd \n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plot\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn import linear_model, datasets\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import confusion_matrix\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 369
    },
    "colab_type": "code",
    "id": "a6Uyz4sixmgZ",
    "outputId": "54335b96-2bc8-4b9c-97cc-f86294e63f49"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>ProfileName</th>\n",
       "      <th>HelpfulnessNumerator</th>\n",
       "      <th>HelpfulnessDenominator</th>\n",
       "      <th>Score</th>\n",
       "      <th>Time</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "      <th>cleanedText</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>150524</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>ACITT7DI6IDDL</td>\n",
       "      <td>shari zychinski</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>939340800</td>\n",
       "      <td>EVERY book is educational</td>\n",
       "      <td>this witty little book makes my son laugh at l...</td>\n",
       "      <td>b'witti littl book make son laugh loud recit c...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>150506</td>\n",
       "      <td>0006641040</td>\n",
       "      <td>A2IW4PEEKO2R0U</td>\n",
       "      <td>Tracy</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1194739200</td>\n",
       "      <td>Love the book, miss the hard cover version</td>\n",
       "      <td>I grew up reading these Sendak books, and watc...</td>\n",
       "      <td>b'grew read sendak book watch realli rosi movi...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Id   ProductId          UserId      ProfileName  HelpfulnessNumerator  \\\n",
       "0  150524  0006641040   ACITT7DI6IDDL  shari zychinski                     0   \n",
       "1  150506  0006641040  A2IW4PEEKO2R0U            Tracy                     1   \n",
       "\n",
       "   HelpfulnessDenominator  Score        Time  \\\n",
       "0                       0      1   939340800   \n",
       "1                       1      1  1194739200   \n",
       "\n",
       "                                      Summary  \\\n",
       "0                   EVERY book is educational   \n",
       "1  Love the book, miss the hard cover version   \n",
       "\n",
       "                                                Text  \\\n",
       "0  this witty little book makes my son laugh at l...   \n",
       "1  I grew up reading these Sendak books, and watc...   \n",
       "\n",
       "                                         cleanedText  \n",
       "0  b'witti littl book make son laugh loud recit c...  \n",
       "1  b'grew read sendak book watch realli rosi movi...  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df=pd.read_csv('final_clean.csv')\n",
    "df.head(2)#loading preprocessed data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 111
    },
    "colab_type": "code",
    "id": "nCcrn9nB3JtW",
    "outputId": "7f212104-4e23-45b2-e1ae-c5bf72cdb5bb"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cleanedText</th>\n",
       "      <th>Text</th>\n",
       "      <th>Score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>b'witti littl book make son laugh loud recit c...</td>\n",
       "      <td>this witty little book makes my son laugh at l...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b'grew read sendak book watch realli rosi movi...</td>\n",
       "      <td>I grew up reading these Sendak books, and watc...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         cleanedText  \\\n",
       "0  b'witti littl book make son laugh loud recit c...   \n",
       "1  b'grew read sendak book watch realli rosi movi...   \n",
       "\n",
       "                                                Text  Score  \n",
       "0  this witty little book makes my son laugh at l...      1  \n",
       "1  I grew up reading these Sendak books, and watc...      1  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "data = df.filter(['cleanedText','Text','Score'],axis=1)\n",
    "data.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zNi1FeRnxmgl"
   },
   "outputs": [],
   "source": [
    "\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "from collections import Counter\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn import cross_validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "A7LG1IcFCgXW"
   },
   "outputs": [],
   "source": [
    "X = data['cleanedText']\n",
    "y = data['Score']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AX_BcCF10LBD"
   },
   "source": [
    "SPLITTING DATA IN TO TRAIN AND TEST AND FEATURIZE TEXT UDING TF-IDF\n",
    "--------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "_WnYJHViBxLN",
    "outputId": "de62132b-1433-4bf5-a087-8cbb848f8875"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(273098,)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "x_train, x_test, y_train, y_test = train_test_split(X,y)\n",
    "y_train.shape\n",
    "#spliting entire data into train and test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "A9FCFIwexmgt"
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "tfidf_vect = TfidfVectorizer()#tfidf featurization to convert text to vectors\n",
    "tfidf_train= tfidf_vect.fit_transform(x_train)\n",
    "tfidf_test = tfidf_vect.transform(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mqJbvL-20h-o"
   },
   "source": [
    "OBTAINING OPTIMAL PENALITY(L1,L2) AND LAMBDA USING GRID-SEARCH CV\n",
    "-------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "39YRvukpxmhF"
   },
   "outputs": [],
   "source": [
    "# Create regularization penalty space\n",
    "penality=('l1','l2')\n",
    "\n",
    "# Create regularization hyperparameter space\n",
    "C = np.logspace(0, 4, 10)\n",
    "\n",
    "# Create hyperparameter options\n",
    "hyperparameters = dict(C=C, penalty=penality)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "FJ3rY9ah6dY3"
   },
   "outputs": [],
   "source": [
    "logistic = linear_model.LogisticRegression()#defining our model\n",
    "cf = GridSearchCV(logistic, hyperparameters, cv=5, verbose=30,n_jobs=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 4193
    },
    "colab_type": "code",
    "id": "01LosKKQxmhc",
    "outputId": "6977ca75-018f-4747-df12-1c6c7fe194a4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 20 candidates, totalling 100 fits\n",
      "[CV] C=1.0, penalty=l1 ...............................................\n",
      "[CV] C=1.0, penalty=l1 ...............................................\n",
      "[CV] C=1.0, penalty=l1 ...............................................\n",
      "[CV] C=1.0, penalty=l1 ...............................................\n",
      "[CV] C=1.0, penalty=l1 ...............................................\n",
      "[CV] C=1.0, penalty=l2 ...............................................\n",
      "[CV] C=1.0, penalty=l2 ...............................................\n",
      "[CV] C=1.0, penalty=l2 ...............................................\n",
      "[CV] ...... C=1.0, penalty=l2, score=0.9235627975100695, total=  16.9s\n",
      "[CV] ...... C=1.0, penalty=l2, score=0.9234712559502014, total=  16.4s\n",
      "[CV] C=1.0, penalty=l2 ...............................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   1 tasks      | elapsed:   18.7s\n",
      "[Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:   18.7s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] C=1.0, penalty=l2 ...............................................\n",
      "[CV] ...... C=1.0, penalty=l2, score=0.9229769315269132, total=  17.2s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   3 tasks      | elapsed:   19.2s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] C=2.7825594022071245, penalty=l1 ................................\n",
      "[CV] ....... C=1.0, penalty=l1, score=0.924731773408034, total=  18.0s\n",
      "[CV] C=2.7825594022071245, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   4 tasks      | elapsed:   19.6s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ...... C=1.0, penalty=l1, score=0.9233064811424386, total=  19.6s\n",
      "[CV] C=2.7825594022071245, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   5 tasks      | elapsed:   20.2s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... C=1.0, penalty=l1, score=0.923489564262175, total=  20.6s\n",
      "[CV] C=2.7825594022071245, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   6 tasks      | elapsed:   21.5s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ...... C=1.0, penalty=l1, score=0.9250823874038814, total=  20.8s\n",
      "[CV] ...... C=1.0, penalty=l1, score=0.9233064811424386, total=  21.3s\n",
      "[CV] C=2.7825594022071245, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   7 tasks      | elapsed:   22.0s\n",
      "[Parallel(n_jobs=-1)]: Done   8 tasks      | elapsed:   22.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] C=2.7825594022071245, penalty=l2 ................................\n",
      "[CV] ...... C=1.0, penalty=l2, score=0.9234501446409609, total=  15.6s\n",
      "[CV] C=2.7825594022071245, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   9 tasks      | elapsed:   34.8s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ...... C=1.0, penalty=l2, score=0.9242585133650677, total=  17.6s\n",
      "[CV] C=2.7825594022071245, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  10 tasks      | elapsed:   36.4s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l2, score=0.9246429879165141, total=  20.8s\n",
      "[CV] C=2.7825594022071245, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  11 tasks      | elapsed:   43.2s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l1, score=0.9237824972537532, total=  24.0s\n",
      "[CV] C=2.7825594022071245, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  12 tasks      | elapsed:   43.8s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l1, score=0.9259428780666422, total=  23.7s\n",
      "[CV] C=7.742636826811269, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  13 tasks      | elapsed:   45.4s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l1, score=0.9239655803734895, total=  26.5s\n",
      "[CV] C=7.742636826811269, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  14 tasks      | elapsed:   46.2s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l1, score=0.9245120656193929, total=  24.3s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  15 tasks      | elapsed:   46.7s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] C=7.742636826811269, penalty=l1 .................................\n",
      "[CV]  C=2.7825594022071245, penalty=l1, score=0.9249725375320396, total=  28.6s\n",
      "[CV] C=7.742636826811269, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:   49.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l2, score=0.9245331380446723, total=  20.8s\n",
      "[CV] C=7.742636826811269, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  17 tasks      | elapsed:   55.8s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l2, score=0.9249725375320396, total=  21.2s\n",
      "[CV] C=7.742636826811269, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  18 tasks      | elapsed:   57.7s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l2, score=0.9249514811966751, total=  21.9s\n",
      "[CV] C=7.742636826811269, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  19 tasks      | elapsed:  1.1min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=2.7825594022071245, penalty=l2, score=0.9256682533870377, total=  27.5s\n",
      "[CV] C=7.742636826811269, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  20 tasks      | elapsed:  1.2min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l1, score=0.9215122665690223, total=  32.5s\n",
      "[CV] C=7.742636826811269, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  21 tasks      | elapsed:  1.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l1, score=0.9213291834492859, total=  33.3s\n",
      "[CV] C=7.742636826811269, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  22 tasks      | elapsed:  1.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l1, score=0.9213658000732332, total=  34.4s\n",
      "[CV] C=21.544346900318832, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  23 tasks      | elapsed:  1.4min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l1, score=0.9222995239838887, total=  34.0s\n",
      "[CV] C=21.544346900318832, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  24 tasks      | elapsed:  1.4min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l2, score=0.9235261808861223, total=  28.4s\n",
      "[CV] C=21.544346900318832, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  25 tasks      | elapsed:  1.4min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l1, score=0.9210150499835219, total=  33.4s\n",
      "[CV] C=21.544346900318832, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  26 tasks      | elapsed:  1.5min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l2, score=0.9242218967411204, total=  31.6s\n",
      "[CV] C=21.544346900318832, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  27 tasks      | elapsed:  1.6min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l2, score=0.9244049798608568, total=  30.9s\n",
      "[CV] C=21.544346900318832, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  28 tasks      | elapsed:  1.7min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l2, score=0.9238895602182431, total=  31.7s\n",
      "[CV] C=21.544346900318832, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  29 tasks      | elapsed:  1.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=7.742636826811269, penalty=l2, score=0.9257964115708531, total=  35.1s\n",
      "[CV] C=21.544346900318832, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  30 tasks      | elapsed:  1.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l1, score=0.9166239472720615, total=  40.1s\n",
      "[CV] C=21.544346900318832, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  31 tasks      | elapsed:  2.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l1, score=0.9167704137678506, total=  39.1s\n",
      "[CV] C=21.544346900318832, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  32 tasks      | elapsed:  2.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l1, score=0.9156719150494325, total=  36.8s\n",
      "[CV] C=59.94842503189409, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  33 tasks      | elapsed:  2.1min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l1, score=0.9152325155620652, total=  40.4s\n",
      "[CV] C=59.94842503189409, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  2.1min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l1, score=0.9166025852283131, total=  36.8s\n",
      "[CV] C=59.94842503189409, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  35 tasks      | elapsed:  2.2min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l2, score=0.9221164408641523, total=  41.8s\n",
      "[CV] C=59.94842503189409, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  36 tasks      | elapsed:  2.6min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l2, score=0.9219150494324423, total=  52.3s\n",
      "[CV] C=59.94842503189409, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  37 tasks      | elapsed:  2.6min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l2, score=0.9223727572317832, total=  47.1s\n",
      "[CV] C=59.94842503189409, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  38 tasks      | elapsed:  2.7min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l2, score=0.9233247894544123, total=  43.0s\n",
      "[CV] C=59.94842503189409, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  39 tasks      | elapsed:  2.8min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=21.544346900318832, penalty=l2, score=0.9220952799443407, total=  48.5s\n",
      "[CV] C=59.94842503189409, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  40 tasks      | elapsed:  2.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l1, score=0.9113694617356279, total=  45.2s\n",
      "[CV] C=59.94842503189409, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  41 tasks      | elapsed:  2.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l1, score=0.9107469791285243, total=  46.6s\n",
      "[CV] C=59.94842503189409, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  42 tasks      | elapsed:  2.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l1, score=0.9101428048333944, total=  46.1s\n",
      "[CV] C=166.81005372000593, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  43 tasks      | elapsed:  3.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l1, score=0.9100695715854998, total=  42.0s\n",
      "[CV] C=166.81005372000593, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  44 tasks      | elapsed:  3.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l1, score=0.9107437108645502, total=  42.6s\n",
      "[CV] C=166.81005372000593, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  45 tasks      | elapsed:  3.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l1, score=0.9074880995972171, total=  41.0s\n",
      "[CV] C=166.81005372000593, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  46 tasks      | elapsed:  3.7min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l2, score=0.9194983522519223, total=  54.5s\n",
      "[CV] C=166.81005372000593, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  47 tasks      | elapsed:  3.8min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l2, score=0.9196082021237642, total= 1.1min\n",
      "[CV] C=166.81005372000593, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  48 tasks      | elapsed:  3.8min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l2, score=0.9193885023800805, total= 1.2min\n",
      "[CV] C=166.81005372000593, penalty=l2 ................................\n",
      "[CV]  C=59.94842503189409, penalty=l2, score=0.9191658427624593, total=  58.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  49 tasks      | elapsed:  3.9min\n",
      "[Parallel(n_jobs=-1)]: Done  50 tasks      | elapsed:  3.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] C=166.81005372000593, penalty=l2 ................................\n",
      "[CV]  C=166.81005372000593, penalty=l1, score=0.9062614426949835, total=  42.0s\n",
      "[CV] C=166.81005372000593, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  51 tasks      | elapsed:  4.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=59.94842503189409, penalty=l2, score=0.9183998535335042, total= 1.1min\n",
      "[CV] C=166.81005372000593, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  52 tasks      | elapsed:  4.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l1, score=0.9056389600878799, total=  48.5s\n",
      "[CV] C=464.15888336127773, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  53 tasks      | elapsed:  4.1min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l1, score=0.905584035151959, total=  48.8s\n",
      "[CV] C=464.15888336127773, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  54 tasks      | elapsed:  4.5min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l1, score=0.9062946281445677, total=  45.9s\n",
      "[CV] C=464.15888336127773, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  55 tasks      | elapsed:  4.5min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l1, score=0.9035518125228854, total=  54.0s\n",
      "[CV] C=464.15888336127773, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  56 tasks      | elapsed:  5.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l2, score=0.9146283412669352, total= 1.3min\n",
      "[CV] C=464.15888336127773, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  57 tasks      | elapsed:  5.2min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l1, score=0.9034785792749909, total=  43.7s\n",
      "[CV] C=464.15888336127773, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  58 tasks      | elapsed:  5.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l1, score=0.9020139143171, total=  46.3s\n",
      "[CV] C=464.15888336127773, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  59 tasks      | elapsed:  5.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l2, score=0.9150677407543024, total= 1.4min\n",
      "[CV] C=464.15888336127773, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  60 tasks      | elapsed:  5.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l2, score=0.9158733064811424, total= 1.6min\n",
      "[CV] C=464.15888336127773, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  61 tasks      | elapsed:  5.5min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l2, score=0.916093006224826, total= 1.6min\n",
      "[CV] C=464.15888336127773, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  62 tasks      | elapsed:  5.5min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l1, score=0.9020505309410473, total=  43.5s\n",
      "[CV] C=1291.5496650148827, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  63 tasks      | elapsed:  5.7min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=166.81005372000593, penalty=l2, score=0.9154491193379471, total= 1.9min\n",
      "[CV] C=1291.5496650148827, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  64 tasks      | elapsed:  5.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l1, score=0.9031088652092717, total=  52.9s\n",
      "[CV] C=1291.5496650148827, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  65 tasks      | elapsed:  6.1min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l1, score=0.9017026730135481, total=  53.1s\n",
      "[CV] C=1291.5496650148827, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  66 tasks      | elapsed:  6.6min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l1, score=0.9020505309410473, total=  53.4s\n",
      "[CV] C=1291.5496650148827, penalty=l1 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  67 tasks      | elapsed:  6.8min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l1, score=0.8997803002563164, total=  57.5s\n",
      "[CV] C=1291.5496650148827, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  68 tasks      | elapsed:  7.1min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l2, score=0.9098315635298425, total= 1.9min\n",
      "[CV] C=1291.5496650148827, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  69 tasks      | elapsed:  7.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l2, score=0.9122482607103625, total= 2.1min\n",
      "[CV] C=1291.5496650148827, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  70 tasks      | elapsed:  7.4min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l1, score=0.9003295496155255, total=  55.0s\n",
      "[CV] C=1291.5496650148827, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  71 tasks      | elapsed:  7.5min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l1, score=0.9008385513933136, total=  50.4s\n",
      "[CV] C=1291.5496650148827, penalty=l2 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  72 tasks      | elapsed:  7.6min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l2, score=0.9108718737412574, total= 2.4min\n",
      "[CV] C=3593.813663804626, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  73 tasks      | elapsed:  7.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=3593.813663804626, penalty=l1, score=0.9000366166239473, total=  51.1s\n",
      "[CV] C=3593.813663804626, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  74 tasks      | elapsed:  8.8min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=3593.813663804626, penalty=l1, score=0.9001281581838154, total= 1.0min\n",
      "[CV] C=3593.813663804626, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  75 tasks      | elapsed:  9.8min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l2, score=0.9122299523983889, total= 4.7min\n",
      "[CV] C=3593.813663804626, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  76 tasks      | elapsed: 10.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=464.15888336127773, penalty=l2, score=0.9104906627608934, total= 4.7min\n",
      "[CV] C=3593.813663804626, penalty=l1 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  77 tasks      | elapsed: 10.2min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=3593.813663804626, penalty=l1, score=0.8979128524350055, total=  52.5s\n",
      "[CV] C=3593.813663804626, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  78 tasks      | elapsed: 10.7min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=3593.813663804626, penalty=l1, score=0.8992859758330282, total=  52.6s\n",
      "[CV] C=3593.813663804626, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  79 tasks      | elapsed: 10.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l2, score=0.9072317832295862, total= 3.6min\n",
      "[CV] C=3593.813663804626, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  80 tasks      | elapsed: 11.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=3593.813663804626, penalty=l1, score=0.8991541250137317, total=  49.7s\n",
      "[CV] C=3593.813663804626, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  81 tasks      | elapsed: 11.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l2, score=0.9084401318198462, total= 4.4min\n",
      "[CV] C=3593.813663804626, penalty=l2 .................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  82 tasks      | elapsed: 11.7min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l2, score=0.9083302819480044, total= 5.6min\n",
      "[CV] C=10000.0, penalty=l1 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  83 tasks      | elapsed: 12.7min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l2, score=0.9081621443480171, total= 5.4min\n",
      "[CV] C=10000.0, penalty=l1 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  84 tasks      | elapsed: 13.0min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ... C=10000.0, penalty=l1, score=0.898645184913951, total= 1.0min\n",
      "[CV] C=10000.0, penalty=l1 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  85 tasks      | elapsed: 13.7min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .. C=10000.0, penalty=l1, score=0.8986268766019773, total= 1.0min\n",
      "[CV] C=10000.0, penalty=l1 ...........................................\n",
      "[CV] .. C=10000.0, penalty=l1, score=0.8967960454046137, total= 1.0min\n",
      "[CV] C=10000.0, penalty=l1 ...........................................\n",
      "[CV] .. C=10000.0, penalty=l1, score=0.8981691688026364, total=  58.3s\n",
      "[CV] C=10000.0, penalty=l2 ...........................................\n",
      "[CV] .. C=10000.0, penalty=l1, score=0.8979274231938189, total=  55.5s\n",
      "[CV] C=10000.0, penalty=l2 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  89 out of 100 | elapsed: 15.6min remaining:  1.9min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=1291.5496650148827, penalty=l2, score=0.9075430245331381, total= 8.9min\n",
      "[CV] C=10000.0, penalty=l2 ...........................................\n",
      "[CV]  C=3593.813663804626, penalty=l2, score=0.9061847742502471, total= 5.6min\n",
      "[CV] C=10000.0, penalty=l2 ...........................................\n",
      "[CV]  C=3593.813663804626, penalty=l2, score=0.9045953863053826, total= 6.7min\n",
      "[CV] C=10000.0, penalty=l2 ...........................................\n",
      "[CV]  C=3593.813663804626, penalty=l2, score=0.9068656169901135, total= 7.0min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  93 out of 100 | elapsed: 17.9min remaining:  1.3min\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  C=3593.813663804626, penalty=l2, score=0.9061699011351153, total= 7.5min\n",
      "[CV]  C=3593.813663804626, penalty=l2, score=0.9074880995972171, total= 8.1min\n",
      "[CV] ... C=10000.0, penalty=l2, score=0.905181252288539, total= 6.2min\n",
      "[CV] .. C=10000.0, penalty=l2, score=0.9066642255584035, total= 7.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  97 out of 100 | elapsed: 22.2min remaining:   41.2s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .. C=10000.0, penalty=l2, score=0.9042074041524772, total= 5.4min\n",
      "[CV] .. C=10000.0, penalty=l2, score=0.9032771878432808, total= 5.8min\n",
      "[CV] .. C=10000.0, penalty=l2, score=0.9035335042109117, total= 6.7min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done 100 out of 100 | elapsed: 23.1min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:23:17.467650\n"
     ]
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "startTime = datetime.now()\n",
    "\n",
    "#lets run GridsearchCV on our data\n",
    "bestmodel=cf.fit(tfidf_train, y_train)\n",
    "\n",
    "print(datetime.now() - startTime)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "colab_type": "code",
    "id": "nUVCXKup8dNr",
    "outputId": "795a1037-608e-4a8f-8456-4fbe0e8dcb59"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Penalty: l2\n",
      "Best C: 2.7825594022071245\n"
     ]
    }
   ],
   "source": [
    "print('Best Penalty:', bestmodel.best_estimator_.get_params()['penalty'])\n",
    "print('Best C:', bestmodel.best_estimator_.get_params()['C'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nGoMQPoCxmhm"
   },
   "outputs": [],
   "source": [
    "y_true, y_pred = y_test,cf.predict(tfidf_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 161
    },
    "colab_type": "code",
    "id": "xYmhE-_rxmhw",
    "outputId": "333b37b9-d27f-41cc-8d6d-0eeeeecb4835"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.83      0.67      0.74     14341\n",
      "          1       0.94      0.97      0.96     76692\n",
      "\n",
      "avg / total       0.92      0.93      0.92     91033\n",
      "\n",
      "(0.925455604011732, 0.925455604011732, 0.925455604011732, None)\n"
     ]
    }
   ],
   "source": [
    "\n",
    "from sklearn.metrics import precision_recall_fscore_support, classification_report\n",
    "\n",
    "print(classification_report(y_test, y_pred))\n",
    "print(precision_recall_fscore_support(y_test, y_pred, average='micro'))\n",
    "#estimating various performance measures on test data by using our trained model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oj_YSyrg1A2B"
   },
   "source": [
    "OBTAINING OPTIMAL PENALITY(L1,L2) AND LAMBDA USING RANDOM-SEARCH CV\n",
    "-------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "KrYOo7yQxmhz"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "clf  =RandomizedSearchCV(logistic, hyperparameters, cv=5, verbose=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 2015
    },
    "colab_type": "code",
    "id": "Kl9uT0wSxmh3",
    "outputId": "019d8c9d-2967-4ce3-9e9e-30036f150dc5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 10 candidates, totalling 50 fits\n",
      "[CV] penalty=l2, C=21.544346900318832 ................................\n",
      "[CV]  penalty=l2, C=21.544346900318832, score=0.9219150494324423, total=  15.7s\n",
      "[CV] penalty=l2, C=21.544346900318832 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   15.7s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  penalty=l2, C=21.544346900318832, score=0.9221164408641523, total=  13.4s\n",
      "[CV] penalty=l2, C=21.544346900318832 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:   29.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  penalty=l2, C=21.544346900318832, score=0.9223727572317832, total=  13.3s\n",
      "[CV] penalty=l2, C=21.544346900318832 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:   42.5s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  penalty=l2, C=21.544346900318832, score=0.9233247894544123, total=  13.5s\n",
      "[CV] penalty=l2, C=21.544346900318832 ................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:   56.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV]  penalty=l2, C=21.544346900318832, score=0.9220952799443407, total=  13.2s\n",
      "[CV] penalty=l1, C=10000.0 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:  1.2min remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .. penalty=l1, C=10000.0, score=0.8986268766019773, total=  14.5s\n",
      "[CV] penalty=l1, C=10000.0 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   6 out of   6 | elapsed:  1.4min remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .. penalty=l1, C=10000.0, score=0.8987733430977664, total=  16.5s\n",
      "[CV] penalty=l1, C=10000.0 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   7 out of   7 | elapsed:  1.7min remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .. penalty=l1, C=10000.0, score=0.8967777370926401, total=  15.9s\n",
      "[CV] penalty=l1, C=10000.0 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   8 out of   8 | elapsed:  1.9min remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .. penalty=l1, C=10000.0, score=0.8981508604906627, total=  15.3s\n",
      "[CV] penalty=l1, C=10000.0 ...........................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   9 out of   9 | elapsed:  2.2min remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .. penalty=l1, C=10000.0, score=0.8979274231938189, total=  17.2s\n",
      "[CV] penalty=l1, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l1, C=7.742636826811269, score=0.921475649945075, total=  11.0s\n",
      "[CV] penalty=l1, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l1, C=7.742636826811269, score=0.9213291834492859, total=  11.4s\n",
      "[CV] penalty=l1, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l1, C=7.742636826811269, score=0.9213841083852069, total=  11.2s\n",
      "[CV] penalty=l1, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l1, C=7.742636826811269, score=0.922336140607836, total=  11.3s\n",
      "[CV] penalty=l1, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l1, C=7.742636826811269, score=0.9209967410011352, total=  10.9s\n",
      "[CV] penalty=l2, C=59.94842503189409 .................................\n",
      "[CV]  penalty=l2, C=59.94842503189409, score=0.9193885023800805, total=  19.6s\n",
      "[CV] penalty=l2, C=59.94842503189409 .................................\n",
      "[CV]  penalty=l2, C=59.94842503189409, score=0.9196082021237642, total=  20.6s\n",
      "[CV] penalty=l2, C=59.94842503189409 .................................\n",
      "[CV]  penalty=l2, C=59.94842503189409, score=0.9183998535335042, total=  18.0s\n",
      "[CV] penalty=l2, C=59.94842503189409 .................................\n",
      "[CV]  penalty=l2, C=59.94842503189409, score=0.9194983522519223, total=  16.4s\n",
      "[CV] penalty=l2, C=59.94842503189409 .................................\n",
      "[CV]  penalty=l2, C=59.94842503189409, score=0.9191658427624593, total=  18.8s\n",
      "[CV] penalty=l1, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l1, C=166.81005372000593, score=0.9074880995972171, total=  14.0s\n",
      "[CV] penalty=l1, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l1, C=166.81005372000593, score=0.9063896008787989, total=  13.8s\n",
      "[CV] penalty=l1, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l1, C=166.81005372000593, score=0.9055657268399854, total=  15.2s\n",
      "[CV] penalty=l1, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l1, C=166.81005372000593, score=0.905584035151959, total=  14.5s\n",
      "[CV] penalty=l1, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l1, C=166.81005372000593, score=0.9062946281445677, total=  14.4s\n",
      "[CV] penalty=l2, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l2, C=1291.5496650148827, score=0.9083302819480044, total= 1.5min\n",
      "[CV] penalty=l2, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l2, C=1291.5496650148827, score=0.9084401318198462, total= 1.2min\n",
      "[CV] penalty=l2, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l2, C=1291.5496650148827, score=0.9072317832295862, total= 1.0min\n",
      "[CV] penalty=l2, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l2, C=1291.5496650148827, score=0.9075430245331381, total= 2.1min\n",
      "[CV] penalty=l2, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l2, C=1291.5496650148827, score=0.9081621443480171, total= 1.2min\n",
      "[CV] penalty=l2, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l2, C=166.81005372000593, score=0.9158733064811424, total=  29.7s\n",
      "[CV] penalty=l2, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l2, C=166.81005372000593, score=0.916093006224826, total=  27.0s\n",
      "[CV] penalty=l2, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l2, C=166.81005372000593, score=0.9146283412669352, total=  26.0s\n",
      "[CV] penalty=l2, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l2, C=166.81005372000593, score=0.9150677407543024, total=  24.6s\n",
      "[CV] penalty=l2, C=166.81005372000593 ................................\n",
      "[CV]  penalty=l2, C=166.81005372000593, score=0.9154491193379471, total=  28.5s\n",
      "[CV] penalty=l2, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l2, C=7.742636826811269, score=0.9235261808861223, total=   9.6s\n",
      "[CV] penalty=l2, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l2, C=7.742636826811269, score=0.9242218967411204, total=  10.6s\n",
      "[CV] penalty=l2, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l2, C=7.742636826811269, score=0.9244049798608568, total=   9.1s\n",
      "[CV] penalty=l2, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l2, C=7.742636826811269, score=0.9257964115708531, total=  10.4s\n",
      "[CV] penalty=l2, C=7.742636826811269 .................................\n",
      "[CV]  penalty=l2, C=7.742636826811269, score=0.9238895602182431, total=  10.0s\n",
      "[CV] penalty=l1, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l1, C=1291.5496650148827, score=0.9017759062614427, total=  15.4s\n",
      "[CV] penalty=l1, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l1, C=1291.5496650148827, score=0.9020505309410473, total=  15.1s\n",
      "[CV] penalty=l1, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l1, C=1291.5496650148827, score=0.8998718418161845, total=  15.6s\n",
      "[CV] penalty=l1, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l1, C=1291.5496650148827, score=0.9003295496155255, total=  14.7s\n",
      "[CV] penalty=l1, C=1291.5496650148827 ................................\n",
      "[CV]  penalty=l1, C=1291.5496650148827, score=0.9008568603757003, total=  12.5s\n",
      "[CV] penalty=l1, C=3593.813663804626 .................................\n",
      "[CV]  penalty=l1, C=3593.813663804626, score=0.9000183083119736, total=  15.3s\n",
      "[CV] penalty=l1, C=3593.813663804626 .................................\n",
      "[CV]  penalty=l1, C=3593.813663804626, score=0.9002746246796045, total=  14.4s\n",
      "[CV] penalty=l1, C=3593.813663804626 .................................\n",
      "[CV]  penalty=l1, C=3593.813663804626, score=0.8979311607469791, total=  14.9s\n",
      "[CV] penalty=l1, C=3593.813663804626 .................................\n",
      "[CV]  penalty=l1, C=3593.813663804626, score=0.8993225924569754, total=  14.9s\n",
      "[CV] penalty=l1, C=3593.813663804626 .................................\n",
      "[CV]  penalty=l1, C=3593.813663804626, score=0.8991724339961185, total=  14.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed: 18.9min finished\n"
     ]
    }
   ],
   "source": [
    "bestmodel1 = clf.fit(tfidf_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "colab_type": "code",
    "id": "rUzNuIn5xmh7",
    "outputId": "1b7db85a-2ee7-46c1-eb2f-1766559962dd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Penalty: l2\n",
      "Best C: 7.742636826811269\n"
     ]
    }
   ],
   "source": [
    "print('Best Penalty:', bestmodel1.best_estimator_.get_params()['penalty'])\n",
    "print('Best C:', bestmodel1.best_estimator_.get_params()['C'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "KkwdIcdnxmh_"
   },
   "outputs": [],
   "source": [
    "y_true, y_pred = y_test,clf.predict(tfidf_test)\n",
    "#predicting test data through trained model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 161
    },
    "colab_type": "code",
    "id": "ZbBxDEohxmiB",
    "outputId": "48658798-1117-4e18-a92e-923b6fd11138"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.71      0.64      0.67     14341\n",
      "          1       0.93      0.95      0.94     76692\n",
      "\n",
      "avg / total       0.90      0.90      0.90     91033\n",
      "\n",
      "(0.9023321213186427, 0.9023321213186427, 0.9023321213186426, None)\n"
     ]
    }
   ],
   "source": [
    "\n",
    "from sklearn.metrics import precision_recall_fscore_support, classification_report\n",
    "print(classification_report(y_test, y_pred))\n",
    "print(precision_recall_fscore_support(y_test, y_pred, average='micro'))\n",
    "#obtaining various performance measures on test data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oi-Mo14I54Vx"
   },
   "source": [
    "Observing Sparcity in L1,L2 Regularization\n",
    "------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "h-2ryaNbxmiL"
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "weights1 =[]#list to store weights of model \n",
    "for item in C:\n",
    "    clf = LogisticRegression(C=item, penalty='l1')\n",
    "    clf.fit(tfidf_train,y_train)#fitting the logistic regression model\n",
    "    weights1.append(np.count_nonzero(clf.coef_))#appending weights to the list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6bKMZ2CRyjt5"
   },
   "outputs": [],
   "source": [
    "weights =[]#list to store weights of model\n",
    "for item in C:\n",
    "    clf1 = LogisticRegression(C=item, penalty='l2')\n",
    "    clf1.fit(tfidf_train,y_train)#fitting the logistic regression model\n",
    "    weights.append(np.count_nonzero(clf1.coef_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 376
    },
    "colab_type": "code",
    "id": "CLrvWrrejObf",
    "outputId": "cd32c000-9683-4f38-9a23-7400f29bf268"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEWCAYAAACufwpNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcXFWd9/HPN93ppJPOBlnIBgmkVZBxEALiMsqIQogiriPoSMjAgIw+LjOOgvM8D+rovNRnVMRREQHFFXAboqCAC4oLSBgQgRDTbEkIJoGEmNCdpLvr9/xxTyWVprqruroq1dX9fb9e9eq6555b9bt1k/u755y7KCIwMzOrhjH1DsDMzEYOJxUzM6saJxUzM6saJxUzM6saJxUzM6saJxUzM6saJxUbMST9WNKyKn3WpZL+T4XL3iLpnGrEUUuSTpC0fgjLv1XSTdWMKX3uByVdXu3Ptf3DScWGTNJLJP1W0jZJWyT9RtKx+zuOiDglIq5KMZ0l6ddD+Ky3R8S/DzUmSbMlrZC0QVJIWjDUzxwuIuKbEXHSUD6jWGKLiP+IiGGflK04JxUbEkmTgR8BnwMOAOYCHwZ21eC7mqv9mftBDvgJ8IbBLjic13c4x2b15aRiQ/UsgIj4dkT0RkRXRNwUEffAnhbDbyR9LrVkHpB0Yn5hScslrZK0XdJDks4rmHeCpPWSPiDpz8BXUvlpku6W9BdJD0pakspvkXSOpMOBS4EXStoh6SlJx0raWLgzlPQGSXcXWylJX5X00T5x/IukTZIel7S8nB8nIjZGxBeAO8qpL+mRtL73AE9LapY0R9L3JG2W9LCkdxXUb5V0laSt6Xd8f+GRf2odLSq2XkW++4L0e26XdL+k1xXMy2/Hz0jaAnyosDWYvndHwatb0lfTvKLbWNJE4MfAnILl5kj6kKRvFHz3ayTdl7bjLWn7Fv5e75N0T/r3dY2k8eX81lYbTio2VH8CetOO7RRJ04rUeQHwEDAduAj4vqQD0rxNwKuBycBy4DOSji5Y9iCyFtAhwLmSjgO+BvwrMBV4KfBI4ZdFxCrg7cDvIqItIqZGxB3Ak8ArC6r+PfD1MtfzIGAKWUvsbODz/axrNZwBvIps/XLAD4E/pO8+EXiPpJNT3YuABcChZOv290P43geBvyFbzw8D35A0u2B+fjvOBD5WuGBEfDL91m3A4cBm4No0u+g2joingVOADfllI2JD4edKehbwbeA9wAzgBuCHkloKqv0dsARYCDwPOGsIv4ENkZOKDUlE/AV4CRDAl4HNaQxhVkG1TcDFEdEdEdcAq8l2mkTE9RHxYGR+CdxEtmPLywEXRcSuiOgi26FfGRE3R0QuIh6LiAfKDPcq0k43JbWTgW+VuWw38JG0DjcAO4Bnl7nsYF0SEevS+h4LzIiIj0TE7oh4iOx3Pj3V/TvgPyJia0SsBy6p9Esj4jsRsSH9rtcAa4DjCqpsiIjPRURPiu0ZJLUC/w18Nv1O5WzjgbwZuD5t727gP4FW4EUFdS5JcW8hS8BHDWK1rcqcVGzIImJVRJwVEfOAI4E5wMUFVR6Lfe9c+miqQ2rd3KZsgP8pYClZiyZvc0TsLJieT3ZEXYlvAKdKaiPbGd8aEY+XueyTEdFTMN0JtFUYRynrCt4fQtY99FT+BXwQyCftOX3qF74fFElnpm7F/Pccyb7bopzPvgJYHRGfKPjcUtt4IHPI/r0AEBG5FMfcgjp/Lnhfy+1iZXBSsapKrYavku2Q8uZKUsH0wcAGSeOA75Edfc6KiKlk3RuFdfveRnsdcFg5oRSJ7THgd8DrgLdRftfX/lYY+zrg4dSFl39Nioilaf7jwLyC+vP7fFYnMKFg+qBiXyjpELIW0DuBA9O2uJeBt0Xfz7iArPV2dkFZqW1c6jbpG8gSa/7zRLaOj5VYzurEScWGRNJz0gD2vDQ9n2xM4LaCajOBd0kaK+lNZH3uNwAtwDiy/vceSacApU5RvQJYLulESWMkzZX0nCL1NgLz+vS9QzYe837gr4AfDGplK5QGjselyXGDHEj+PfCXNHjfKqlJ0pHae8r2tcCFkqZJmkuWFArdDbwlLbcEeFk/3zORbAe/OcW8nH0PDAaUtt27gNf26RortY03AgdKmtLPR18LvCpt77HAv5CdWfjbcmOz/ctJxYZqO9kA7u2SniZLJveS/efPux1oB54gG+B9Y0Q8GRHbyXZE1wJbgbcAKwb6soj4PWmwF9gG/JKCI9kCPwfuA/4s6YmC8h+k+j9IA8X7QxfZGAzAA2m6LBHRC5xKNk7wMNlveDnZYDrAR4D1ad5Pge+y7+nc707LPwW8lWy8o9j33A98iqwlt5Es6f6m3DjJxj5mAKsKzuS6tNQ2Ti3bbwMPpW63OX3iWk02Dva5tO6nAqdGxO5BxGb7kfyQLqslSWcB50TES+odS56kB4HzIuKn9Y6l2iSdD5weEf21SMxqyi0VG1UkvYGsm+fn9Y6lGpRdsf/i1BX4bLIW4n7p1jMrxlfF2qgh6RbgCOBt6SyikaAF+BLZNRpPAVcDX6hrRDaqufvLzMyqxt1fZmZWNaOu+2v69OmxYMGCeodhZtYw7rzzziciYkY5dUddUlmwYAErV66sdxhmZg1D0qOla2Xc/WVmZlXjpGJmZlXjpGJmZlXjpGJmZlXjpGJmZlXjpGJmZlXjpGJmZlUz6q5TMTMbCXp6c3R299K1u5fO3b107u7Z876rSHlz0xjOP6Gc59sNjZOKmVkNRAQ7u3N07u55xo6+q7sn7fALyorUy5JGz7710vzdvYO7J+qMSeOcVMzMaqm7N1f0qL6zz86/K79T31N33x19Z3cvO3f30tm9b2thsPfrbR3bxISWJlpb8n+baR07hlmTx2dlY/eWT0h1xqeywvLWPWXNtKbplub9M9rhpGJmw1axo/3CnX9+uu/RfuHOv6s794yj/ay8l+7ewe31xzYp7bD37qwntDQxeXwzB00et6c8v/Mfv+d9c0GiyKbzO/98+fjmJsaMUY1+yf3HScXMhqS7N1eww+7pc+S/79F+vmxnd/Gj/cKdf1d39Y72J4xt4qDJY/eWjd33aH9vefOe6b5H+xNamhjb5HObSnFSMRvhIqJPf36Rgd0iR/X9dfXs7K7e0X7hDn1K69iiR/utLc/c0fc92s/v/MePHYPU+Ef7jcxJxWwY6Hu0X7jz7+rnaL9w57+zxMDuYEh9jvbHNu/pxpk9Ze/RfuHOv7XI0X7fnX++799H+yObk4pZGXK5YGdP7zN21sWO9jv7tAqKHe13FXT/7KzgaL+laUyRI/fsaH/25PF9unT2Pdqf0DJwV4+P9m0onFRsRMrlgg3bunh82859j+r36b7pZ2C32ABwFY728zv0qRPG7jmrp+/Rft8uodYi5a1jfbRvw5eTijW03lywdksnHZt2sGbTdjo27mDNph10bNpRMhHkj/b3OYIf28yUCS3MLjbY+4wdff9dPeOafbRvo5OTijWE3T05Hn3y6T0JY82mHazZuJ2Hnnia3T17LwI7aPJ42me1cfpx82mfOYl501qZOK5PV09qGTT7aN+s6pxUbFjZ2d3LQ5ufzlodm3awZuMOOjbv4JEnnqYnt3fcYf4BrbTPnMRLnzWDRTPbaJ/ZxmEz25g8fmwdozczJxWri6d39dBR0Oro2LSdNZt2sG5LJ/ncMUaw4MCJLJrZxsnPnUX7zEksmtnGoTMmMqHF/3TNhiP/z7Sa2tbZTcfm7VmLY9Pe8Y7HnuraU2dsk1g4fSJHzpnCa4+aS/usNtpnTmLB9AmMa26qY/RmNlhOKlYVT+7YlY1zbNpBx8bte5LHpu279tQZ1zyGRTPbOHbBNM6YOZ9FMyfRPquNgw+Y4LOZzEYIJxUrW0Sw8S+79pxplSWQ7P3Wzu499Sa2NLFoVjbe0T6zjfZZbSyaMYm501ppGgH3NjKz/jmp2DPkcsFjT3XtPU13TwtkB9t39eypN6V1LM+a1caSIw/KWh0pgRw0ebxPpzUbpZxURrGe3hzrtnaxpqC7qqPINR7T28bRPrON1x09d89ZVu0zJzG9rcXJw8z24aQyCuzuyfHIk0/vOUU33/p4aPPT+zzoZ/aU8Sya2cYZxx2cBsvbWDSzjakTWuoYvZk1EieVEWRndy8Pbt7b2sgnkEee7KQ3nacrwfxpE2if2cbL8td4zJrEYTMmMsnXeJjZEDmpNLhtXd1c8rM1/HTVRtZu6dzz7ImmMeKQA7PkccqRs2mf1cZhM7JXa4tP0zWz2nBSaVC5XPCdO9fxyZ+sZkvnbl5x+Cxe9/y5tKfTdA850Nd4mNn+56TSgO5au5UPrbiPP6zfxuJDpnHVa47jyLlT6h2WmZmTSiPZvH0Xn/zJA3znzvXMnDSOi998FKcdNcdnYJnZsOGk0gC6e3Nc9dtH+OxP17Czp5fzXnYo/+vl7bSN8+Yzs+Gl5vfGkNQk6S5JP0rTCyXdLmmNpGsktaTycWm6I81fUPAZF6by1ZJOLihfkso6JF1Q63Wph990PMHSz97KR69fxdGHTOMn73kpF55yuBOKmQ1L++OGS+8GVhVMfwL4TES0A1uBs1P52cDWiFgEfCbVQ9IRwOnAc4ElwBdSomoCPg+cAhwBnJHqjgjrt3Zy/jfu5K2X386unhxfPnMxX11+LIfNaKt3aGZm/appUpE0D3gVcHmaFvBy4LupylXAa9P709I0af6Jqf5pwNURsSsiHgY6gOPSqyMiHoqI3cDVqW5D29ndy8U//RMnfuqX/GL1Jt530rO46b0v5ZVHzPLYiZkNe7XuQ7kYeD8wKU0fCDwVEfkbSK0H5qb3c4F1ABHRI2lbqj8XuK3gMwuXWden/AXFgpB0LnAuwMEHHzyE1amdiODG+zby0evvZ/3WLl71vNn829LDmTO1td6hmZmVrWZJRdKrgU0RcaekE/LFRapGiXn9lRdrZUWRMiLiMuAygMWLFxetU08dm3bw4R/ex61rnuDZsybxrX98AS86bHq9wzIzG7RatlReDLxG0lJgPDCZrOUyVVJzaq3MAzak+uuB+cB6Sc3AFGBLQXle4TL9lTeE7Tuzq+G/8ptHaG1p4qJTj+Btxx/iZ6ebWcOqWVKJiAuBCwFSS+V9EfFWSd8B3kg2BrIMuC4tsiJN/y7N/3lEhKQVwLckfRqYA7QDvydrwbRLWgg8RjaY/5ZarU815XLBD+56jI//5AGe2LGLNy+ez/tOfjbT28bVOzQzsyGpx3mpHwCulvRR4C7gilR+BfB1SR1kLZTTASLiPknXAvcDPcA7IqIXQNI7gRuBJuDKiLhvv65JBf64fhsXrbiX/1n7FEfNn8rlZy7mr+dPrXdYZmZVoYhhN8RQU4sXL46VK1fu9+99cscu/vOm1Vx9xzoOnNjCB5Y8hzccPY8xfhKimQ1zku6MiMXl1PUVdDXW05vjm7ev5VM3raZzdy//8OKFvPsV7Uz2bebNbARyUqmhiGD5V+/g1jVP8JJF07no1CNonzWp9IJmZg3KSaWGftPxJLeueYILTnkO5730UF+8aGYjns9draEv3NLBrMnjWP7iBU4oZjYqOKnUyF1rt/LbB5/kH//mUD8sy8xGDSeVGvnCLQ8ydcJYzjhueN4WxsysFpxUauBPG7dz8/0bWfbCBUz0LerNbBRxUqmBS295kAktTZz1ogX1DsXMbL9yUqmydVs6ue4PG3jLcQczbWJLvcMxM9uvnFSq7LJfPcQYwTl/c2i9QzEz2++cVKpo0/adXLNyHW84eh4HTRlf73DMzPY7J5UquvLXj9DTm+O8lx1W71DMzOrCSaVKtnV1843bHmXpX81m4fSJ9Q7HzKwunFSq5Bu3PcqOXT2cf4JbKWY2ejmpVEHX7l6u/PXDnPDsGTx3zpR6h2NmVjdOKlVwzR1refLp3fzTCYvqHYqZWV05qQxRd2+OL9/6MMcumMZxCw+odzhmZnXlpDJE1929gcee6nIrxcwMJ5UhyeWCL97SweGzJ3PCs2fUOxwzs7pzUhmCm+7/Mw9ufprzTzjMz0sxM8NJZUi+9KuHOOTACSw98qB6h2JmNiw4qVSopzfHPeu3sfSvZtPc5J/RzAwGmVQkvU5SW62CaSSPb9tJby5YcOCEeodiZjZslJ1UJB0GXAv8fe3CaRzrtnQCMH+ak4qZWd5gWir/AHwi/R311uaTygFOKmZmeWUlFUlNwJvIkso2SX9d06gawNotnTSPEbN9i3szsz3KbaksBX4bEduBK4FzahdSY1i7pZO501o9SG9mVqDcPeLZwBXp/Q+ApZJG9bNy123t4mB3fZmZ7aNkUpE0FZgaEbcCRMRO4LvAy2sc27C2bkunx1PMzPpoLlUhIp4CTuhT9oFaBdQItu/sZsvTu33ml5lZH4MeEJD0oRrE0VDWbekCcPeXmVkflYwyv6bqUTSY/OnETipmZvuqJKmM+jsnrnNSMTMrqpKkckzVo2gw67Z2Mnl8M1MmjK13KGZmw0olSWVl1aNoMGt95peZWVHu/qrA2i2d7voyMyuikqRyfdWjaCC5XLB+iy98NDMrppKkcls5lSSNl/R7SX+QdJ+kD6fyhZJul7RG0jX5K/MljUvTHWn+goLPujCVr5Z0ckH5klTWIemCCtZl0DZu38nu3py7v8zMiqgkqXykzHq7gJdHxF8DRwFLJB1PdlPKz0REO7CV7BYwpL9bI2IR8JlUD0lHAKcDzwWWAF+Q1JRucvl54BTgCOCMVLemfI2KmVn/ajamEpkdaXJsegXZ7V2+m8qvAl6b3p+WpknzT1T24PfTgKsjYldEPAx0AMelV0dEPBQRu4GrU92a8jUqZmb9qySpnFduxdSiuBvYBNwMPAg8FRE9qcp6YG56PxdYB5DmbwMOLCzvs0x/5TW1dksnEsyZ2lrrrzIzaziVJJWyb3sfEb0RcRQwj6xlcXixaulvsRZQVFD+DJLOlbRS0srNmzeXDnwA67Z0MmdKKy3NvuW9mVlflewZFw92gXRTyluA44GpkvI3spwHbEjv1wPzAdL8KcCWwvI+y/RXXuz7L4uIxRGxeMaMGYMNfx/ZNSpupZiZFVNJUtlUTiVJM9Jt85HUCrwCWAX8AnhjqrYMuC69X5GmSfN/HhGRyk9PZ4ctBNqB3wN3AO3pbLIWssH8FRWsz6D4GhUzs/6VvPV9EWeVWW82cFU6S2sMcG1E/EjS/cDVkj4K3MXeh39dAXxdUgdZC+V0gIi4T9K1wP1AD/COiOgFkPRO4EagCbgyIu6rYH3K1rW7l83bdzmpmJn1o5KkcgNwdKlKEXEP8Pwi5Q+Rja/0Ld8JvKmfz/oY8LEi5TekePaLDduy04nnTnP3l5lZMb5NyyB07uoFYNI430jSzKyYSpLKl6seRYPo6s6SSmtLU50jMTMbnipJKj2lq4xM+aQyfqyTiplZMZUklbdXPYoG0bU7tVScVMzMivKYyiDsdPeXmdmAKkkqp1Y9igaxZ0zFLRUzs6IqSSqXVj2KBuHuLzOzgVWSVGp+08bhas9AfYvv+2VmVkwle8e7qh5Fg9jZ3csYQUuTk4qZWTGV7B3/q+pRNIiu3b20jm0ie8yLmZn1VUlSubzqUTSIru5en/llZjYAn1I8CF3dvb7w0cxsAJUklQ9XPYoGsbO712d+mZkNoJKkclTVo2gQXbvd/WVmNpBKksprqh5Fg3D3l5nZwDymMghd3Tl3f5mZDaCSpHJM1aNoEDt3e0zFzGwglSSVlVWPokH4lGIzs4G5+2sQPKZiZjawSpLK9VWPokG4+8vMbGCVJJXbqh5Fg8i6v3zfLzOz/lSyh/xI1aNoAN29OXpy4ZaKmdkAPKZSJj+f3systEqSynlVj6IB7NztRwmbmZVSSVI5p+pRNAA/StjMrLRKksriqkfRAJxUzMxKqySpbKp6FA0g/3z68e7+MjPrVyVJ5axqB9EI3FIxMyutkqRyQ9WjaAA7nVTMzEryKcVl6tqdA2DcWF/8aGbWn0r2kF+uehQNoCeXJZXmMU4qZmb9KWsPKekwSePS5P2S3iVpag3jGnZyEQA0jRmVDTUzs7KUe9j9PaBX0iLgCmAh8K2aRTUM9WYNFZrkpGJm1p9yk0ouInqA1wEXR8R7gdm1C2v4yeWylop7v8zM+lfuLrJb0hnAMuBHqWxsbUIannrd/WVmVlK5SWU58ELgYxHxsKSFwDdqF9bw05taKu7+MjPrX3OZ9V4ZEe/KT6TE0lWjmIal/ED9GLdUzMz6VW5LZVmRsrOqGMew55aKmVlpAyYVSWdI+iGwUNKKgtcvgCdLLDtf0i8krZJ0n6R3p/IDJN0saU36Oy2VS9Ilkjok3SPp6ILPWpbqr5G0rKD8GEl/TMtcItVuj9+bc0vFzKyUUt1fvwUeB6YDnyoo3w7cU2LZHuBfIuJ/JE0C7pR0M1kL52cR8XFJFwAXAB8ATgHa0+sFwBeBF0g6ALiI7O7IkT5nRURsTXXOJXvE8Q3AEuDH5az4YPk6FTOz0gZMKhHxKPAo2SD9oETE42QJiYjYLmkVMBc4DTghVbsKuIUsqZwGfC0iArhN0lRJs1PdmyNiC0BKTEsk3QJMjojfpfKvAa+lRknF16mYmZVW7hX1r09dT9sk/UXSdkl/KfdLJC0Ang/cDsxKCSefeGamanOBdQWLrU9lA5WvL1Je7PvPlbRS0srNmzeXG/Y+9g7UV7S4mdmoUO4u8pPAayJiSkRMjohJETG5nAUltZFdkf+eiBgoERVrAkQF5c8sjLgsIhZHxOIZM2aUCrmonl4P1JuZlVJuUtkYEasG++GSxpIllG9GxPfzn5W6tUh/8w/9Wg/ML1h8HrChRPm8IuU14YsfzcxKK3X21+slvR5YKemadDbY6wvKB1pWZPcJWxURny6YtYK9pygvA64rKD8znQV2PLAtdY/dCJwkaVo6U+wk4MY0b7uk49N3nVnwWVWXywVjBDU8wczMrOGVOvvr1IL3nWQ79LwAvk//Xgy8DfijpLtT2QeBjwPXSjobWAu8Kc27AVgKdKTvWg4QEVsk/TtwR6r3kfygPXA+8FWglWyAviaD9JC1VNxKMTMbWKmzv5ZX+sER8Wv6f6DXiUXqB/COfj7rSuDKIuUrgSMrjXEwspaKk4qZ2UDKuk2LpEuKFG8DVkZEzbqchpPenFsqZmallDtQPx44CliTXs8DDgDOlnRxjWIbVnojfOaXmVkJ5d5QchHw8vRMFSR9EbgJeCXwxxrFNqzkcuFbtJiZlVBuS2UuMLFgeiIwJyJ6gV1Vj2oY8kC9mVlp5bZUPgncnW6NIuClwH9Imgj8tEaxDSu9OTxQb2ZWQllJJSKukHQDcBxZUvlgROQvNPzXWgU3nORyQZNv0WJmNqBSFz8+J/09muyZ9OvIri05qPDW9KOBB+rNzEor1VL5Z7Jby3+qyLwAXl71iIYpD9SbmZVW6uLHc9Pfv90/4QxfHqg3Myut3FvfT5D0vyVdlqbbJb26tqENL705d3+ZmZVS7tDzV4DdwIvS9HrgozWJaJjKhbu/zMxKKTepHBYRnwS6ASKii/7v6zUiuaViZlZauUllt6RW0kOwJB3GKLnoMa83h1sqZmYllHvx40XAT4D5kr5Jdlv7s2oV1HDUm8vR7KRiZjagcpPKmcD1wHeBh4B3R8QTNYtqGOoNt1TMzEopN6l8BXgJ2Q0kDyW7ZcuvIuKzNYtsmMnlgibnFDOzAZV7m5afS/olcCzwt8DbgecCoyap+HkqZmallfuQrp+R3Zn4d8CtwLERsamWgQ03uQg/n97MrIRyz/66h+w6lSPJHtB1ZDobbNSIADdUzMwGVm7313sBJLUBy8nGWA4CxtUutOElCFR2DjYzG53K7f56J/A3wDHAo8CVZN1go0YEuPfLzGxg5Z791Qp8Grgz/0jh0SZwUjEzK6Xc7q//V+tAhruIQKPrzjRmZoPmQYIyuaViZlaak0qZcoFPKTYzK8FJpVwR7vwyMyvBSaVM7v4yMyvNSaVMEaPsATJmZhVwUilT4Nu0mJmV4qRSJrdUzMxKc1Ipk6+oNzMrzUmlTAG4rWJmNjAnlTJFhFsqZmYlOKmUybe+NzMrzUmlTNmt751VzMwG4qRSJg/Um5mV5qRSJl9Rb2ZWWs2SiqQrJW2SdG9B2QGSbpa0Jv2dlsol6RJJHZLukXR0wTLLUv01kpYVlB8j6Y9pmUtU4ysTfet7M7PSatlS+SqwpE/ZBcDPIqId+FmaBjgFaE+vc4EvQpaEgIuAFwDHARflE1Gqc27Bcn2/q6oCfEaxmVkJNUsqEfErYEuf4tOAq9L7q4DXFpR/LTK3AVMlzQZOBm6OiC0RsRW4GViS5k2OiN9FRABfK/isGq2Qc4qZWSn7e0xlVkQ8DpD+zkzlc4F1BfXWp7KBytcXKS9K0rmSVkpauXnz5ooCz8ZUnFbMzAYyXAbqi+2to4LyoiLisohYHBGLZ8yYUVGA4eepmJmVtL+TysbUdUX6uymVrwfmF9SbB2woUT6vSHnNBL740cyslP2dVFYA+TO4lgHXFZSfmc4COx7YlrrHbgROkjQtDdCfBNyY5m2XdHw66+vMgs+qiVz41vdmZqU01+qDJX0bOAGYLmk92VlcHweulXQ2sBZ4U6p+A7AU6AA6geUAEbFF0r8Dd6R6H4mI/OD/+WRnmLUCP06vmvGt783MSqtZUomIM/qZdWKRugG8o5/PuRK4skj5SuDIocQ4GNHfSI6Zme0xXAbqG4IvfjQzG5iTSpl863szs9KcVMrk3i8zs9KcVMrkuxSbmZXmpFImP0/FzKw0J5UyuaViZlaak0qZ/DwVM7PSnFTKlLVUnFXMzAbipFK2fu9XaWZmiZNKmXybFjOz0pxUBsG9X2ZmA3NSKZM7v8zMSnNSKVP2kC43VczMBuKkUiafUmxmVpqTipmZVY2TSpl89peZWWlOKmUKP07YzKwkJxUzM6saJ5Uy+ZRiM7PSnFTK5bsUm5mV5KQyCL5OxcxsYE4qZXL3l5lZaU4qZcrO/qp3FGZmw5uTSpkCX6diZlaKk8oguKViZjYwJ5UyhQdVzMxKclIp05IjD+Lw2ZPrHYaZ2bDWXO8AGsVn3nwqT0fkAAAGXklEQVRUvUMwMxv23FIxM7OqcVIxM7OqcVIxM7OqcVIxM7OqcVIxM7OqcVIxM7OqcVIxM7OqcVIxM7OqUYyy+49I2gw8WuHi04EnqhhOI/A6j3yjbX3B6zxYh0TEjHIqjrqkMhSSVkbE4nrHsT95nUe+0ba+4HWuJXd/mZlZ1TipmJlZ1TipDM5l9Q6gDrzOI99oW1/wOteMx1TMzKxq3FIxM7OqcVIxM7OqcVIpg6QlklZL6pB0Qb3jGQpJ8yX9QtIqSfdJencqP0DSzZLWpL/TUrkkXZLW/R5JRxd81rJUf42kZfVap3JIapJ0l6QfpemFkm5PsV8jqSWVj0vTHWn+goLPuDCVr5Z0cn3WpHySpkr6rqQH0vZ+4UjezpLem/5N3yvp25LGj8TtLOlKSZsk3VtQVrXtKukYSX9My1wiSYMKMCL8GuAFNAEPAocCLcAfgCPqHdcQ1mc2cHR6Pwn4E3AE8EngglR+AfCJ9H4p8GNAwPHA7an8AOCh9Hdaej+t3us3wHr/M/At4Edp+lrg9PT+UuD89P6fgEvT+9OBa9L7I9K2HwcsTP8mmuq9XiXW+SrgnPS+BZg6UrczMBd4GGgt2L5njcTtDLwUOBq4t6CsatsV+D3wwrTMj4FTBhVfvX+g4f5KP+6NBdMXAhfWO64qrt91wCuB1cDsVDYbWJ3efwk4o6D+6jT/DOBLBeX71BtOL2Ae8DPg5cCP0n+WJ4DmvtsYuBF4YXrfnOqp73YvrDccX8DktJNVn/IRuZ1TUlmXdpLNaTufPFK3M7CgT1KpynZN8x4oKN+nXjkvd3+Vlv/Hmrc+lTW81OR/PnA7MCsiHgdIf2emav2tfyP9LhcD7wdyafpA4KmI6EnThbHvWa80f1uq30jrC1nLejPwldTtd7mkiYzQ7RwRjwH/CawFHifbbncy8rdzXrW269z0vm952ZxUSivWn9jw52FLagO+B7wnIv4yUNUiZTFA+bAi6dXApoi4s7C4SNUoMa8h1rdAM1kXyRcj4vnA02TdIv1p6PVOYwinkXVZzQEmAqcUqTrStnMpg13PIa+/k0pp64H5BdPzgA11iqUqJI0lSyjfjIjvp+KNkman+bOBTam8v/VvlN/lxcBrJD0CXE3WBXYxMFVSc6pTGPue9UrzpwBbaJz1zVsPrI+I29P0d8mSzEjdzq8AHo6IzRHRDXwfeBEjfzvnVWu7rk/v+5aXzUmltDuA9nQWSQvZoN6KOsdUsXQmxxXAqoj4dMGsFUD+DJBlZGMt+fIz01kkxwPbUvP6RuAkSdPSUeJJqWxYiYgLI2JeRCwg23Y/j4i3Ar8A3piq9V3f/O/wxlQ/Uvnp6ayhhUA72YDmsBQRfwbWSXp2KjoRuJ8Rup3Jur2OlzQh/RvPr++I3s4FqrJd07ztko5Pv+OZBZ9VnnoPODXCi+wMij+RnQnyb/WOZ4jr8hKy5uw9wN3ptZSsP/lnwJr094BUX8Dn07r/EVhc8Fn/AHSk1/J6r1sZ634Ce8/+OpRsZ9EBfAcYl8rHp+mONP/QguX/Lf0OqxnkGTF1Wt+jgJVpW/832Vk+I3Y7Ax8GHgDuBb5OdgbXiNvOwLfJxo26yVoWZ1dzuwKL02/4IPBf9DnZo9TLt2kxM7OqcfeXmZlVjZOKmZlVjZOKmZlVjZOKmZlVjZOKmZlVjZOKWQFJB0m6WtKDku6XdIOkZ9U7LrNG4aRilqSLvX4A3BIRh0XEEcAHgVn1jax6JE2R5P/3VjP+x2W2198C3RFxab4gIu6OiFvrGFO1vQRYLelDkg6udzA28jipmO11JNmdbUesiLie7BbwTwHXSbpR0pvyD68yGypfUW+WSHoXsDAi3lvvWPYXSS8EriRroT2v3vFY42suXcVs1LiPvTcf3EPSx4BXpclj2NuaWQHcBVyUps8B3kH2jJoNwHnAD9O8S8meIvqPaXop8BWy8ZqVwJfJHpQE8H+BF9TiOyNiQ1qnI4DlwOuAXwKX9f+zmJXPLRWzJA3U3wZcHhFfTmXHAhMi4pd1Da5K0jPKv0D2wLIryB6ju6O+UdlI4qRiVkDSHLLnrRwD7AQeIXuQ2Zp6xlUtkg4HiIhV9Y7FRiYnFTMzqxqf/WVmZlXjpGJmZlXjpGJmZlXjpGJmZlXjpGJmZlXjpGJmZlXjpGJmZlXz/wEmWD/6C84PsQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#plotting sparsity of l1 regularization using weights (vs) C\n",
    "                                              ---------------      \n",
    "plt.plot(C,weights1)\n",
    "ymin, ymax = plt.ylim()\n",
    "plt.xlabel('C-------------->')\n",
    "plt.ylabel('weights------------->')\n",
    "plt.title('Sparcity in L1 regularization')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "aB6YQqW-krKg"
   },
   "source": [
    "From the above plot we can clearly see that L1 regularization  creates sparsity\n",
    "---------\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 376
    },
    "colab_type": "code",
    "id": "_6HroVIYy1yn",
    "outputId": "bf7ba60e-c5a0-417d-f6e0-4f22e8a818b8"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEWCAYAAACufwpNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XucXVVh9vHfIyEIaAyXcEtiAU1BsAXCyOWtUhWFQJV4wb5gW0KEBi1gX20rUPsK3vqpvSFYDOVihBYJvFFL5Bap4qWWBCaCQEiQcM1wHYQADUq4PO8few0cJpOZM4d9cpiZ5/v5nM/svfbae9Y6OznP7L322Vu2iYiIqMNrOt2AiIgYPRIqERFRm4RKRETUJqESERG1SahERERtEioREVGbhEqMeJKukjSrpm2dLen/1rGtkUKSJb25xXXfKOl/JG1Uc5veIen2OrcZG0ZCJVom6e2S/lvSE5Iek/QzSW/b0O2wfYjtC0qbjpb0X69gWx+3/cVW1pX0I0nHDlD+25Iuk9Rb3qdFknZptY2vJrbvs/0628+/ku30DzbbP7U9Kt6jsSahEi2RNAG4HPgasCUwGfg88Ewbfte4ure5gU0EFgK7ANsC1wOXNbty3UcBdRkF+yXaIKESrfptANsX237e9q9tf9/2zfDiEcPPJH2tHMmskHRg38qSZktaLukpSXdJOq5h2Tsl9Ug6SdJDwLxSPlPSTZKelHSnpBml/EeSjpX0FuBsYP9ySma1pLdJerjxA1DShyXdNFCnJH1T0pf6teMvJD0i6UFJs4f7Rtm+3vb5th+z/SxwOrCLpK0GacNcSVdKWgO8S9Imkv5R0n2lP2dL2rRhnc+U9j1Q3osX//LvfwQ12NGcpD+QdGN5j1dJOq1h2Y5lu8dIug/4YUPZOEl973vf6zeS7inr7iPpurJPHpT0L5LGl2U/Kb/iF2W9/9333jf87reUfqyWtEzSYf3er7MkXVH+PS2R9KZh7qaoSUIlWvVL4HlJF0g6RNIWA9TZF7gL2Bo4FfiOpC3LskeA9wETgNnA6ZKmN6y7HdUR0G8BcyTtA1wI/BXVX/4HAPc0/jLby4GPA9eVUzITbd8A/Ap4b0PVPwb+rcl+bge8gepI7BjgrPX0dTgOAB6y/atB6nwU+DLweuC/gK9QBfmewJtLez4HUML108B7yrLffwVtWwMcRfUe/wHwCUkf6Ffn94G3AAc3Ftrue99fB2wBLAYuLoufBz5F9W9hf+BA4M/KegeUOnuU9S9p3K6kjYHvAd8HtgFOBC7qdwrxSKoj5S2AlVTvXXRAQiVaYvtJ4O2AgXOBXkkLJW3bUO0R4Ku2ny0fFLdTfVBh+wrbd7ryY6oPjHc0rPsCcKrtZ2z/muoD/Ru2r7H9gu37ba9osrkXUAUJJdQOBr7V5LrPAl8ofbgS+B+q01gtkTQFOIsqBAZzme2f2X6B6pTinwKfKkc7TwF/CxxR6v4hMM/2MttPU324tsT2j2zfUt7jm6lCoX9InWZ7Tdkv63MmVUB9tmx3qe3Ftp+zfQ/wrwNsd332A14H/J3ttbZ/SHXq9ciGOt8pR4TPARdRhW90QEIlWmZ7ue2jbU8B3grsAHy1ocr9fvkdS+8tdShHN4vLwPVq4FCqv2L79Nr+TcP8VODOFpv678D7Jb2O6gP4p7YfbHLdX5UPqj5PU33ADZukSVTh+XXbFw9RfVXD9CRgM2BpOf2zGri6lEP1nq5az7rDbeO+kq5VdVHBE1RHflv3qzbo9supzHcCHy2h2HexwuWSHpL0JFUo9t/u+uwArOrbVnEv1dFan4caplveR/HKJVSiFuWo4ZtU4dJnsiQ1zL8ReEDSJsC3gX8EtrU9EbgSaKzb//bZq4BmzpOvc9tt2/cD1wEfBP6E5k991aacMvs+sNB2M6dmGvvxKPBrYPdySm+i7TeU00wADwJTGupP7betNVSh1Ge7QX7vt6guKphq+w1UY1TqV2e9tzaX9A7gi8BM2080LJoLrACm2Z4A/PUA212fB4Cpkho/r94I3N/k+rEBJVSiJZJ2LQPYU8r8VKrTEYsbqm0DfFLSxpI+QnUe/kpgPLAJ0As8J+kQ4KAhfuX5wGxJB0p6jaTJknYdoN7DwJS+QeAGFwKfAX4H+O6wOjs84yS9tuG1saor5RYBP7N98nA3WP5CP5dq3GkbgNL/vjGNS6nem7dI2owy1tLgJuBDkjYrg/fHDPLrXg88Zvs3ZRzro822s/wbuAQ4yvYvB9juk8D/lP32iX7LHwZ2Xs+ml1AF42fK+/lO4P3A/GbbFhtOQiVa9RTVQPyScoXSYuBW4C8a6iwBplH9pf1l4HDbvypjAp+k+jB8nOqDa+Fgv8z29ZQBfeAJ4MdUg/j9/RBYBjwk6dGG8u+W+t+1vWZ4XR2WuVRHFX2veVRHSG+j+uBvvDrqjcPY7klUA9CLy+mj/6SM7di+imoM49pS57qyTt/l3acDa6k+uC+gGnNYnz8DviDpKapwunQYbTyQ6ihoQUMfl5Vlf0m1n5+iCshL+q17GnBBOb33h40LbK8FDgMOofq39HWq4Gp2TC02IOUhXdEOko4GjrX99k63pY+kO4HjbP9np9vSTqourb4V2KTfeFBE2+VIJcYESR+mGgv4Yafb0g6SPihpfBm7+QrwvQRKdEJCJUY9ST+iOi11fL8riEaT46jGqO6k+k5I/zGLiA0ip78iIqI2OVKJiIjajLkbwm299dbecccdO92MiIgRZenSpY/anjRUvTEXKjvuuCPd3d2dbkZExIgi6d5m6uX0V0RE1CahEhERtUmoREREbRIqERFRm4RKRETUJqESERG1SahERERtEioREVGbhEpERNQmoRIREbVpa6hImihpgaQVkpZL2l/SaZLul3RTeR3aUP8USSsl3d7wqFQkzShlKyWd3FC+k6Qlku6QdMkAj5CNiIgNqN1HKmcAV9veFdgDWF7KT7e9Z3ldCSBpN+AIYHdgBvB1SRtJ2gg4i+pRorsBR5a6UD2M6HTb06geSzvYs7cjIqLN2hYqkiYABwDnQ/WcadurB1llJjDf9jO276Z61vY+5bXS9l3lWdXzgZmSBLwbWFDWvwD4QHt6ExERzWjnkcrOVE+imyfpRknnSdq8LDtB0s2SvlEefwowGVjVsH5PKVtf+VbA6oZHpvaVR0REh7QzVMYB04G5tvcC1gAnUz3W9U3AnsCDwD+V+hpgG26hfB2S5kjqltTd29s7rE5ERETz2hkqPUCP7SVlfgEw3fbDtp8vzwo/l+r0Vl/9qQ3rTwEeGKT8UWCipHH9ytdh+xzbXba7Jk0a8hkzERHRoraFiu2HgFWSdilFBwK3Sdq+odoHgVvL9ELgCEmbSNoJmAZcD9wATCtXeo2nGsxfaNvAtcDhZf1ZwGXt6k9ERAyt3U9+PBG4qITBXcBs4ExJe1KdqroHOA7A9jJJlwK3Ac8Bx9t+HkDSCcAiYCPgG7aXle2fBMyX9CXgRspFARER0Rmq/uAfO7q6upzHCUdEDI+kpba7hqqXb9RHRERtEioREVGbhEpERNQmoRIREbVJqERERG0SKhERUZuESkRE1CahEhERtUmoREREbRIqERFRm4RKRETUJqESERG1SahERERtEioREVGbhEpERNQmoRIREbVJqERERG0SKhERUZuESkRE1CahEhERtUmoREREbRIqERFRm4RKRETUpq2hImmipAWSVkhaLmn/hmV/KcmSti7zknSmpJWSbpY0vaHuLEl3lNeshvK9Jd1S1jlTktrZn4iIGFy7j1TOAK62vSuwB7AcQNJU4L3AfQ11DwGmldccYG6puyVwKrAvsA9wqqQtyjpzS92+9Wa0uT8RETGItoWKpAnAAcD5ALbX2l5dFp8OfAZwwyozgQtdWQxMlLQ9cDBwje3HbD8OXAPMKMsm2L7OtoELgQ+0qz8RETG0dh6p7Az0AvMk3SjpPEmbSzoMuN/2L/rVnwysapjvKWWDlfcMUL4OSXMkdUvq7u3tfUWdioiI9WtnqIwDpgNzbe8FrAFOAz4LfG6A+gONh7iF8nUL7XNsd9numjRpUhNNj4iIVrQzVHqAHttLyvwCqpDZCfiFpHuAKcDPJW1X6k9tWH8K8MAQ5VMGKI+IiA5pW6jYfghYJWmXUnQg8HPb29je0faOVMEwvdRdCBxVrgLbD3jC9oPAIuAgSVuUAfqDgEVl2VOS9itXfR0FXNau/kRExNDGtXn7JwIXSRoP3AXMHqTulcChwErg6b66th+T9EXghlLvC7YfK9OfAL4JbApcVV4REdEhqi6cGju6urrc3d3d6WZERIwokpba7hqqXr5RHxERtUmoREREbRIqERFRm4RKRETUJqESERG1SahERERtEioREVGbhEpERNQmoRIREbVJqERERG0SKhERUZumQ0XSZpJWSzqwnQ2KiIiRazhHKn8ILAOObVNbIiJihBtOqHyMKlD2LM81iYiIeJmmQkXSrsBrbC8HLgb+uK2tioiIEanZI5VjgHll+psM/rCtiIgYo4YMFUkbAx8G5gPYvg/4laQhH9YSERFjSzOPE94Y+JDtNQ1lxwLPt6dJERExUg0ZKrafBm7qm5c03fbP29qqiIgYkVr58uN5tbciIiJGhVZCRbW3IiIiRoVWQuXztbciIiJGhVZCZc/aWxEREaNCK6FyWLMVJU2UtEDSCknLJe0v6YuSbpZ0k6TvS9qh1JWkMyWtLMunN2xnlqQ7ymtWQ/nekm4p65wpKafmIiI6qN1jKmcAV9veFdgDWA78g+3ftb0ncDnwuVL3EGBaec0B5gJI2hI4FdgX2Ac4teE2MXNL3b71ZrTQn4iIqEkrobJ3M5UkTQAOAM4HsL3W9mrbTzZU2xxwmZ4JXOjKYmCipO2Bg4FrbD9m+3HgGmBGWTbB9nW2DVwIfKCF/kRERE1aCZXuJuvtDPQC8yTdKOk8SZsDSPqypFXAH/HSkcpkYFXD+j2lbLDyngHK1yFpjqRuSd29vb1NNj8iIoarnae/xgHTgbm29wLWACcD2P6s7anARcAJg2zXLZSvW2ifY7vLdtekSZOabH5ERAxXK6FyRZP1eoAe20vK/AKqkGn0Lar7ivXVn9qwbArwwBDlUwYoj4iIDmklVBY3U8n2Q8AqSbuUogOB2yRNa6h2GLCiTC8EjipXge0HPGH7QWARcJCkLcoA/UHAorLsKUn7lau+jgIua6E/ERFRk2ZuKNnfF6iu2mrGicBFksYDd1HdMv+8EjQvAPcCHy91rwQOBVYCT5e62H5M0heBG/p+v+3HyvQnqG7FvylwVXlFRESHqLpwahgrSDeWMZIRqaury93dzV5rEBERAJKW2h7ykSetnP46roV1IiJiDGglVI6tvRURETEqtBIqeeJjREQMqJVQeaT2VkRExKjQSqgcXXcjIiJidGglVK6svRURETEq5MmPERFRm1ZC5dzaWxEREaNCK6HyXO2tiIiIUaGVUPn40FUiImIsyphKRETUppVQeX/trYiIiFGhlVA5u/ZWRETEqNBKqAz4yN6IiIhWQuXG2lsRERGjQiuh8i+1tyIiIkaFVkLlvNpbERERo0IuKY6IiNq0Eiqfr70VERExKrQSKnvW3oqIiBgVWgmVw2pvRUREjAoZU4mIiNq0Eip7196KiIgYFca1sE43ML2ZipImUl2C/FbAwMeAD1HdP2wtcCcw2/bqUv8U4BjgeeCTtheV8hnAGcBGwHm2/66U7wTMB7YEfg78ie21LfRpSJ//3jJue+DJdmw6IqLtdtthAqe+f/e2/552n/46A7ja9q7AHsBy4BrgrbZ/F/glcAqApN2AI4DdgRnA1yVtJGkj4CzgEGA34MhSF+ArwOm2pwGPUwVSRER0SCtHKlc0U0nSBOAA4GiAcgSxFvh+Q7XFwOFleiYw3/YzwN2SVgL7lGUrbd9VtjsfmClpOfBu4KOlzgXAacDcFvo0pA2R8BERI10rRyqLm6y3M9ALzJN0o6TzJG3er87HgKvK9GRgVcOynlK2vvKtgNW2n+tXvg5JcyR1S+ru7e1tsvkRETFcrYTKF5qsN45q7GWu7b2ANcDJfQslfZbq0cQX9RUNsA23UL5uoX2O7S7bXZMmTWqy+RERMVztHFPpAXpsLynzCygD/JJmAe8D/si2G+pPbVh/CvDAIOWPAhMljetXHhERHdJKqBzXTCXbDwGrJO1Sig4EbitXcp0EHGb76YZVFgJHSNqkXNU1DbgeuAGYJmknSeOpBvMXljC6lpfGZGYBl7XQn4iIqEkrA/XHUn3YN+NE4KISBncBs6lCYhPgGkkAi21/3PYySZcCt1GdFjve9vMAkk4AFlFdUvwN28vK9k8C5kv6EtVzXs5voT8REVETvXT2qckVpJ/bbup7Kq9GXV1d7u7u7nQzIiJGFElLbXcNVa+V01+PtLBORESMAa2EytF1NyIiIkaHVkLlytpbERERo0LuUhwREbVpJVTOrb0VERExKjQVKpLeJGmTMnubpE+WOxBHRES8qNkjlW8Dz0t6M9V3QXYCvtW2VkVExIjUbKi8UG7c+EHgq7Y/BWzfvmZFRMRI1GyoPCvpSKpboVxeyjZuT5MiImKkajZUZgP7A1+2fXe5N9e/t69ZERExEjV776/32v5k30wJll+3qU0RETFCNXukMmuAsqNrbEdERIwCgx6plHGUjwI7SVrYsOj1wK/a2bCIiBh5hjr99d/Ag8DWwD81lD8F3NyuRkVExMg0aKjYvhe4l2qQPiIiYlDNfqP+Q5LukPSEpCclPSXpyXY3LiIiRpZmr/76e+D9tpe3szERETGyNXv118MJlIiIGMpQV399qEx2S7oE+A/gmb7ltr/TxrZFRMQIM9Tpr/c3TD8NHNQwbyChEhERLxrq6q/ZG6ohEREx8jU1UC/pzAGKnwC6bV9Wb5MiImKkanag/rXAnsAd5fW7wJbAMZK+2qa2RUTECNNsqLwZeLftr9n+GvAe4C1Uz1c5aH0rSZooaYGkFZKWS9pf0kckLZP0gqSufvVPkbRS0u2SDm4on1HKVko6uaF8J0lLyndoLpE0fjidj4iIejUbKpOBzRvmNwd2sP08DVeDDeAM4GrbuwJ7AMuBW4EPAT9prChpN+AIYHdgBvB1SRtJ2gg4CzgE2A04stQF+Apwuu1pwOPAMU32JyIi2mA4X368SdKPAAEHAH8raXPgPwdaQdKEUu9oANtrgbXA6rK8/yozgfm2nwHulrQS2KcsW2n7rrLefGCmpOXAu6lueAlwAXAaMLfJPkVERM2aChXb50u6kupDXsBf236gLP6r9ay2M9ALzJO0B7AU+HPba9ZTfzKwuGG+p5QBrOpXvi+wFbC6POa4f/2XkTQHmAPwxje+cT2/PiIiXqlBT39J2rX8nE71TPpVwH3AdqVsMOOA6cBc23sBa4CTB6m/zqEL1Xdhhlu+bqF9ju0u212TJk0avNUREdGyoY5UPk31F/4/DbDMVKef1qcH6LG9pMwvYPBQ6QGmNsxPAfqOhgYqfxSYKGlcOVpprB8RER0w1Jcf55Sf7xruhm0/JGmVpF1s3w4cCNw2yCoLgW9J+mdgB2AacD3VEck0STsB91MN5n/UtiVdCxwOzKd6OmW+MxMR0UHN3vp+M0l/I+mcMj9N0vuaWPVE4CJJN1N9z+VvJX1QUg/VM1qukLQIwPYy4FKq4LkaON728+Uo5ARgEdXVY5eWugAnAZ8ug/pbAec31+2IiGgH2QMOQ7y8UnUzyaXAUbbfKmlT4Drbe7a7gXXr6upyd3d3p5sRETGiSFpqu2uoes1+T+VNtv8eeBbA9q8ZeKA8IiLGsGZDZW05OjGApDcx+JceIyJiDGr2y4+nUo1zTJV0EfB7lC81RkRE9Gk2VI4CrqC6LPguqi8xPtq2VkVExIjUbKjMA94OvJfqm/I3SfqJ7TPa1rKIiBhxmr1Nyw8l/Rh4G/Au4ONUN35MqERExIuafUjXD6juTHwd8FPgbbYfaWfDIiJi5Gn26q+bqe4w/FaqB3T1fVclIiLiRc2e/voUgKTXAbOpxli2AzZpX9MiImKkafb01wnAO4C9gXuBb1CdBouIiHhRs1d/bQr8M7C04fklERERL9Ps6a9/aHdDIiJi5Gt2oD4iImJICZWIiKhNQiUiImqTUImIiNokVCIiojYJlYiIqE1CJSIiapNQiYiI2iRUIiKiNgmViIioTUIlIiJq09ZQkTRR0gJJKyQtl7S/pC0lXSPpjvJzi1JXks6UtFLSzZKmN2xnVql/h6RZDeV7S7qlrHOmJLWzPxERMbh2H6mcAVxte1dgD2A5cDLwA9vTgB+UeYBDgGnlNQeYCyBpS+BUYF9gH+DUviAqdeY0rDejzf2JiIhBtC1UJE0ADgDOB7C91vZqYCZwQal2AfCBMj0TuNCVxcBESdsDBwPX2H7M9uPANcCMsmyC7etsG7iwYVsREdEB7TxS2RnoBeZJulHSeZI2B7a1/SBA+blNqT8ZWNWwfk8pG6y8Z4DydUiaI6lbUndvb+8r71lERAyonaEyDpgOzLW9F7CGl051DWSg8RC3UL5uoX2O7S7bXZMmTRq81RER0bJ2hkoP0GN7SZlfQBUyD5dTV5SfjzTUn9qw/hTggSHKpwxQHhERHdK2ULH9ELBK0i6l6EDgNmAh0HcF1yzgsjK9EDiqXAW2H/BEOT22CDhI0hZlgP4gYFFZ9pSk/cpVX0c1bCsiIjqg2WfUt+pE4CJJ44G7gNlUQXappGOA+4CPlLpXAocCK4GnS11sPybpi8ANpd4XbD9Wpj8BfBPYFLiqvCIiokNUXTg1dnR1dbm7u7vTzYiIGFEkLbXdNVS9fKM+IiJqk1CJiIjaJFQiIqI2CZWIiKhNQiUiImqTUImIiNokVCIiojYJlYiIqE1CJSIiapNQiYiI2iRUIiKiNgmViIioTUIlIiJqk1CJiIjaJFQiIqI2CZWIiKhNQiUiImqTUImIiNokVCIiojYJlYiIqE1CJSIiapNQiYiI2rQ1VCTdI+kWSTdJ6i5le0i6rpR/T9KEhvqnSFop6XZJBzeUzyhlKyWd3FC+k6Qlku6QdImk8e3sT0REDG5DHKm8y/aetrvK/HnAybZ/B/gu8FcAknYDjgB2B2YAX5e0kaSNgLOAQ4DdgCNLXYCvAKfbngY8DhyzAfoTERHr0YnTX7sAPynT1wAfLtMzgfm2n7F9N7AS2Ke8Vtq+y/ZaYD4wU5KAdwMLyvoXAB/YQH2IiIgBtDtUDHxf0lJJc0rZrcBhZfojwNQyPRlY1bBuTylbX/lWwGrbz/Urj4iIDml3qPye7elUp66Ol3QA8LEyvRR4PbC21NUA67uF8nVImiOpW1J3b2/vcPsQERFNamuo2H6g/HyEavxkH9srbB9ke2/gYuDOUr2Hl45aAKYADwxS/igwUdK4fuUDteMc2122uyZNmlRP5yIiYh1tCxVJm0t6fd80cBBwq6RtStlrgL8Bzi6rLASOkLSJpJ2AacD1wA3AtHKl13iqwfyFtg1cCxxe1p8FXNau/kRExNDaeaSyLfBfkn5BFQ5X2L6a6uqtXwIrqI4s5gHYXgZcCtwGXA0cb/v5MmZyArAIWA5cWuoCnAR8WtJKqjGW89vYn4iIGIKqP/jHjq6uLnd3d3e6GRERI4qkpQ1fDVmvfKM+IiJqk1CJiIjaJFQiIqI2CZWIiKhNQiUiImqTUImIiNqMuUuKJfUC97a4+tZU3+QfS9LnsWGs9Xms9RdeeZ9/y/aQtyQZc6HySkjqbuY67dEkfR4bxlqfx1p/YcP1Oae/IiKiNgmViIioTUJleM7pdAM6IH0eG8Zan8daf2ED9TljKhERUZscqURERG0SKhERUZuEShMkzZB0u6SVkk7udHteCUlTJV0rabmkZZL+vJRvKekaSXeUn1uUckk6s/T9ZknTG7Y1q9S/Q9KsTvWpWZI2knSjpMvL/E6SlpT2X1IeAkd5UNwlpc9LJO3YsI1TSvntkg7uTE+aI2mipAWSVpT9vf9o38+SPlX+Xd8q6WJJrx1t+1nSNyQ9IunWhrLa9qukvSXdUtY5U9JAj25fP9t5DfICNqJ65PHOwHjgF8BunW7XK+jP9sD0Mv164JfAbsDfAyeX8pOBr5TpQ4GrAAH7AUtK+ZbAXeXnFmV6i073b4i+fxr4FnB5mb8UOKJMnw18okz/GXB2mT4CuKRM71b2/ybATuXfxUad7tcg/b0AOLZMjwcmjub9DEwG7gY2bdi/R4+2/QwcAEwHbm0oq22/Uj1Ucf+yzlXAIcNqX6ffoFf7q7y5ixrmTwFO6XS7auzfZcB7gduB7UvZ9sDtZfpfgSMb6t9elh8J/GtD+cvqvdpewBTgB8C7gcvLf5hHgXH99zPVU0b3L9PjSj313/eN9V5tL2BC+YBVv/JRu59LqKwqH5Tjyn4+eDTuZ2DHfqFSy34ty1Y0lL+sXjOvnP4aWt8/1D49pWzEK4f7ewFLgG1tPwhQfm5Tqq2v/yPtffkq8BnghTK/FbDa1eOq4eXtf7FvZfkTpf5I6vPOQC8wr5zyO0/S5ozi/Wz7fuAfgfuAB6n221JG937uU9d+nVym+5c3LaEytIHOJ47467AlvQ74NvB/bD85WNUByjxI+auOpPcBj9he2lg8QFUPsWzE9JnqL+/pwFzbewFrqE6LrM+I73MZR5hJdcpqB2Bz4JABqo6m/TyU4fbxFfc9oTK0HmBqw/wU4IEOtaUWkjamCpSLbH+nFD8safuyfHvgkVK+vv6PpPfl94DDJN0DzKc6BfZVYKKkcaVOY/tf7FtZ/gbgMUZWn3uAHttLyvwCqpAZzfv5PcDdtnttPwt8B/hfjO793Keu/dpTpvuXNy2hMrQbgGnlCpLxVAN6CzvcppaVKznOB5bb/ueGRQuBvitAZlGNtfSVH1WuItkPeKIcXi8CDpK0RfkL8aBS9qpj+xTbU2zvSLX/fmj7j4BrgcNLtf597nsvDi/1XcqPKFcN7QRMoxrUfNWx/RCwStIupehA4DZG8X6mOu21n6TNyr/zvj6P2v3coJb9WpY9JWm/8h4e1bCt5nR6wGkkvKiuoPgl1VUgn+23HFqYAAACF0lEQVR0e15hX95OdTh7M3BTeR1KdS75B8Ad5eeWpb6As0rfbwG6Grb1MWBlec3udN+a7P87eenqr52pPixWAv8P2KSUv7bMryzLd25Y/7PlvbidYV4V04G+7gl0l339H1RX+Yzq/Qx8HlgB3Ar8G9UVXKNqPwMXU40ZPUt1ZHFMnfsV6Crv353Av9DvYo+hXrlNS0RE1CanvyIiojYJlYiIqE1CJSIiapNQiYiI2iRUIiKiNgmVCEDSdpLmS7pT0m2SrpT0251uV8RIk1CJMa98yeu7wI9sv8n2bsBfA9t2tmXt03dr9Ii6jRu6SsSo9y7gWdtn9xXYvqmD7dkQviZpMnAe8G3bv+l0g2J0yJFKBLyV6m62Y4btPwb+kureWMskfU3SHh1uVowCCZWIMcr2UtvHA7tTblUi6dMdblaMcDn9FQHLeOmGgy+S9GXgD8rs3rx0NLMQuBE4tcwfCxxP9WyaB4DjgO+VZWdTPT30T8v8ocA8qvGabuBcqgckAXwO2Lfdv9P2saV/48qy2VQ3Tfwc8O8Dv0URzcm9v2LMKwP1i4HzbJ9byt4GbGb7xx1tXJuUI5ITgJ8C59v+SYebFKNEQiUCkLQD1TNW9gZ+A9xD9QCzOzrZrnaR9B7geg/+gLaIYUuoREREbTJQHxERtUmoREREbRIqERFRm4RKRETUJqESERG1SahERERtEioREVGb/w9+XL963hxh3gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(C,weights)\n",
    "ymin, ymax = plt.ylim()\n",
    "plt.xlabel('C--------------->')\n",
    "plt.ylabel('weights-------------->')\n",
    "plt.title('Sparcity in L2 regularization')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Mouwoyg_6B_M"
   },
   "source": [
    "Checking Multicollinearity\n",
    "-----------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(273098, 62063)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 347
    },
    "colab_type": "code",
    "id": "x63stfPPQ6CE",
    "outputId": "ad0a70c8-d5e7-46fb-89da-2e07876f3441"
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "import seaborn as ssb\n",
    "noise = np.random.normal(loc=0.2,scale=0.1,size=10000)#Creating random noise\n",
    "noise = noise.reshape(10000,1)\n",
    "noise_counts = tfidf_train[0:10000] + noise#adding noise to traindata(10000 points)\n",
    "m1 = LogisticRegression(C=2.7,penalty='l2')#fitting model with optimal hyperparameters for traindata\n",
    "m1.fit(tfidf_train[0:10000],y_train[0:10000])\n",
    "m2 = LogisticRegression(C=2.7,penalty='l2')\n",
    "m2.fit(noise_counts,y_train[0:10000])#fitting model with optimal hyperparameters for noise added data\n",
    "weights = np.array(m1.coef_)#obtaining coefficients of modeled traindata features\n",
    "weight_noise = np.array(m2.coef_)#obtaining coefficients of modeled (noise+traindata )features\n",
    "final_weight = np.subtract(weights,weight_noise)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/chandhana520/anaconda3/lib/python3.6/site-packages/matplotlib/axes/_axes.py:6462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced by the 'density' kwarg.\n",
      "  warnings.warn(\"The 'normed' kwarg is deprecated, and has been \"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmYXFd95vHvr3pfpN61WK3NlsAIY7DdtiGQDIMJ2IbBZnFikgENcaLJBAIJkwdMmAkESIKBsQOBEBTsYAa8B4/1BIORZWxjG0uWbWFJ1tbaW2q1elHvrV6qzvxxT1VXt1pSVVcvulfv53n01L23TlXdo+6u955z7rnXnHOIiMj5JzbbOyAiIrNDASAicp5SAIiInKcUACIi5ykFgIjIeUoBICJynjprAJjZXWZ23My2pW37upntNLNXzOxhM6tMe+5zZtZoZrvM7N1p26/12xrN7Napr4qIiGQjkxbAD4Brx21bD1zinLsU2A18DsDMVgE3A6/3r/lnM8szszzgO8B1wCrgw76siIjMkrMGgHPuaaBj3LZfOOdG/OrzQL1fvgG4zzk36JzbDzQCV/l/jc65fc65IeA+X1ZERGZJ/hS8xx8B9/vlRQSBkNTktwEcHrf96rO9cW1trVu2bNkU7KKIyPnjxRdfbHPO1Z2tXE4BYGafB0aAHyc3TVDMMXFLY8JrUJjZGmANwJIlS9i8eXMuuygict4xs4OZlJv0WUBmthp4L/CHbvSCQk3A4rRi9cDRM2w/hXNurXOuwTnXUFd31gATEZFJmlQAmNm1wGeB9znn+tOeWgfcbGZFZrYcWAlsAl4AVprZcjMrJBgoXpfbrouISC7O2gVkZvcCbwdqzawJ+ALBWT9FwHozA3jeOfenzrntZvYA8CpB19DHnXNx/z6fAB4D8oC7nHPbp6E+IiKSITuXLwfd0NDgNAYgIpIdM3vROddwtnKaCSwicp5SAIiInKcUACIi5ykFgIjIeUoBIJKFgaE4b/3qEzzX2DbbuyKSMwWASBY6B4Y40jnA3ra+2d4VkZwpAESyEE8Ep00nEufu6dMimVIAiGQhkQge4woAiQAFgEgW4n7iZOIcnkApkikFgEgWkkf+agFIFCgARLKQPPKPqwUgEaAAEMmCBoElShQAIlkY7QKa5R0RmQIKAJEsqAtIokQBIJIFdQFJlCgARLKgFoBEiQJAJAvJvn+1ACQKFAAiWdA8AIkSBYBIFtQFJFGiABDJggaBJUoUACJZiKsFIBGiABDJQkITwSRCFAAiWVAXkESJAkAkCxoElihRAIhkQfMAJEoUACJZ0CCwRIkCQCQLySN/NQAkCs4aAGZ2l5kdN7NtaduqzWy9me3xj1V+u5nZt8ys0cxeMbPL016z2pffY2arp6c6ItNLg8ASJZm0AH4AXDtu263ABufcSmCDXwe4Dljp/60BvgtBYABfAK4GrgK+kAwNkTBJDQIrACQCzhoAzrmngY5xm28A7vbLdwM3pm3/oQs8D1Sa2ULg3cB651yHc+4EsJ5TQ0XknKezgCRKJjsGMN851wzgH+f57YuAw2nlmvy2020XCRWdBSRRMtWDwDbBNneG7ae+gdkaM9tsZptbW1undOdEcqWzgCRKJhsALb5rB/943G9vAhanlasHjp5h+ymcc2udcw3OuYa6urpJ7p7I9EjoctASIZMNgHVA8kye1cAjads/6s8GejPQ5buIHgPeZWZVfvD3XX6bSKikzgJSC0AiIP9sBczsXuDtQK2ZNRGczfNV4AEzuwU4BNzkiz8KXA80Av3AxwCccx1m9mXgBV/uS8658QPLIuc8nQUkUXLWAHDOffg0T10zQVkHfPw073MXcFdWeydyjhmdBzDLOyIyBTQTWCQLGgSWKFEAiGRBg8ASJQoAkSyk5gGoBSARoAAQyUJcg8ASIQoAkSyoC0iiRAEgkoVkC0BdQBIFCgCRLKgFIFGiABDJQlw3hJEIUQCIZEGDwBIlCgCRLKgLSKJEASCSBQ0CS5QoAESykJwIphaARIECQCQLCV0OWiJEASCSBQ0CS5QoAESyoEFgiRIFgEgWki0A9QBJFCgARLKQPPLX/QAkChQAIlnQLSElShQAIlnQTeElShQAIlnQPACJEgWASBacG70YnFMrQEJOASCShfTBXzUCJOwUACJZSO/6UTeQhJ0CQCQLiTEtAAWAhJsCQCQLagFIlCgARLKQSIwuazKYhJ0CQCQLYwaB1QKQkFMAiGRBXUASJTkFgJn9pZltN7NtZnavmRWb2XIz22hme8zsfjMr9GWL/Hqjf37ZVFRAZCalD/yqC0jCbtIBYGaLgE8CDc65S4A84GbgNuAO59xK4ARwi3/JLcAJ59wK4A5fTiRU0o/608cDRMIo1y6gfKDEzPKBUqAZeAfwkH/+buBGv3yDX8c/f42ZWY6fLzKjxnQBqQUgITfpAHDOHQG+ARwi+OLvAl4EOp1zI75YE7DILy8CDvvXjvjyNePf18zWmNlmM9vc2to62d0TmRYJDQJLhOTSBVRFcFS/HLgAKAOum6Bo8q9koqP9U/6CnHNrnXMNzrmGurq6ye6eyLTQILBESS5dQO8E9jvnWp1zw8BPgN8CKn2XEEA9cNQvNwGLAfzzFUBHDp8vMuPSv/PVBSRhl0sAHALebGalvi//GuBV4JfAh3yZ1cAjfnmdX8c//4TT5RQlZOIJR0Fe0JhVF5CEXS5jABsJBnNfArb691oLfBb4tJk1EvTx3+lfcidQ47d/Grg1h/0WmRVBAAR/NmoBSNjln73I6TnnvgB8YdzmfcBVE5Q9CdyUy+eJzLaESwZAXGMAEnqaCSyShfQWgOYBSNgpAESykHCOQj8GoC4gCTsFgEgW4glHQX4stSwSZgoAkSykdwHpJDYJOwWASBYSjtGzgNQCkJBTAIhkIZ7QGIBEhwJAJAtx58jXWUASEQoAkSwk0mYCqwUgYacAEMlC3KXPA1AASLgpAEQy5JzDaRBYIkQBIJKh5Pe9uoAkKhQAIhlKHvGrC0iiQgEgkqHk3cAKdTVQiQgFgEiGxrcANAYgYacAEMlQ8oi/IN/fEEYtAAk5BYBIhhKntABmc29EcqcAEMlQssunUIPAEhEKAJEMpbqANAgsEaEAEMlQ8to/GgSWqFAAiGRIg8ASNQoAkQwlxo0BqAUgYacAEMmQ5gFI1CgARDI0fhBYXUASdgoAkQyNzgPwF4PTPAAJOQWASIaSLYDCfLUAJBoUACIZ0hiARI0CQCRDyXkA+bFkF5ACQMItpwAws0oze8jMdprZDjN7i5lVm9l6M9vjH6t8WTOzb5lZo5m9YmaXT00VRGZGsgsoP88wUxeQhF+uLYBvAj93zl0MvBHYAdwKbHDOrQQ2+HWA64CV/t8a4Ls5frbIjEoe8cfMyDNTC0BCb9IBYGZzgd8B7gRwzg055zqBG4C7fbG7gRv98g3AD13geaDSzBZOes9FZljyiD8vZsRihr7/JexyaQFcCLQC/2ZmL5vZ982sDJjvnGsG8I/zfPlFwOG01zf5bWOY2Roz22xmm1tbW3PYPZGplTziz/MtAHUBSdjlEgD5wOXAd51zlwF9jHb3TMQm2HbKX5Bzbq1zrsE511BXV5fD7olMreQ8gFjMyIupC0jCL5cAaAKanHMb/fpDBIHQkuza8Y/H08ovTnt9PXA0h88XmVHxtC4gM50FJOE36QBwzh0DDpvZa/2ma4BXgXXAar9tNfCIX14HfNSfDfRmoCvZVSQSBmMGgWPqApLwy8/x9X8O/NjMCoF9wMcIQuUBM7sFOATc5Ms+ClwPNAL9vqxIaKQPAussIImCnALAObcFaJjgqWsmKOuAj+fyeSKzKXntnzxLngWkAJBw00xgkQyluoBiqAUgkaAAEMnQmC6gmOlqoBJ6CgCRDKUCwIxYTJeCkPBTAIhkKNnlY7oUhESEAkAkQ+MvBRFXC0BCTgEgkqH0s4DyzFIzg0XCSgEgkqFE+llAuhSERIACQCRD6ZeCiOlicBIBCgCRDI25GqhaABIBCgCRDCWP+GOpQeBZ3iGRHCkARDI09n4AaBBYQk8BIJKhuO4HIBGjABDJUGLcILDmAUjYKQBEMjRmHkBM8wAk/BQAIhkaHQT28wDUApCQUwCIZCh9EDimmcASAQoAkQylAiA5CKwWgIScAkAkQwnnMAuuBhoz3Q9Awk8BIJKheMKRZwZAXgycWgAScgoAkQzFnSMWSwaA5gFI+CkARDKUSGsBmOYBSAQoAEQyFE8ER/6A7gcgkaAAEMlQwjn897/OApJIUACIZCiecKkWQDAPYJZ3SCRHCgCRDMXdaADkxdAgsISeAkAkQ4mEI2ZpZwGpC0hCTgEgkqFTu4AUABJuOQeAmeWZ2ctm9h9+fbmZbTSzPWZ2v5kV+u1Ffr3RP78s188WmUlxpxaARMtUtAA+BexIW78NuMM5txI4Adzit98CnHDOrQDu8OVEQiMxrgWgMQAJu5wCwMzqgfcA3/frBrwDeMgXuRu40S/f4Nfxz1/jy4uEQsKlzQPQ/QAkAnJtAfwj8BkgeUJcDdDpnBvx603AIr+8CDgM4J/v8uXHMLM1ZrbZzDa3trbmuHsiUyeueQASMZMOADN7L3DcOfdi+uYJiroMnhvd4Nxa51yDc66hrq5usrsnMuXGdwFpHoCEXX4Or30r8D4zux4oBuYStAgqzSzfH+XXA0d9+SZgMdBkZvlABdCRw+eLzKj4mNNAUQtAQm/SLQDn3Oecc/XOuWXAzcATzrk/BH4JfMgXWw084pfX+XX88084XU9XQiSRfhaQBoElAqZjHsBngU+bWSNBH/+dfvudQI3f/mng1mn4bJFpM2YegH/UQLCEWS5dQCnOuSeBJ/3yPuCqCcqcBG6ais8TmQ1xN/rFn7wsdNw5YhMOb4mc+zQTWCRDwf0AguVkEKgbSMJsSloAIlF2z8ZDABztHCDuHPdsPMTWpi4gGBcQCSu1AEQylHCkBoGTUxjVApAwUwCIZMg5l/riTwaB5gJImCkARDKUfhpoqgWgLiAJMQWASIYcpC4FkQwCdQFJmCkARDKUcA5jbAtAg8ASZgoAkQw5d2oLQAEgYaYAEMlQwjmSVzBPTv1SF5CEmQJAJEOJiVoAOgtIQkwBIJIhl94C0FlAEgEKAJEMTdQCUBeQhJkCQCRDboJ5ABoEljBTAIhkKOFIdQGpBSBRoAAQyZBLuydwTNcCkghQAIhkKL0FYJoHIBGgABDJUEItAIkYBYBIhpxaABIxCgCRDI1tASQHgWdxh0RypAAQyZDTDWEkYhQAIhlKTHRDGHUBSYgpAEQyFNwPIDkPINimFoCEmQJAJEPpt4RMDgLrWkASZgoAkQyl3xQ+2QJIqAUgIaYAEMlAsq//lBaAAkBCTAEgkoFkT0/ylpAxXQxOIkABIJKB5Bd97JQWwGztkUjuJh0AZrbYzH5pZjvMbLuZfcpvrzaz9Wa2xz9W+e1mZt8ys0Yze8XMLp+qSohMt+SBfmoMwG/XILCEWS4tgBHgfzrnXge8Gfi4ma0CbgU2OOdWAhv8OsB1wEr/bw3w3Rw+W2RGjR8DGL0lpAJAwmvSAeCca3bOveSXe4AdwCLgBuBuX+xu4Ea/fAPwQxd4Hqg0s4WT3nORGTS+BaCZwBIFUzIGYGbLgMuAjcB851wzBCEBzPPFFgGH017W5LeJnPNO1wJQF5CEWc4BYGblwL8Df+Gc6z5T0Qm2nfLXY2ZrzGyzmW1ubW3NdfdEcvL73/s1T+46njYIPO6WkGoBSIjlFABmVkDw5f9j59xP/OaWZNeOfzzutzcBi9NeXg8cHf+ezrm1zrkG51xDXV1dLrsnkrOtR7o42nVy9DTQU64FNEs7JjIFcjkLyIA7gR3OudvTnloHrPbLq4FH0rZ/1J8N9GagK9lVJHIuGhpJ0D8U5+RQ/LQtAHUBSZjl5/DatwIfAbaa2Ra/7a+BrwIPmNktwCHgJv/co8D1QCPQD3wsh88WmXZdA8MA9A+PpA0CB4+ms4AkAiYdAM65Z5i4Xx/gmgnKO+Djk/08kZmWDICBtBaA6WqgEiGaCSxyGl0DQwAMDMdPaQHofgASBQoAkdNItgBODicYGdcC0DwAiQIFgMhpdPYPp5YHhuJA+uWgNQ9Awk8BIHIayRYAQN/gCDA66JU+D6Ctd5DPPvRKKiREwkIBIHIaYwJgKAiAU1oACXhmTxv3bz7MK02dM7+TIjlQAIicRnoXUN9gsgsoWE+2BOIuaAEAtPcNzeTuieRMASByGt1pLYB+3wIYHQQ2YhZ0AbX6AEgGgUhYKABETqNzYJgFc4uB0TGAWNrMl7yYBS2AnuDIv61HASDhogAQOY2ugWGWVJcC0O8HeJMtAAjGARIJR3tf8MXf2qsuIAkXBYDIaXT2D1E7p5DCvNjpWwCJ0TEAdQFJ2CgARE6ja2CEipJCSgrz6JugBZBn47qAFAASMgoAkQk45+gaGKKipICSgrwJWwAx3wJIdgEpACRsFAAiExgYjjMcd1SWFlBSmMdIYuzloCHoAjrRP8xw3FGYF0u1BETCQgEg4j2xs4UHNwd3LU1OAku2AJIsvQVgRkv3SQBWzCtnYDieaimIhIECQMT7l6f2ccf63cDoJLCKkqAFkDS2BQDHfQBcvGAOoG4gCRcFgIh3oK2Po10nOTkcT7UAKksKKD1NCyDPjJbu4Av/4oVjA2B3Sw93P3dgZnZcZJIUACIEE72O+4lcB9v7Uy2AuWdoAcRixsBwcHbQxQvmAtDm5wL88NcH+MK67XSfHJ1NLHKuUQCIAAfa+1LL+9v6UpeBSA4CJ9m4eQDJxxXzyoHRFsCell4AGo/3Tut+i+RCASBCcNQ/utxHp78b2PhB4Ni4eQAA1WWF1M0pAkidCZT84m9sUQDIuUsBIEJw1A9QXpTPgfY+ugaGyYsZ5UX5Y1sAaa+J+RZAbXkRBXkxKksLaOsdpKNvKHVl0D3He2asDiLZUgDIeenp3a388d2bU7d0PNDWR92cIl4zv5z9bX109g9TUVKAmZ21BVBbXugfi2jrHUwd/ZvBHnUByTlMASDnpQdfbOLxHS3saO4Ggi6g5TVlLKst42B7P10Dw1SWFABQWpifel36GEByua486P6pLS+krXcwddR/xZKq1FgAwGce+g3/40cvTme1RLKiAJDzjnOOF/Z3ALDJP+5v72NpTSnLaspo7jpJS/dJ5voAOG0LINkFNCcZAEW09Q6xp6WX0sI8fntlHUc6B+gbHGEknuDRrcd4fEdL6t4CIrNNASCR1zc4wj8/2chJf8rm4Y4BjvkJXC8c6KB3cITWnkGW1QYtAIBtR7qpLA0CoKggdsq9gGE0AGrK0rqAeoIuoBXzynntguDMoH2tffymqYvewRGG4y4VOgAdfUMMjuhewjI7FAASefe9cJiv/XwXD798BICN+9sBuLS+gk37OzjgB4CX15axvCYIgIHhOBW+BRAzo9i3AmLj7gcAwRd/8FhIz+AIrzZ3s2JeOSvmBZPD9hzv4bnGNgAK8ozn9gaff3I4zrvueIov/8er01d5kTNQAEikjMQT/N1PX2XXsdGzbx5+uck/BgGwaX8HlaUF/MFVS2jvG+KJnccBWFZTxtLa0tTrkmMAQOpMoLN1AUFwVL9y3hyW1pRSkGfsOd7Ls3vbeP0Fc7liaRXP+jB4bPsx2nqHePilI6lrCDnn+OWu4wwMqVUg008BIKG27UgXRzsHUusPv3yEf/3Vfr64bjsAe1p62HakmyXVpWza38Hhjn42HejgymXVXH1hDQAP+AvALa0pZW5xQapLpyI9AHwLYPylIGDsWUBJK+eVU5AXY3ltGVubunjpYCdvXVHLWy+qZfvRbjr6hnhwcxOl/l4Dj25tBuCnW5v52L+9wG0/35l6r4GhOBt2tJDwZyyJTJUZDwAzu9bMdplZo5ndOtOfL+HgnDvlKPiRLUf49hN7cC74ItzT0sMHvvscN6993vevJ/inJxopyo/x633tPLe3jYdfPkJezPjmzW8C4F9/tY+D7f1cvbyaZTWl1M0pounEAPPmFFFWFJzts7QmaAVUlBamPrt0ghZAzP/1pM4CmjMaAMmZwSvmlfPs3jaG4okgAFbWAvDQi4d5dm8ba37nQi6sLePBzU0MxxN847FdAPx440EOdwST0z7//7Zyy92buevZ/an339HczQMvHE79X0Bwg/r0dZGzmdEAMLM84DvAdcAq4MNmtmom90FmVvfJ4TFnvcQTjke3NtN0YnTm7bYjXXzy3pf5zeFOAIZGEnzqvi286Uu/4OfbjgGw/tUW/vL+LXzjF7v5pycaGRpJ8Bf3b6E4P0bTiX6+8h+v8pOXmjjU0c/tv/cm5s8t4o71u3lky1HetqKWy5ZUcdWyan70/EEArlpejZlx1fJqgNTgb/pyegugeIIWQGvPIAY8tr2FezYeSnXtFObHWOzvJbxi3hycC/r+r1xWxaWLKphTlM/t/qqjH7qinpsaFrPpQAe3/WwnB9r7+bv3X0LMjDvW7+bn25r5yUtHqCkr5GuP7WJ3Sw+7W3q4ee3zfObfX+GbG/YAcKzrJO/7zjO879vPpi5R3Ts4wm0/38lj24+l9nkknmDT/o4xl61OJBwdfbqXwfko/+xFptRVQKNzbh+Amd0H3ABM+ShYcEenYQ629zOScCypLqWmrJDW3kEOdfRTnJ/HkppSygrzONp5kqbOfqrLCllaXYYZHO7op7VnkAUVxdRXlTI4EudAWz89g8GNwhdWlHCif4j9bX0kEo7ldWXUlBVxtHOA/W19lBXlcWFtOWVF+Rxs7+NQRz91c4q4qK4cB+w93sux7pMsqS7lwroyek+OsKulh+6BEVbOL2dJdSnNnSd51Z+nvmrhXBZUFNN4vJddLd1UlBRwyQUVlBbl80pTJ7uP9bC0pow3Lq5kJJHgpYOdNJ3oZ9XCuVy6uJJjXSd5fl87XQPDNCyt4pJFFWw/2s0zjW3kx4zfXlnLhbXlPLWnlV/tbuWCyhJ+d9V8KksL+NnWYzy/r53LllRy/RsW0tE3xD2bDrH9SDfvvmQBH7hsES8ePMGdz+ynvW+QP7x6Kde/YSH3bTrE/33+IEX5Mf77f7qIhqVVfOWnO9h6pIuSgjw+ec1KSgpi/P2jOxmKJ/jZtmb+6l2v5ZnGNn61p40l1aX82Y9fZM3vXMQPntvPJYsquLC2jNvX7+bXe9vZfrSb733kCrYc7uS7T+7l0a3NvLG+guvfsID2vkH+5pGgG+gz174WgPdfvohNBzooK8xj1cLg4m1XL6/mp680s7ymjHs2HgKgeyD4cvzN4U6GRhLAaVoAZpQU5qXGAsp9C6KmrJD7Xwi6lZKXi66vKk3NJ7j6whoe39HCb6+spb6qlA9evohv/GIX339mPw1Lq/iDq5ZwqKOftU/v4/EdLVyyaC7/+tEG3vutZ/jkvS9zon+IovwY73nDQv7x8T0MDMdZt+Vo6vpF7//Os3zm2ou5ff1uDvlWxO811HPdJQv5h5/tYHdLL/PnFvHX17+O4oI87li/m53Henjn6+bxyWtWcrhjgB88t5+W7kF+/8rF3HjZIl7Y38HDLx+hIM/4wOX1XLmsmqd2t/LLncepryrh+jcsZGFFMU/uauXlwydYdUEF77h4HiPxBE/vbmV/Wz9XLqvity6q5XhP8LvYMzjClcuqueSCCva29vLSoRMU5sW4YmkVi6tL2dHczavN3cybU8wb64Pf9R3N3exv62NpdSmrLpjLSNyxo7mb1t5BP/BeTmf/MLuO9TA0kuA18+dwQWUxzV0naWztpSg/xsp5c6gsLeBQRz8H2/uoKi1khe+yO9jeT3PXABdUlrC0ppShkQQH2oJ5IUuqS1lUVULXwDAH2vuIJxxLq0upLS+itXeQg+39FBfEWFpdRllRHs1dJznSOUBVaSFLqksxg6YT/bT2DLGgophFlSUMxRMcau+nd3CE+qoSFswtpnNgmIP+/RuWVU/p9+J4Mx0Ai4DDaetNwNVT/SHHuk7yu3c8Rc/JsedbxwzGd6NOtM0M0lvS49dzea+JTFQmk9dNhZiBg9QRKQRHvt0nh1NHlwCLq0vYsPM43/hFUK68KJ/XLZzDPz2xh2/5cq+ZX87FC+Zy+/rd3L5+NzGD919WT2f/EF/3XRt1c4q47YNvYMOO46l+7ndcPI+/ee8q/v7RHfzDz3aSFzO+/qFLec+lC/nEPS/zL0/tpb6qhDtXX0lFSQHHewZ5bm87N11RT3vvEAsrillYEfyRX7akins3HQYX1GNgOM6JvmHu2XiIgaE4+THjgsoSHtgcDAwnr93T2T96BJzs0y9NuwREaWEexqmngc4pHv0TKsiLUVwQS10XCGDenGIALqobDZjigqDhfUFlSWrbynnl7DzWw+V+/+eVF1OUH6N/KM41F8/nlztbue6SBfxo4yHmFufz4J/+FhfVlTEcT/C9p/Yxf24RD/zpW3AO/ugHL/AX929hUWUJ9/zJ1Tzb2MY/P7mXBzY3UV9VwpdueD0Pbm7iU/dtAYKzn25523Ie3HyY9337WQCWVJdSX1XC1x/blfrZ1VeVMBJ3PL7jpbT6FfGLV4f43tP7UtvKCvO4d9Nh/nfa71lBno3pwpoumf4tZbItl7/7qfheeWN9BY984m1MJ5vJPkMzuwl4t3Puj/36R4CrnHN/nlZmDbDGr74W2JXBW9cCbVO8u7MtanWKWn0genWKWn0genXKtD5LnXN1Zys00y2AJmBx2no9cDS9gHNuLbA2mzc1s83OuYbcd+/cEbU6Ra0+EL06Ra0+EL06TXV9ZvosoBeAlWa23MwKgZuBdTO8DyIiwgy3AJxzI2b2CeAxIA+4yzm3fSb3QUREAjPdBYRz7lHg0Sl+26y6jEIianWKWn0genWKWn0genWa0vrM6CCwiIicO3QpCBGR81RoAsDMqs1svZnt8Y9Vpym32pfZY2ar07YXmtlaM9ttZjvN7IMzt/cTy7VOac+vM7Nt07/HZ5ZLfcys1Mx+6n82283sqzO792P274yXKzGzIjO73z+/0cyWpT33Ob99l5m9eyb3+0wmWycz+10ze9HMtvrHd8z0vp9DvIB7AAAD+ElEQVROLj8n//wSM+s1s7+aqX0+kxx/7y41s1/7v52tZlac0Yc650LxD/gacKtfvhW4bYIy1cA+/1jll6v8c38LfMUvx4DasNfJP/8B4B5gW5jrA5QC/9mXKQR+BVw3C3XIA/YCF/r9+A2walyZPwP+xS/fDNzvl1f58kXAcv8+eefAzyWXOl0GXOCXLwGOzHZ9cq1T2vP/DjwI/FWY60MwlvsK8Ea/XpPp792s/yCz+A/aBSz0ywuBXROU+TDwvbT17wEf9suHgbLZrscU16kceMZ/8ZwLAZBTfcaV+ybwJ7NQh7cAj6Wtfw743LgyjwFv8cv5BBNzbHzZ9HKz/HOZdJ3GlTGgHSgKe52AG4GvA188RwIgl9+764EfTeZzQ9MFBMx3zjUD+Md5E5SZ6FITi8ys0q9/2cxeMrMHzWz+9O5uRiZdJ7/8ZeD/AP3jXzRLcq0PAP7n9V+ADdO0n2dy1v1LL+OcGwG6CI66MnntbMilTuk+CLzsnBucpv3MxqTrZGZlwGcJegXOFbn8jF4DODN7zH+/fSbTD53x00DPxMweBxZM8NTnM32LCbY5gnrWA8865z5tZp8GvgF8ZFI7moXpqpOZvQlY4Zz7y/F9m9NpGn9GyffPB+4FvuX8RQNn2Bn37yxlMnntbMilTsGTZq8HbgPeNYX7lYtc6vS3wB3OuV6ziYrMilzqkw+8DbiS4GBwg5m96Jw76wHUORUAzrl3nu45M2sxs4XOuWYzWwgcn6BYE/D2tPV64EmCZms/8LDf/iBwy1Ts89lMY53eAlxhZgcIfo7zzOxJ59zbmUbTWJ+ktcAe59w/TsHuTsZZL1eSVqbJB1YF0JHha2dDLnXCzOoJ/nY+6pzbO/27m5Fc6nQ18CEz+xpQCSTM7KRz7tvTv9unlevv3VPOuTYAM3sUuJxMWtCz3feVRR/Z1xk7wPi1CcpUA/sJBhWr/HK1f+4+4B1++b8BD4a9TmlllnFujAHk+jP6CsHAXGwW65BPMDC9nNHBuNePK/Nxxg7GPeCXX8/YQeB9nBuDwLnUqdKX/+Bs12Oq6jSuzBc5N8YAcvkZVQEvEZxIkQ88Drwno8+d7Ypn8R9UQ5Boe/xj8kujAfh+Wrk/Ahr9v4+lbV8KPE0wWr4BWBL2OqU9v4xzIwAmXR+CIx4H7AC2+H9/PEv1uB7YTXBWxuf9ti8B7/PLxQStyEZgE3Bh2ms/71+3i1k4i2mq6wT8L6Av7WeyBZg32/XJ9eeU9h5f5BwIgCn4vfuvwHZgGxMceJ3un2YCi4icp8J0FpCIiEwhBYCIyHlKASAicp5SAIiInKcUACIi5ykFgIjIeUoBICJynlIAiIicp/4/RWdBwPvuEQYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ssb.distplot(final_weight,label='PERTUBATION FOR CHECKING MULTICOLLINEARITY')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SjwPImq56gs_"
   },
   "source": [
    "**since the weights did not differ much i.e almost all the values are 0 we can use these features for modeling**\n",
    "---------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modelling using optimal hyperparameters obtained by GRID-SEARCH\n",
    "=========="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 89
    },
    "colab_type": "code",
    "id": "JVGKo6QrK15q",
    "outputId": "f3bdcdcf-e633-4813-92c4-b36133821681"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training set 93.64294136170899 %\n",
      "Accuracy on test set:  92.5521514176178 %\n",
      "confusion matrix for test data\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD8CAYAAAC8TPVwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGNpJREFUeJzt3X+cjXX+//HHiyFk/P6xGhJF0n5bFKm2UqQhFZ/VfrT50ZamiN3tx/Zjt2+t2t1v9SW7NhVijSWT+FQKSYqmRLSUFGuQDAqNrN9mznl//pjLdLbGzBnz43hfnvdu79uc877e13Xe1+02vebldb2v65hzDhER8UOlRE9ARETip6AtIuIRBW0REY8oaIuIeERBW0TEIwraIiIeUdAWEfGIgraIiEcUtEVEPJJU3h/QplFH3XIpP7D9QE6ipyAnoD37Nlhpj5G7a2PcMadKg5al/ryKpkxbRMQj5Z5pi4hUqGgk0TMoVwraIhIukbxEz6BcKWiLSKg4F030FMqVgraIhEtUQVtExB/KtEVEPKILkSIiHlGmLSLiD6fVIyIiHtGFSBERj6g8IiLiEV2IFBHxiDJtERGP6EKkiIhHdCFSRMQfzqmmLSLiD9W0RUQ8ovKIiIhHlGmLiHgkkpvoGZQrBW0RCReVR0REPKLyiIiIR5Rpi4h4REFbRMQfThciRUQ8opq2iIhHVB4REfGIMm0REY8o0xYR8YgybRERj+TpSxBERPyhTFtExCOqaYuIeESZtoiIR5Rpi4h4RJm2iIhHtHpERMQjziV6BuVKQVtEwkU1bRERj4Q8aFdK9ARERMqUi8bfimFmdcxsppmtNbPPzewiM6tnZgvMbH3ws24w1sxsjJllmdknZtYh5jiDgvHrzWxQTP/5ZrY62GeMmVlxc1LQFpFwiUTib8X7K/CGc64N8BPgc+ABYKFzrhWwMHgP0ANoFbQ04FkAM6sHPAJcCHQCHjka6IMxaTH7pRY3IQVtEQmXaDT+VgQzqwVcBkwEcM4dcc59C1wPpAfD0oHewevrgSku31Kgjpk1Aa4GFjjncpxzu4EFQGqwrZZz7gPnnAOmxBzrmBS0RSRcShC0zSzNzFbEtLSYI7UEdgJ/N7OVZva8mZ0KNHbObQcIfjYKxqcAW2L2zw76iurPLqS/SLoQKSLhUoKba5xz44Hxx9icBHQAhjvnlpnZX/muFFKYwurR7jj6i6RMW0RCxUVd3K0Y2UC2c25Z8H4m+UH866C0QfBzR8z4ZjH7NwW2FdPftJD+Iiloi0i4lFFN2zn3FbDFzM4OuroCnwGzgaMrQAYBrwavZwMDg1UknYE9QflkPtDdzOoGFyC7A/ODbXvNrHOwamRgzLGOSeUREQmX+FaFxGs4MM3MqgIbgV+Sn+zOMLNbgS+BG4Kxc4GeQBZwIBiLcy7HzB4DlgfjHnXO5QSvhwCTgerAvKAVSUFbRMKlDG+ucc6tAi4oZFPXQsY64M5jHGcSMKmQ/hXAj0syJ5VHSmnAbf2YvTiD1959kYFpNwIw7Le3sfjjObz89jRefnsal3W9+D/2aZLSmI82LeaWof0L+haueJXZi6bz8tvTmPlmOuK/SpUqkfn+bF58aQIA897MIHPJa2QueY2165cwbfpzANSpU4up05/l/aVzeHvR/3BO29YFx6hdO5kpU59m+T/f5MOP5tOxU/uEnItXyqg8cqJSpl0KrdqcyQ39e/Pz1EHkHsljwotjWLzgPQDSx01n0jNTC93vwcfuJnPhkh/0D/yvO/g2Z0+5zlkqzpChN7Nu3QaSk2sC0KN7v4Jt/5g2ljmvvwXAPfcOZfUnn9H/xiG0at2SUU+N4LpeAwB4/MmHeWvBuwzsP4wqVapQo0a1ij8R34T8gVHFZtpm1sbM7g9usfxr8Pqcipjcia5lqzP4+KPVHDp4mEgkwvIl/6TbNV2K3Kdrj8vZsnkrWes2VswkJSFOO+1HXJ16BVPSZ/xgW82ap3LZZRcx5/UFAJzd5iwWL8r/I77+Xxs5/fQUGjaqT3JyTS65pGPBMXJzc9mzZ2/FnYSvQp5pFxm0zex+IIP89YQfkl9IN2C6mRW1XvGksH7tBjpe1J46dWtTrfopXN7tYpqc1hiAm265gVcXvcCf/vJ/qVU7GYDqNapx2/CBjB054QfHcs4xccbTzFowhZ8P6FOh5yFl7/EnH+Lhh54gWkhg6HVtdxYvXsLevfsA+HT151x73dUAdDj/PJqdnkLKaU0444xm7NqVwzPPPUnm+7P529N/pkaN6hV6Hl6Kuvibh4rLtG8FOjrnHnfOTQ3a4+TfP39r+U/vxLZx/RdM+NsUJr70NBMyxrB2zXry8iJMnzyLqzr1ofcVN7Hz613cP+I3AAy/73YmPzedA/sP/uBYv+g1mJ91G8BtN/6aX9zSlws6q3bpq6tTr2Dnzm9YterTQrf3veFaZr70WsH70U+No06d2mQueY3b7xjIJx9/Rl5eHklJSfyk3blMfH4al15yHfsPHOSue+6oqNPwV9k+e+SEU1xNOwqcBmz+Xn+TYFuhgltB0wAa12xOneoNSzPHE9qsF2Yz64XZANz1u6F8tX0H3+zMKdj+0tRXeHbqaADO63AuV/e6kt8+PJzk2slEo1EOHzrMtEkvsePrXQDk7NrNW3MXcV6Hc1mxdGXFn5CUWufO59OjZ1eu6t6FatVOITm5JuOfH0Xa4HuoW68O559/Hjfd+F3w3bt3H3cOub/g/SdrFrN5czbVq1dj69av+GjFxwC8+so87rpbQbs4ztOyR7yKC9q/ARaa2Xq+u3f+dOAsYNixdoq9NbRNo45+/hskTvUa1CVn126apDTmqmuuoF/PW2jYqD47d3wDQLeeXVi/dgMA/a/77rEGw357Gwf2H2TapJeoXqMalawS+/cfoHqNalzSpTNjRz6fkPOR0hvxh5GM+MNIAH566YUM/9Vg0gbfA0CfPj144413OHz4SMH42rWTOXDgELm5uQy6+b9Z8v5y9u7dx969+9i6dTtntWpB1vpNXN7lYtatzUrIOXnF07JHvIoM2s65N8ysNfnlkBTy69nZwHLnnJ//tihjYyY9QZ26tcnLy+PRB57k33v28sTYEZxzbmscjq1fbueRe/9c5DHqN6zP05OfBKBy5SRe/583eO+dDypi+lLB/qtvL0aPGvcffa3PPotx40cSiUZYtzaLYUO/u1x03z0jeH7iaKpUrcIXm7Zw55D7KnrK/gn5F/uaK+flMWHPtOX4bD+QU/wgOens2beh2C8BKM7+R2+KO+ac+vC0Un9eRdM6bREJl7xwFwEUtEUkXEJeHlHQFpFwOZkvRIqI+OZkX/InIuIXZdoiIh5R0BYR8Yint6fHS0FbREIlju9+9JqCtoiEi4K2iIhHtHpERMQjyrRFRDyioC0i4g8XUXlERMQfyrRFRPyhJX8iIj5R0BYR8Ui4S9oK2iISLi4v3FFbQVtEwiXcMVtBW0TCRRciRUR8okxbRMQfyrRFRHyiTFtExB8uL9EzKF8K2iISKk6ZtoiIRxS0RUT8EfZMu1KiJyAiUpZcNP4WDzOrbGYrzez14P1kM9tkZquC1i7oNzMbY2ZZZvaJmXWIOcYgM1sftEEx/eeb2epgnzFmZsXNR5m2iISKixQb90rq18DnQK2Yvt8652Z+b1wPoFXQLgSeBS40s3rAI8AFgAM+MrPZzrndwZg0YCkwF0gF5hU1GWXaIhIqZZlpm1lT4Brg+Tg++npgisu3FKhjZk2Aq4EFzrmcIFAvAFKDbbWccx845xwwBehd3IcoaItIqLioxd3MLM3MVsS0tO8d7i/Affzw8uafghLIaDM7JehLAbbEjMkO+orqzy6kv0gK2iISKiXJtJ1z451zF8S08UePY2a9gB3OuY++9xEPAm2AjkA94P6juxQ2nePoL5KCtoiEinMWdyvGJcB1ZvYFkAFcaWZTnXPbgxLIYeDvQKdgfDbQLGb/psC2YvqbFtJfJAVtEQmVsqppO+cedM41dc6dAfQD3nbO9Q9q0QQrPXoDnwa7zAYGBqtIOgN7nHPbgflAdzOra2Z1ge7A/GDbXjPrHBxrIPBqceen1SMiEirRsl898n3TzKwh+eWNVcAdQf9coCeQBRwAfgngnMsxs8eA5cG4R51zOcHrIcBkoDr5q0aKXDkCYPkXLctPm0Ydw/3ILTku2w/kFD9ITjp79m0odcTd3KFb3DGn+T/fKvcIX9aUaYtIqLiod3G4RBS0RSRUyrl4kHAK2iISKsq0RUQ8EsdSPq8paItIqETKf/VIQiloi0ioKNMWEfGIatoiIh7R6hEREY8o0xYR8UgkGu5HKiloi0ioqDwiIuKRqFaPiIj4Q0v+REQ8ovJIKWV9W+wXMchJ6OC2zERPQUJK5REREY9o9YiIiEdCXh1R0BaRcFF5RETEI1o9IiLikWK+ZN17CtoiEioOZdoiIt7IU3lERMQfyrRFRDyimraIiEeUaYuIeESZtoiIRyLKtEVE/BHybxtT0BaRcIkq0xYR8YceGCUi4hFdiBQR8UjUVB4REfFGJNETKGcK2iISKlo9IiLikbCvHgn3l6mJyEnHlaAVxcyqmdmHZvaxma0xsxFBfwszW2Zm683sRTOrGvSfErzPCrafEXOsB4P+dWZ2dUx/atCXZWYPxHN+CtoiEipRi78V4zBwpXPuJ0A7INXMOgNPAKOdc62A3cCtwfhbgd3OubOA0cE4zKwt0A84F0gFnjGzymZWGRgL9ADaAjcGY4ukoC0ioRItQSuKy7cveFslaA64EpgZ9KcDvYPX1wfvCbZ3NTML+jOcc4edc5uALKBT0LKccxudc0eAjGBskRS0RSRUIhZ/K06QEa8CdgALgA3At865vGBINpASvE4BtgAE2/cA9WP7v7fPsfqLpKAtIqFSkkzbzNLMbEVMS4s9lnMu4pxrBzQlPzM+p5CPPFoeL+zPgDuO/iJp9YiIhEpJ7oh0zo0Hxscx7lszWwR0BuqYWVKQTTcFtgXDsoFmQLaZJQG1gZyY/qNi9zlW/zEp0xaRUHEWfyuKmTU0szrB6+pAN+Bz4B2gbzBsEPBq8Hp28J5g+9vOORf09wtWl7QAWgEfAsuBVsFqlKrkX6ycXdz5KdMWkVApw2ePNAHSg1UelYAZzrnXzewzIMPM/gisBCYG4ycC/zCzLPIz7H4Azrk1ZjYD+AzIA+50zkUAzGwYMB+oDExyzq0pblKW/4eg/CRVTQn7Q7fkOBzclpnoKcgJqEqDlqW+M+ZvzfrHHXOGb5nq3Z04yrRFJFR0G7uIiEf0aFYREY8oaIuIeCTsF9EUtEUkVFTTFhHxiL4EQUTEI9GQF0gUtEUkVHQhUkTEI+HOsxW0RSRklGmLiHgkz8Kdaytoi0iohDtkK2iLSMioPCIi4hEt+RMR8Ui4Q7aCtoiEjMojIiIeiYQ811bQFpFQUaYtIuIRp0xbRMQfyrSlSBPGj+Kant3YsXMX7dp3BeC889ryzNOPc2rNGmzenM2AgcPYu3cfSUlJjB83kvbtf0xSUhJTp87kiSefLjhWpUqVWLZ0Htu2fsX1fQYl6pTkOGzanM29D/+/gvfZ27YzbPAABvx3HwD+/sJMRo2dSOacDOrWqc2kaTOZ8+Y7AEQiETZu3kLmnAxq10rmvaUrePwvzxGJRvnZtakMHvBzAJauWMmosROJRh01alTjT7+/h9ObnlbxJ3uCC/uSv0qJnoDvpkyZwTW9bvqPvnHP/X9+9/s/075DN155ZR733jMEgL59e3HKKVVp36EbnS5M5bbB/WnevGnBfr8aPpi1a9dX6PylbLRo3pRZ6WOZlT6WGZPGUK1aNbpefjEA27/eyQfLV9KkcaOC8bfc1Ldg/G/uuJkL2v0fatdKJhKJ8MdRY3l21GPMnjaOuW8tYsOmzQA8NnIsjz9yH7PSx3LNVVcwbvL0hJzric6VoPlIQbuUMt9bRs7ub/+j7+zWZ/Ju5lIA3lqYSZ8+PQFwznHqqTWoXLky1atX50huLv/+9z4AUlKa0LNHVyZN0v+Ivlu6YhXNUppw2o8aA/DkmHHcPfRW7BjfqDL3rcX0vOpyAFZ//i9Ob3oazVKaUKVKFXp0vZy3g98lA/bvPwDA3n37adigfrmfi4/ycHE3Hx130DazX5blRMJkzZp1XHttdwD6/qwXzYJ/ws6aNYf9+w+Q/eVKNm34kKeeeo7dQcB/atQIHnjwj0SjYa/Ihd+8hYvp2S0/CL+TuZRGDRvQplXLQscePHSI95au4KouPwVgx85d/KhRw4LtjRs1YMfObwAY8cBvGHLvw3Tt3Z/X5i9k8IAbyvlM/ORK8J+PSpNpjzjWBjNLM7MVZrYiGt1fio/w0+C0uxl6x80sWzqP5ORTOXIkF4BOHdsRiURo1rwDZ7XuzF133U6LFqfn18R37OKfK1cneOZSWrm5uSx6bxndr7yUg4cOMX5KBsMGDzjm+EXvLaP9eW2pXSsZAFdIHDmaoU958WWeHfkoC1+ZSu+e3XlyzITyOAXvRUvQfFTkhUgz++RYm4DGx9rPOTceGA+QVDXFzz9npbBu3QZ6XPMLAFq1aknPHvkXKPv168P8NxeRl5fHzp3fsGTJcs4//ye0b3cu1/bqTo/UK6lW7RRq1UomffIYBt38q0SehhyHzKUrOKf1mTSoV5d/bdjE1m1f8bNBQwH4eucubrhlOBkT/kKD+vWAo1l5l4L9GzdqwFc7dha8/3rHLho2qE/O7m9Zl7WR885tA0CPrpdx+z0PVdyJecTXDDpexWXajYGBwLWFtG/Kd2r+atgwv9ZoZvzuwV8zbvw/ANiyZStXdLkEgBo1qnPhhR1Yty6L3z/0OGe0vICzWnfmpv5Deeed9xWwPTV3wSJ6XtUFgNZntuDdORm8OSudN2el07hhA16a9LeCgL13335WrFzNFZdeVLD/j9u05svsbWRv+4rc3FzmLVzMFT/tTK3kZPbtP8AXX2YDsGT5Slo2P73Cz88HJ3WmDbwO1HTOrfr+BjNbVC4z8szUf4zl8ssuokGDenyxcQUjHh1JzZqnMmTIzQC88spcJqe/CMAzz05m4vOj+XjV25gZ6ekvsnr15wmcvZSlg4cO8cHylTxyX3x/cBcuXsLFnTpQo3q1gr6kpMr87q4h3H73Q0QiEfr06s5ZLZsD8If7f8Vdv/8TVsmolVyTxx68q1zOw3eRwmpMIWKunE/wZCyPSPEObstM9BTkBFSlQctjrLGJ3y+a94k75ryw+eVSf15F0801IhIqYa9pK2iLSKj4WquOl4K2iIRK2G9jV9AWkVBReURExCNhXz2ioC0ioaLyiIiIR8J+IVJP+RORUCnLB0aZ2SQz22Fmn8b0/cHMtprZqqD1jNn2oJllmdk6M7s6pj816Msyswdi+luY2TIzW29mL5pZ1eLmpKAtIqESxcXd4jAZSC2kf7Rzrl3Q5gKYWVugH3BusM8zZlbZzCoDY4EeQFvgxmAswBPBsVoBu4Fbi5uQgraIhIpzLu4Wx7HeBXLi/OjrgQzn3GHn3CYgC+gUtCzn3Ebn3BEgA7jezAy4EpgZ7J8O9C7uQxS0RSRUIri4WykMM7NPgvJJ3aAvBdgSMyY76DtWf33gW+dc3vf6i6SgLSKhUpLySOyz/4OWFsdHPAucCbQDtgOjgv7CnmPijqO/SFo9IiKhUpKH4MU++78E+3x99LWZTSD/aaiQnyk3ixnaFNgWvC6sfxdQx8ySgmw7dvwxKdMWkVAp4wuRP2BmTWLe9gGOriyZDfQzs1PMrAXQCvgQWA60ClaKVCX/YuVsl//X5R2gb7D/IODV4j5fmbaIhEpZ3sZuZtOBLkADM8sGHgG6mFk78ksZXwC3Azjn1pjZDOAzIA+40zkXCY4zDJgPVAYmOefWBB9xP5BhZn8EVgITi52TnqctiaDnaUthyuJ52pemdI075mRuXajnaYuIJJJuYxcR8YiCtoiIR8q75JtoCtoiEirKtEVEPKIvQRAR8UjEhfvhrAraIhIqqmmLiHhENW0REY+opi0i4pGoyiMiIv5Qpi0i4hGtHhER8YjKIyIiHlF5RETEI8q0RUQ8okxbRMQjkfwviwktBW0RCRXdxi4i4hHdxi4i4hFl2iIiHtHqERERj2j1iIiIR3Qbu4iIR1TTFhHxiGraIiIeUaYtIuIRrdMWEfGIMm0REY9o9YiIiEd0IVJExCMqj4iIeER3RIqIeESZtoiIR8Je07aw/1U6kZhZmnNufKLnIScW/V5ISVRK9AROMmmJnoCckPR7IXFT0BYR8YiCtoiIRxS0K5bqllIY/V5I3HQhUkTEI8q0RUQ8oqBdQcws1czWmVmWmT2Q6PlI4pnZJDPbYWafJnou4g8F7QpgZpWBsUAPoC1wo5m1Teys5AQwGUhN9CTELwraFaMTkOWc2+icOwJkANcneE6SYM65d4GcRM9D/KKgXTFSgC0x77ODPhGRElHQrhhWSJ+W7YhIiSloV4xsoFnM+6bAtgTNRUQ8pqBdMZYDrcyshZlVBfoBsxM8JxHxkIJ2BXDO5QHDgPnA58AM59yaxM5KEs3MpgMfAGebWbaZ3ZroOcmJT3dEioh4RJm2iIhHFLRFRDyioC0i4hEFbRERjyhoi4h4REFbRMQjCtoiIh5R0BYR8cj/Ar4qcH/EHQdHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = LogisticRegression(C=2.7,penalty='l2')\n",
    "model.fit(tfidf_train,y_train)\n",
    "pred = model.predict(tfidf_train)\n",
    "acc_train = accuracy_score(y_train,pred)\n",
    "print('Accuracy on training set',acc_train*100,'%')\n",
    "#obtaining trainingset accuracy\n",
    "\n",
    "predict = model.predict(tfidf_test)\n",
    "acc_test = accuracy_score(y_test,predict)\n",
    "confusion = confusion_matrix(y_test,predict)\n",
    "#obtaining testset accuracy\n",
    "print('Accuracy on test set: ',acc_test*100,\"%\")\n",
    "print('confusion matrix for test data')\n",
    "Q= ssb.heatmap(confusion, annot=True, fmt=\"d\")\n",
    "#printing confusion matrix for visualization\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modelling using optimal hyperparameters obtained by RANDOMIZED-SEARCH\n",
    "=======\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training set 94.22588228401526 %\n",
      "Accuracy on test set:  92.52029483813561 %\n",
      "confusion matrix for test data\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD8CAYAAAC8TPVwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGGhJREFUeJzt3Xt4VdW19/HvICGoWAUEkYsiIIrgqYjcKYgoGMEWEG2hXqiHNtaCtq/ntOjroxyp59TL06r0KM+LimJLuRSLIModESzKRQHlKgGqhHBREC8BhCTj/SOLGDRkbyDJZi5/H5/5sPdcc+09l4TBYKy51jJ3R0REwlAl1RMQEZHkKWiLiAREQVtEJCAK2iIiAVHQFhEJiIK2iEhAFLRFRAKioC0iEhAFbRGRgKRX9Bc0P7utLrmUb8nN253qKchJ6PO8zXain3Hok81Jx5yqtZuc8PdVNmXaIiIBqfBMW0SkUhUWpHoGFUpBW0TipSA/1TOoUAraIhIr7oWpnkKFUtAWkXgpVNAWEQmHMm0RkYDoRKSISECUaYuIhMO1ekREJCA6ESkiEhCVR0REAqITkSIiAVGmLSISEJ2IFBEJiE5EioiEw101bRGRcKimLSISEJVHREQCokxbRCQgBYdSPYMKpaAtIvGi8oiISEBUHhERCYgybRGRgChoi4iEw3UiUkQkIKppi4gEROUREZGAKNMWEQmIMm0RkYAo0xYRCUi+HoIgIhIOZdoiIgFRTVtEJCDKtEVEAqJMW0QkIMq0RUQCotUjIiIBcU/1DCqUgraIxEvMa9pVUj0BEZFyVViYfEvAzGqY2WQzW29m68yso5nVMrM5ZrYx+rVmNNbMbKSZZZvZe2bWusTnDIrGbzSzQSX6Lzez96N9RpqZJZqTgraIxIsXJt8SexKY6e7NgUuBdcA9wDx3bwbMi94DXAs0i1oWMArAzGoBw4H2QDtg+OFAH43JKrFfZqIJKWiLSLwUFCTfymBmZwBdgecA3P2gu+8F+gBjo2Fjgb7R6z7Ai17kbaCGmdUDrgHmuPsed/8UmANkRtvOcPe33N2BF0t81lEpaItIvJRfeaQJ8DHwvJmtMLNnzaw6UNfdtwNEv54djW8AbC2xf07UV1Z/Tin9ZVLQFpF4OYagbWZZZra8RMsq8UnpQGtglLtfBuTxdSmkNKXVo/04+suk1SMiEi/HcHGNu48GRh9lcw6Q4+5LoveTKQraO82snrtvj0ocu0qMP7fE/g2B3Ki/2zf6F0T9DUsZXyZl2iISK17oSbcyP8d9B7DVzC6Kuq4C1gLTgMMrQAYBU6PX04Bbo1UkHYDPovLJLKCnmdWMTkD2BGZF274wsw7RqpFbS3zWUSnTFpF4Kd912ncC48wsA9gM3EZRsjvJzAYDHwE3RmNfA3oB2cC+aCzuvsfMfg8si8aNcPc90es7gBeAU4EZUSuTgraIxEuCVSHHwt1XAm1K2XRVKWMdGHKUzxkDjCmlfzlwybHMSUFbROJFV0RKWW75xQCmvTGBVxZO5NasgQD8afT/MGX+OKbMH8e85VOZMn9c8fisu37GrCX/YMbiyfzgyg4AZFTLYNLMF3j59XG8snAid/4uq9TvkrBUqVKFRYtfYdLkZ4v77h/+H7y7ch7L3pnNL+8oKov26n01i5e8xptvTWfBoql06Ph1YjfioWEsWTaTZe/M5tHHHqj0YwhSOV4ReTJSpn0CmjVvyo039+XHmYM4dDCfZyaO5I05b3J31v8tHjPswd/wxedfAtD0wsb06teD67r8hLPPqcPzk58is0N/Dn51kJ/1v4N9eftJT09j3CvPsnDeYla9szpVhybl4I4ht/HBhk1873unA3DTLTfQsGE9Lr/satyd2nXOAuCNBYt57dW5ALS8pDljX/wzbVr3oF371nTocDkd2/cCYPbcSfygS3veXLSk9C+UIjG/YVTCTNvMmpvZsOi6+Cej1xdXxuROdk2anc+qd97nwP6vKCgoYNnid7m6d7cjxmT+6Gpe/ccsAK7KvILXpszh0MFDbPsol4+2bOX7rVsCsC9vPwDpVdNJr5qOx/wHL+7q1z+HazKvZOwLE4v7fv7zm3jkD38u/r395OPdAOTl7SseU/20U7/+vXen2inVyMioSrVqGaRXrcquXZ9U3kGEKuaZdplB28yGARMoWgS+lKKznwaMN7OyFpl/J2xcv4m2HS+jRs0zOeXUalxxdSfq1a9bvL1Nh8vY/fFuPtxSdDFU3Xp12J67s3j7jtxd1D2nDlD0T+kp88fxz7WzWfzGEt57d03lHoyUq4cfvZ8H7nuYwhKBoXHj87i+f28WLJrKS1PG0LTp+cXbrvthT5a/O4e/v/QcQ+4YBsDSpStYtPBtPti0hA82LWHe3IV8sGFTZR9KeAo9+RagRJn2YKCtuz/s7n+N2sMU3fRkcMVP7+S2eeO/eObPL/Lc3/+XZyaMZP2ajeTnf33muvf1PXl1yuyvdyjlBl4eXQBVWFhIv+430e3S3nz/spY0a960wucvFSMzszuffLyblSuPLG9lVMvgwFdf0a1LH154fiJPjXqkeNv0V2bTpnUPBg64nfseuBuAJk0acdFFF3DxhZ1o3qwjV1zRkU6d21bqsQSpnO49crJKFLQLgfql9NeLtpWq5KWhe/d/fCLzO+m99Ldp9L/6Fm7pczufffp5cVadlpZGj95X8trLc4rH7szddUQmfk79s9m148h/7n7x+ZcsXfwOXbp3rJwDkHLXvuPlXNv7Kt5fu5Dnx46k6xUdeea5P5G7bQfTXp4JwCvTZtHykubf2nfxP5fRuPF51DqrJtf9qCfLlq4gL28feXn7mDP7Ddq2u6yyDyc4XliYdAtRoqD9G2Cemc0ws9FRm0nR7Qh/fbSd3H20u7dx9zY1Tq1TnvM96dSqXXSHxXoN6tKj95XF9euOXduxZeOH7Ny+q3js/FkL6dWvB1UzqtLgvPo0anIe7727hppn1eB7ZxSdrKp2SjU6dm3H5o3/qvRjkfLx4PDHuPjCzvxbi67cNuguFr7xFr8YfDfTp8+ha7dOAPygS3s2ZW8BijLqwy5t1ZKMjKrs2f0pOVtz6dylPWlpaaSnp9O5S3s2rM9OyTEFJeblkTJXj7j7TDO7kKJySAOK6tk5wDJ3D/PfFuVs5JhHqFHzTPLz8xlxz6N8/tkXAPTu15PpU2YdMTZ7w2ZmTJ3Lq29OoiC/gBHDHqWwsJA6dWvz8J//i7S0KphVYea0uSyY82YqDkcq0ON/HMWzY55gyNB/J+/LPIYOKTot9KO+mQwc2I9D+fkc2H+An916FwAvT5lB1ys68vbSGbg7c+cuZOaM+ak8hDDE/MG+VtGrFJqf3TbMv86kQuXm7U71FOQk9Hne5oRPbkkkb8RNScec6g+MO+Hvq2xapy0i8ZIf7yKAgraIxEvMyyMK2iISL4GeYEyWgraIxEqoS/mSpaAtIvGiTFtEJCAK2iIiAQn08vRkKWiLSKwkevZj6BS0RSReFLRFRAKi1SMiIgFRpi0iEhAFbRGRcHiByiMiIuFQpi0iEg4t+RMRCYmCtohIQOJd0lbQFpF48fx4R20FbRGJl3jHbAVtEYkXnYgUEQmJMm0RkXAo0xYRCYkybRGRcHh+qmdQsRS0RSRWXJm2iEhAYh60q6R6AiIi5ckLk2/JMLM0M1thZtOj9y+Y2RYzWxm1VlG/mdlIM8s2s/fMrHWJzxhkZhujNqhE/+Vm9n60z0gzs0TzUdAWkVgp76AN/BpY942+37p7q6itjPquBZpFLQsYBWBmtYDhQHugHTDczGpG+4yKxh7eLzPRZBS0RSRWvMCSbomYWUOgN/BsEl/dB3jRi7wN1DCzesA1wBx33+PunwJzgMxo2xnu/pa7O/Ai0DfRlyhoi0islHOm/QTwO75dKf/vqATyuJlVi/oaAFtLjMmJ+srqzymlv0wK2iISK15oSTczyzKz5SVa1uHPMbPrgF3u/s43vuJeoDnQFqgFDDu8S2nTOY7+Mmn1iIjEyrEs+XP30cDoo2zuDPzIzHoBpwBnmNlf3f3maPtXZvY88J/R+xzg3BL7NwRyo/5u3+hfEPU3LGV8mZRpi0isuFvSrezP8XvdvaG7nw8MAOa7+81RLZpopUdfYHW0yzTg1mgVSQfgM3ffDswCeppZzegEZE9gVrTtCzPrEH3WrcDURMenTFtEYqUSLq4ZZ2Z1KCpvrAR+GfW/BvQCsoF9wG0A7r7HzH4PLIvGjXD3PdHrO4AXgFOBGVErkxWdtKw4zc9uG++7t8hxyc3bneopyEno87zNiZd0JPBRm6uSjjnnLZ93wt9X2ZRpi0iseGFwcfiYKGiLSKwoaIuIBKSCK74pp6AtIrGiTFtEJCCJlvKFTkFbRGKlIIl7ioRMQVtEYkWZtohIQFTTFhEJiFaPiIgERJm2iEhACgrjfR88BW0RiRWVR0REAlKo1SMiIuHQkj8RkYCoPHKCsvcmfHqOfAftz12U6ilITKk8IiISEK0eEREJSMyrIwraIhIvKo+IiAREq0dERAJS8Q9jTy0FbRGJFUeZtohIMPJVHhERCYcybRGRgKimLSISEGXaIiIBUaYtIhKQAmXaIiLhiPnTxhS0RSReCpVpi4iEQzeMEhEJiE5EiogEpNBUHhERCUZBqidQwRS0RSRWtHpERCQgWj0iIhKQuK8eifcTMEXkO6fQkm9lMbNTzGypma0yszVm9mDU39jMlpjZRjObaGYZUX+16H12tP38Ep91b9S/wcyuKdGfGfVlm9k9yRyfgraIxErhMbQEvgK6u/ulQCsg08w6AI8Aj7t7M+BTYHA0fjDwqbtfADwejcPMWgADgJZAJvC0maWZWRrwFHAt0AIYGI0tk4K2iMRKgSXfyuJFvozeVo2aA92ByVH/WKBv9LpP9J5o+1VmZlH/BHf/yt23ANlAu6hlu/tmdz8ITIjGlklBW0Ri5VgybTPLMrPlJVpWyc+KMuKVwC5gDrAJ2Ovu+dGQHKBB9LoBsBUg2v4ZcFbJ/m/sc7T+MulEpIjEyrFcEenuo4HRZWwvAFqZWQ1gCnBxacOiX0vL3b2M/tKS5oTnURW0RSRWKuIRke6+18wWAB2AGmaWHmXTDYHcaFgOcC6QY2bpwJnAnhL9h5Xc52j9R6XyiIjESnmdiDSzOlGGjZmdClwNrANeB26Ihg0Cpkavp0XvibbPd3eP+gdEq0saA82ApcAyoFm0GiWDopOV0xIdnzJtEYmVcryMvR4wNlrlUQWY5O7TzWwtMMHMHgJWAM9F458D/mJm2RRl2AMA3H2NmU0C1gL5wJCo7IKZDQVmAWnAGHdfk2hSVvQXQcVJz2gQ97Xuchz25y5K9RTkJFS1dpMTLm48ed7NScecX3/01+Aun1SmLSKxoluziogEREFbRCQgca/HKmiLSKzo1qwiIgHRQxBERAJSGPMCiYK2iMSKTkSKiAQk3nm2graIxIwybRGRgORbvHNtBW0RiZV4h2wFbRGJGZVHREQCoiV/IiIBiXfIVtAWkZhReUREJCAFMc+1FbRFJFaUaYuIBMSVaYuIhCPumbaexn4CGjasz9zZf+f99xawauV87hw6GID+/a9j1cr5HDywlctbf794/MCB/Vi+bHZxO3hgK5de2hKA348YxpZNy9i754OUHIucmC0f5tB/0JDi1r7H9fxl4pTi7c//bTKXdL6WT/d+BsDSd9+jQ8/+xeNHjRkHwPadH3Pb0GH88KdZ9Lnpdv4y6eXiz5g1fxF9brqdf/tBL1av08/J0RTiSbcQKdM+Afn5+fz2dw+yYuVqTj+9OkuXzGTuvIWsWbOeG3/8C0Y99fAR48ePn8L48UV/kC+5pDn/mDyGVauKHr48ffocnnr6edavfbPSj0NOXONGDXlp7FMAFBQU0L3vLVx1RSegKBC/tWwF9eqefcQ+rS+9hKcfe/CIvvS0NH575y9ocdEF5OXt48eD76JT28to2rgRFzRpxBP/cz8PPjaycg4qUGGG4uQp0z4BO3bsYsXK1QB8+WUe69dvpEH9c1i/PpsPPthU5r4DftKXiZOmFr9fsvRdduzYVaHzlcrx9vKVnNugHvXPqQvAoyP/H3f/ajCWxBNV6tSuRYuLLgCgevXTaNLoXHZ+vBuApuefR+NGDSts3nGRjyfdQnTcQdvMbivPiYSuUaOGtLr0EpYsXZHU+Btv+CETJr6ceKAEZ8a8N+h19RUAvL7obc6uU5vmzZp8a9yq1eu4ftCv+OV/3E/25g+/tX3b9p2s27iJ77e8qMLnHCd+DP+F6EQy7QePtsHMssxsuZktLyzMO4GvCEP16qcxaeIz3P2fw/niiy8Tjm/X9jL27d/PmjUbKmF2UpkOHTrEgjeX0LN7F/YfOMDoFycw9Oe3fGtci4uaMuelsfxj7NP8tP8PueveEUds37dvP//nvocYdtftnF69emVNPxYKj6GFqMygbWbvHaW9D9Q92n7uPtrd27h7mypV4v0Dl56ezt8nPsP48VN4+eUZSe3zkx/3YeLEqYkHSnAWvb2ciy9sSu1aNdm6bTvbcnfQf9Cv6Nl/EDs//oQb//1OPtm9h9OrV+e0004FoGunduTn5xefpDyUn89v7nuI3j2vpEe3zqk8nCDFPdNOdCKyLnAN8Ok3+g1YXCEzCswzo//IuvXZPPHk6KTGmxn9+1/HlVddX8Ezk1R4bc4CevXoBsCFTRuz8NUJxdt69h/ExOdGUrPGmXyyew9n1aqJmfH+2g0UulPjzDNwdx74wxM0aXQugwboZ+R4hJpBJytR0J4OnO7uK7+5wcwWVMiMAtK5U1tuufkG3nt/LcuXzQbg/vsfJqNaBk8+/hB16tRi2tQXWbVqDb2uuwmArl06sG3bdrZs+eiIz3r4D/cx4Cf9OO20U/nX5uWMef5vjPj9nyr9mOT47T9wgLeWrWD47+5KOHb2628yccqrpKWncUpGBo89eA9mxrurVvPKzHk0a3o+/QcNAeDXtw+ia6d2zH3jn/zh8VHs2fsZv/rtcJo3a8Lox/+7og8rOAUeZgadLPMKPsD0jAbx/j8ox2V/7qJUT0FOQlVrN0lijU3ZftqoX9Ix528fTjnh76tsWqctIrESaq06WQraIhIr3/WatohIUEK9PD1ZCtoiEisqj4iIBCTuq0cUtEUkVlQeEREJiE5EiogEJO41bd2aVURipTwfgmBmY8xsl5mtLtH3X2a2zcxWRq1XiW33mlm2mW0ws2tK9GdGfdlmdk+J/sZmtsTMNprZRDPLSDQnBW0RiRV3T7ol4QUgs5T+x929VdReAzCzFsAAoGW0z9NmlmZmacBTwLVAC2BgNBbgkeizmlF0j6fBiSakoC0isVKAJ90ScfeFwJ4kv7oPMMHdv3L3LUA20C5q2e6+2d0PAhOAPmZmQHdgcrT/WKBvoi9R0BaRWDmW8kjJe/9HLSvJrxka3aZ6jJnVjPoaAFtLjMmJ+o7Wfxaw193zv9FfJgVtEYmVYymPlLz3f9SSucfyKKAp0ArYDvwx6i/t5lN+HP1l0uoREYmVil6n7e47D782s2couoU1FGXK55YY2hDIjV6X1v8JUMPM0qNsu+T4o1KmLSKxUtFPrjGzeiXe9gMOryyZBgwws2pm1hhoBiwFlgHNopUiGRSdrJzmRWdCXwduiPYfBCR8pJUybRGJlfK8jN3MxgPdgNpmlgMMB7qZWSuKShn/Am4HcPc1ZjYJWAvkA0PcvSD6nKHALCANGOPua6KvGAZMMLOHgBXAcwnnpIcgSCroIQhSmvJ4CELnBt2Tjjn/3DZfD0EQEUkl3XtERCQgFV09SDUFbRGJFWXaIiIBifsNoxS0RSRWCjzeN2dV0BaRWFFNW0QkIKppi4gERDVtEZGAFKo8IiISDmXaIiIB0eoREZGAqDwiIhIQlUdERAKiTFtEJCDKtEVEAlJQ9NyB2FLQFpFY0WXsIiIB0WXsIiIBUaYtIhIQrR4REQmIVo+IiAREl7GLiARENW0RkYCopi0iEhBl2iIiAdE6bRGRgCjTFhEJiFaPiIgERCciRUQCovKIiEhAdEWkiEhAlGmLiAQk7jVti/vfSicTM8ty99GpnoecXPRzIceiSqon8B2TleoJyElJPxeSNAVtEZGAKGiLiAREQbtyqW4ppdHPhSRNJyJFRAKiTFtEJCAK2pXEzDLNbIOZZZvZPamej6SemY0xs11mtjrVc5FwKGhXAjNLA54CrgVaAAPNrEVqZyUngReAzFRPQsKioF052gHZ7r7Z3Q8CE4A+KZ6TpJi7LwT2pHoeEhYF7crRANha4n1O1CcickwUtCuHldKnZTsicswUtCtHDnBuifcNgdwUzUVEAqagXTmWAc3MrLGZZQADgGkpnpOIBEhBuxK4ez4wFJgFrAMmufua1M5KUs3MxgNvAReZWY6ZDU71nOTkpysiRUQCokxbRCQgCtoiIgFR0BYRCYiCtohIQBS0RUQCoqAtIhIQBW0RkYAoaIuIBOT/A6+NmP64rcZMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = LogisticRegression(C=7.7,penalty='l2')\n",
    "model.fit(tfidf_train,y_train)\n",
    "pred = model.predict(tfidf_train)\n",
    "acc_train = accuracy_score(y_train,pred)\n",
    "print('Accuracy on training set',acc_train*100,'%')\n",
    "#obtaining trainingset accuracy\n",
    "\n",
    "predict = model.predict(tfidf_test)\n",
    "acc_test = accuracy_score(y_test,predict)\n",
    "confusion = confusion_matrix(y_test,predict)\n",
    "#obtaining testset accuracy\n",
    "print('Accuracy on test set: ',acc_test*100,\"%\")\n",
    "print('confusion matrix for test data')\n",
    "Q= ssb.heatmap(confusion, annot=True, fmt=\"d\")\n",
    "#printing confusion matrix for visualization\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 233
    },
    "colab_type": "code",
    "id": "RqPU3GeYQXnh",
    "outputId": "6caf50a8-c748-4174-bb07-d1b5e994764a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<=======>  Top 10 words<=======> \n",
      "            words           coefficients\n",
      "3125         ashy  9.993292623282772e-05\n",
      "16566    dwscribe  9.952742774882282e-05\n",
      "39055      overpo   9.93693490951796e-05\n",
      "15251  disappoint       9.92824715922875\n",
      "14872      diatet  9.909617983225983e-05\n",
      "4170    barbecuas  9.899783298224324e-05\n",
      "25059       heniz  9.899783298224324e-05\n",
      "44543   recimmend  9.886826846074295e-05\n",
      "16299       drudg  9.772513205085405e-05\n",
      "55895     tresemm  9.690362121250458e-05\n"
     ]
    }
   ],
   "source": [
    "feat = tfidf_vect.get_feature_names()\n",
    "feat = np.array(feat)\n",
    "features = np.vstack((feat,np.absolute(model.coef_)))\n",
    "features= pd.DataFrame(data=features.T,columns=('words','coefficients'))\n",
    "top_feat= features.sort_values('coefficients',axis=0,inplace=False,ascending=False,kind='quicksort')\n",
    "print('<=======>  Top 10 words<=======> ')\n",
    "print(top_feat[0:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "h3kpYFDemCZn"
   },
   "source": [
    "CONCLUSION\n",
    "======\n",
    "\n",
    "\n",
    "1.  Using tfidf featurization  to get text vectors\n",
    "\n",
    "2.  Obtaining optimal parameters using hyperparameter tuning using grid search and random search .\n",
    "\n",
    "3.  Obtaining best penality (L1,L2) \n",
    "2.  Observing sparcity in  L1 and L2 regularization.\n",
    "\n",
    "5.   Checking collinearity in features using pertubation technique,since featues are  not collinear we can use these features for modeling .\n",
    "2. Obtaining  various performance measures of the model.\n",
    "5. Grid search optimal parameters are C=2.7 and PENALITY=l2,Random search optimal parameters are C=7.7 and PENALITY=l2\n",
    "\n",
    "7.     Accuracy on training set is 93.64 and on test set is  92.55 when grid search is used for tuning hyperparameters,and 94.22 , 92.52 when random search is used\n",
    "\n",
    "9.    Obtaining top most important features .\n",
    "\n",
    "  \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "Amazon_Grid_search_Random_search_final.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}