LR with OvR - Best.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](inn.png)\n",
    "\n",
    "# Innoplexus Online Hiring Hackathon: Sentiment Analysis\n",
    "\n",
    "## Problem Statement\n",
    "\n",
    "### Sentiment Analysis for drugs/medicines\n",
    "Nowadays the narrative of a brand is not only built and controlled by the company that owns the brand. For this reason, companies are constantly looking out across Blogs, Forums, and other social media platforms, etc for checking the sentiment for their various products and also competitor products to learn how their brand resonates in the market. This kind of analysis helps them as part of their post-launch market research. This is relevant for a lot of industries including pharma and their drugs.\n",
    " \n",
    "\n",
    "**The challenge is that the language used in this type of content is not strictly grammatically correct. Some use sarcasm. Others cover several topics with different sentiments in one post. Other users post comments and reply and thereby indicating his/her sentiment around the topic.**\n",
    "\n",
    "Sentiment can be clubbed into 3 major buckets - **Positive, Negative and Neutral Sentiments.**\n",
    "\n",
    "\n",
    "You are provided with data containing samples of text. This text can contain one or more drug mentions. Each row contains a unique combination of the text and the drug mention. Note that the same text can also have different sentiment for a different drug.\n",
    "\n",
    "Given the text and drug name, the task is to predict the sentiment for texts contained in the test dataset. Given below is an example of text from the dataset:\n",
    "\n",
    "\n",
    "Example:\n",
    "\n",
    "*Stelara is still fairly new to Crohn's treatment. This is why you might not get a lot of replies. I've done some research, but most of the \"time to work\" answers are from Psoriasis boards. For Psoriasis, it seems to be about 4-12 weeks to reach a strong therapeutic level. The good news is, Stelara seems to be getting rave reviews from Crohn's patients. It seems to be the best med to come along since Remicade. I hope you have good success with it. My daughter was diagnosed Feb. 19/07, (13 yrs. old at the time of diagnosis), with Crohn's of the Terminal Illium. Has used Prednisone and Pentasa. Started Imuran (02/09), had an abdominal abscess (12/08). 2cm of Stricture. Started ​Remicade in Feb. 2014, along with 100mgs. of Imuran.*\n",
    "\n",
    "\n",
    "For Stelara the above text is **positive** while for Remicade the above text is **negative**.\n",
    "\n",
    "### Data Description\n",
    "**train.csv**\n",
    "Contains the labelled texts with sentiment values for a given drug\n",
    " \n",
    "|Variable|\tDefinition|\n",
    "|----|----|\n",
    "|unique_hash |Unique ID|\n",
    "|text|text pertaining to the drugs|\n",
    "|drug |drug name for which the sentiment is provided|\n",
    "|sentiment\t|(Target) 0-positive, 1-negative, 2-neutral  |\n",
    "\n",
    "\n",
    "**test.csv**\n",
    "test.csv contains texts with drug names for which the participants are expected to predict the correct sentiment\n",
    " \n",
    "\n",
    "### Evaluation Metric\n",
    "The metric used for evaluating the performance of the classification model would be macro F1-Score.\n",
    " \n",
    "\n",
    "## Public and Private Split\n",
    "\n",
    "The texts in the test data are further randomly divided into Public (40%) and Private (60%) data.\n",
    "Your initial responses will be checked and scored on the Public data.\n",
    "The final rankings would be based on your private score which will be published once the competition is over.\n",
    "\n",
    "# Approaches\n",
    "\n",
    "\n",
    "\n",
    "# Leaderboard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 122
    },
    "colab_type": "code",
    "id": "t_vnzL-5cIPm",
    "outputId": "e075797d-e89f-431c-b5ef-d79d55845f49"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code\n",
      "\n",
      "Enter your authorization code:\n",
      "··········\n",
      "Mounted at /content/gdrive/\n"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/gdrive/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Hts87_u-cM9e"
   },
   "outputs": [],
   "source": [
    "root_path = 'gdrive/My Drive/AV/AV INnoplexus/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4KMYX3wacO1U"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "import textblob\n",
    "\n",
    "import os\n",
    "# print(os.listdir(\"../input\"))\n",
    "os.environ['PYTHONHASHSEED'] = '10000'\n",
    "np.random.seed(10001)\n",
    "import random\n",
    "import tensorflow as tf\n",
    "random.seed(10002)\n",
    "session_conf = tf.ConfigProto(intra_op_parallelism_threads=6, inter_op_parallelism_threads=5)\n",
    "from keras import backend\n",
    "\n",
    "tf.set_random_seed(10003)\n",
    "backend.set_session(tf.Session(graph=tf.get_default_graph(), config=session_conf))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-dVvW7rjcXno"
   },
   "outputs": [],
   "source": [
    "train=pd.read_csv(root_path+'train.csv')\n",
    "test=pd.read_csv(root_path+'test.csv')\n",
    "s=pd.read_csv(root_path+'sample_submission.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 345,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 241
    },
    "colab_type": "code",
    "id": "SEBrBh3acZPs",
    "outputId": "64fb701e-9097-4902-82f2-aca0b663f2c2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of classes: 3\n",
      "Class distribution:\n",
      "2    0.724569\n",
      "1    0.158553\n",
      "0    0.116878\n",
      "Name: sentiment, dtype: float64\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/sklearn/preprocessing/_encoders.py:415: FutureWarning: The handling of integer data will change in version 0.22. Currently, the categories are determined based on the range [0, max(values)], while in the future they will be determined based on the unique values.\n",
      "If you want the future behaviour and silence this warning, you can specify \"categories='auto'\".\n",
      "In case you used a LabelEncoder before this OneHotEncoder to convert the categories to integers, then you can now use the OneHotEncoder directly.\n",
      "  warnings.warn(msg, FutureWarning)\n",
      "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:100: DeprecationWarning: The ``n_values_`` attribute was deprecated in version 0.20 and will be removed 0.22.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "enc = OneHotEncoder(sparse=False)\n",
    "enc.fit(train[\"sentiment\"].values.reshape(-1, 1))\n",
    "print(\"Number of classes:\", enc.n_values_[0])\n",
    "\n",
    "print(\"Class distribution:\\n{}\".format(train[\"sentiment\"].value_counts()/train.shape[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 346,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 425
    },
    "colab_type": "code",
    "id": "H5QlaOW-crSR",
    "outputId": "de60b280-35b6-41f8-901b-513477157ef8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['gilenya', 'fingolimod', 'ocrevus', 'cladribine', 'humira',\n",
       "       'tagrisso', 'lucentis', 'pan-retinal photocoagulation', 'remicade',\n",
       "       'stelara', 'ocrelizumab', 'dexamethasone', 'pemetrexed', 'cimzia',\n",
       "       'tarceva', 'nivolumab', 'tecentriq', 'ipilimumab', 'mekinist',\n",
       "       'opdivo', 'dexamethasone implant', 'eylea', 'erlotinib',\n",
       "       'alectinib', 'entyvio', 'crizotinib', 'keytruda', 'mavenclad',\n",
       "       'osimertinib', 'vedolizumab', 'atezolizumab', 'durvalumab',\n",
       "       'alimta', 'tysabri', 'avastin', 'golimumab', 'tofacitinib',\n",
       "       'ixifi', 'teriflunomide', 'ranibizumab', 'afatinib',\n",
       "       'upadacitinib', 'zykadia', 'ustekinumab', 'xalkori',\n",
       "       'pembrolizumab', 'lemtrada', 'siponimod', 'simponi', 'inflectra',\n",
       "       'entrectinib', 'yervoy', 'vitrectomy', 'bevacizumab', 'gefitinib',\n",
       "       'amjevita', 'lorlatinib', 'pemrolizumab', 'tafinlar',\n",
       "       'infliximab-dyyb', 'ozurdex', 'gilotrif', 'imfinzi', 'iressa',\n",
       "       'laser photocoagulation', 'renflexis', 'alecensa', 'pf-00547659',\n",
       "       'cyltezo', 'almita', 'macugen', 'remsima', 'necitumumab',\n",
       "       'panretinal photocoagulation', 'alunbrig', 'aflibercept',\n",
       "       'ceritinib', 'ketruda', 'certolizumab pegol', 'nivolumabb',\n",
       "       'photodynamic therapy', 'geftinib', 'trametinib', 'aubagio',\n",
       "       'ofatumumab', 'movectro', 'portrazza', 'pemetrexed disodium',\n",
       "       'arzerra', 'dabrafenib', 'giotrif', 'brigatinib', 'brolucizumab',\n",
       "       'pegaptanib', 'alemtuzumab', 'rhumab 2h7', 'filgotinib',\n",
       "       'alectnib', 'crizotnib', 'ct-p13', 'guselkumab', 'cyramza'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 346,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_drugs=train.drug.unique()\n",
    "all_drugs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "swCA2wg0HJZf"
   },
   "outputs": [],
   "source": [
    "common=np.intersect1d(train.text,test.text)\n",
    "test_common=test.query('text in @common')\n",
    "train=train.query('text not in @common')\n",
    "test=test.query('text not in @common')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 349,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 68
    },
    "colab_type": "code",
    "id": "N-maHq-NdAnp",
    "outputId": "b47d7051-0e6c-4678-9d7f-139537fcaf80"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
      "[nltk_data]   Package wordnet is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 349,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download('wordnet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 350,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 193
    },
    "colab_type": "code",
    "id": "3wOHaKzcca4g",
    "outputId": "eefba178-7d0e-4a9a-e0a6-a2394c702777"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>word_count</th>\n",
       "      <th>has_upper</th>\n",
       "      <th>after_comma</th>\n",
       "      <th>sentence_start</th>\n",
       "      <th>sentence_end</th>\n",
       "      <th>drug_count</th>\n",
       "      <th>char_count</th>\n",
       "      <th>avg_word</th>\n",
       "      <th>stopwords</th>\n",
       "      <th>numerics</th>\n",
       "      <th>word_density</th>\n",
       "      <th>punctuation_count</th>\n",
       "      <th>upper</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sentiment</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>308.270665</td>\n",
       "      <td>0.998379</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.870340</td>\n",
       "      <td>0.465154</td>\n",
       "      <td>0.340357</td>\n",
       "      <td>1847.124797</td>\n",
       "      <td>4.893720</td>\n",
       "      <td>123.085900</td>\n",
       "      <td>3.735818</td>\n",
       "      <td>5.770187</td>\n",
       "      <td>54.282010</td>\n",
       "      <td>13.423015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>251.029869</td>\n",
       "      <td>0.995221</td>\n",
       "      <td>0.001195</td>\n",
       "      <td>0.882915</td>\n",
       "      <td>0.451613</td>\n",
       "      <td>0.311828</td>\n",
       "      <td>1425.449223</td>\n",
       "      <td>4.681141</td>\n",
       "      <td>110.222222</td>\n",
       "      <td>2.487455</td>\n",
       "      <td>5.581805</td>\n",
       "      <td>40.305854</td>\n",
       "      <td>10.480287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>296.737579</td>\n",
       "      <td>0.994665</td>\n",
       "      <td>0.001000</td>\n",
       "      <td>0.899633</td>\n",
       "      <td>0.444815</td>\n",
       "      <td>0.353118</td>\n",
       "      <td>1783.860287</td>\n",
       "      <td>4.872842</td>\n",
       "      <td>116.764255</td>\n",
       "      <td>3.466155</td>\n",
       "      <td>5.747094</td>\n",
       "      <td>55.731577</td>\n",
       "      <td>13.310770</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           word_count  has_upper  ...  punctuation_count      upper\n",
       "sentiment                         ...                              \n",
       "0          308.270665   0.998379  ...          54.282010  13.423015\n",
       "1          251.029869   0.995221  ...          40.305854  10.480287\n",
       "2          296.737579   0.994665  ...          55.731577  13.310770\n",
       "\n",
       "[3 rows x 13 columns]"
      ]
     },
     "execution_count": 350,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#\n",
    "#     df[\"phrase_count\"] = df.groupby(\"drug\")[\"text\"].transform(\"count\")\n",
    "# \n",
    "\n",
    "def transform(df):\n",
    "#     df[\"phrase_count\"] = df.groupby(\"drug\")[\"text\"].transform(\"count\")\n",
    "  \n",
    "  \n",
    "    df['drug_count']=df['text'].apply(lambda x: len(np.intersect1d(x.split(),all_drugs)))\n",
    "#     df[\"word_count\"] = df[\"text\"].apply(lambda x: len(x.split()))\n",
    "    df[\"has_upper\"] = df[\"text\"].apply(lambda x: x.lower() != x).map({True:1,False:0})\n",
    "    df['upper'] = df['text'].apply(lambda x: len([x for x in x.split() if x.isupper()]))\n",
    "    df[\"sentence_end\"] = df[\"text\"].apply(lambda x: x.endswith(\".\")).map({True:1,False:0})\n",
    "    df[\"after_comma\"] = df[\"text\"].apply(lambda x: x.startswith(\",\")).map({True:1,False:0})\n",
    "    df[\"sentence_start\"] = df[\"text\"].apply(lambda x: \"A\" <= x[0] <= \"Z\").map({True:1,False:0})\n",
    "    df[\"text\"] = df[\"text\"].apply(lambda x: x.lower())\n",
    "    import string\n",
    "    punctuation=string.punctuation\n",
    "    df['word_count']=df['text'].apply(lambda x: len(str(x).split(\" \")))\n",
    "    df['char_count'] = df['text'].str.len()\n",
    "    def avg_word(sentence):\n",
    "        words = sentence.split()\n",
    "        return (sum(len(word) for word in words)/len(words))\n",
    "\n",
    "    df['avg_word'] = df['text'].apply(lambda x: avg_word(x))\n",
    "    from nltk.corpus import stopwords\n",
    "    stop = stopwords.words('english')\n",
    "\n",
    "    df['stopwords'] = df['text'].apply(lambda x: len([x for x in x.split() if x in stop]))\n",
    "    df['numerics'] = df['text'].apply(lambda x: len([x for x in x.split() if x.isdigit()]))\n",
    "    \n",
    "    df['word_density'] = df['char_count'] / (df['word_count']+1)\n",
    "    df['punctuation_count'] = df['text'].apply(lambda x: len(\"\".join(_ for _ in x if _ in punctuation))) \n",
    "#     df['drug']=pd.factorize(df['drug'])[0]\n",
    "#     df[\"text\"]=df[\"text\"].apply(lambda x: \" \".join([a for a in x.split() if a not in all_drugs]))\n",
    "    return df\n",
    "\n",
    "train = transform(train)\n",
    "test = transform(test)\n",
    "\n",
    "# dense_features = [\"phrase_count\", \"word_count\", \"has_upper\", \"after_comma\", \"sentence_start\", \"sentence_end\",'char_count','avg_word','stopwords','numerics','word_density','punctuation_count','drug','upper']\n",
    "dense_features = [ \"word_count\", \"has_upper\", \"after_comma\", \"sentence_start\", \"sentence_end\",'drug_count','char_count','avg_word','stopwords','numerics','word_density','punctuation_count','upper']\n",
    "train.groupby(\"sentiment\")[dense_features].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-yTim3HscltJ"
   },
   "outputs": [],
   "source": [
    "from nltk.stem import PorterStemmer,SnowballStemmer\n",
    "# st = SnowballStemmer('english')\n",
    "# df_train['text']=df_train['text'].apply(lambda x: \" \".join([st.stem(word) for word in x.split()]))\n",
    "# df_test['text']=df_test['text'].apply(lambda x: \" \".join([st.stem(word) for word in x.split()]))\n",
    "from nltk.stem import WordNetLemmatizer\n",
    "import re\n",
    "import nltk\n",
    "from nltk.corpus import stopwords\n",
    "def url_to_words(raw_text):\n",
    "    raw_text=raw_text.strip()\n",
    "    no_coms=re.sub(r'\\.com','',raw_text)\n",
    "    no_urls=re.sub('https?://www','',no_coms)\n",
    "    no_urls1=re.sub('https?://','',no_urls)\n",
    "    try:\n",
    "        no_encoding=no_urls1.decode(\"utf-8-sig\").replace(u\"\\ufffd\", \"?\")\n",
    "    except:\n",
    "        no_encoding = no_urls1\n",
    "    letters_only = re.sub(\"[^a-zA-Z0-9]\", \" \",no_encoding) \n",
    "    words = letters_only.split()                             \n",
    "    stops = stopwords.words('english')         \n",
    "    meaningful_words = [w for w in words if not w in stops]\n",
    "#     st = SnowballStemmer('english')\n",
    "#     d=[st.stem(word) for word in meaningful_words]\n",
    "#     lemmatizer=WordNetLemmatizer()\n",
    "#     dd=[lemmatizer.lemmatize(word) for word in d]\n",
    "    return( \" \".join( meaningful_words ))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Zrpb77mocoRn"
   },
   "outputs": [],
   "source": [
    "train['text']=train['text'].apply(url_to_words)\n",
    "test['text']=test['text'].apply(url_to_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 353,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "0loQh3CodRUV",
    "outputId": "d028403e-8e09-4b45-a1aa-99f5ec77df1d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4453, 17)"
      ]
     },
     "execution_count": 353,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.utils import shuffle\n",
    "# train=shuffle(train)\n",
    "train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 354,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "IhuNOSfmHluK",
    "outputId": "e4ce38b7-d138-4fbb-c661-d9afa2418eea"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    2999\n",
       "1     837\n",
       "0     617\n",
       "Name: sentiment, dtype: int64"
      ]
     },
     "execution_count": 354,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.sentiment.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 355,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 238
    },
    "colab_type": "code",
    "id": "qtC-LdOAdSo1",
    "outputId": "2517f3c1-3d7b-4a70-c7bc-478e22dfc684"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['drug_count',\n",
       " 'has_upper',\n",
       " 'upper',\n",
       " 'sentence_end',\n",
       " 'after_comma',\n",
       " 'sentence_start',\n",
       " 'word_count',\n",
       " 'char_count',\n",
       " 'avg_word',\n",
       " 'stopwords',\n",
       " 'numerics',\n",
       " 'word_density',\n",
       " 'punctuation_count']"
      ]
     },
     "execution_count": 355,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "rem_col=['text','sentiment','unique_hash','drug']\n",
    "col=[v for v in list(train.columns) if v not in rem_col] \n",
    "col"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jND9-_pmdT0o"
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer,TfidfVectorizer\n",
    "v_name = TfidfVectorizer(ngram_range=(1,3),stop_words=\"english\", analyzer='word',max_features=50000)\n",
    "name_tr =v_name.fit_transform(train['text'])\n",
    "name_ts =v_name.transform(test['text'])\n",
    "\n",
    "\n",
    "vc_name = TfidfVectorizer(ngram_range=(1,7),stop_words=\"english\", analyzer='char',max_features=50000)\n",
    "name_tcr =vc_name.fit_transform(train['text'])\n",
    "name_tcs =vc_name.transform(test['text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "NVfIntWfi8zK"
   },
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "l6Vy6je1dVid"
   },
   "outputs": [],
   "source": [
    "from scipy.sparse import csr_matrix\n",
    "from scipy import sparse\n",
    "final_features = sparse.hstack((train[col],name_tr,name_tcr )).tocsr()\n",
    "final_featurest = sparse.hstack((test[col],name_ts,name_tcs )).tocsr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AORjhqOedYUP"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "import math\n",
    "from sklearn.metrics import accuracy_score,f1_score,mean_squared_error,mean_squared_log_error\n",
    "X=final_features\n",
    "y=train['sentiment']\n",
    "X_train,X_val,y_train,y_val = train_test_split(X,y,test_size=0.25,random_state = 1994)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 361,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 88
    },
    "colab_type": "code",
    "id": "RQBlwdLwdZov",
    "outputId": "31d985b7-7007-4af2-e687-a3528a6cf50e"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5329982957993784\n"
     ]
    }
   ],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.linear_model import LogisticRegression,SGDClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.multiclass import OneVsRestClassifier\n",
    "from sklearn.svm import SVC\n",
    "# m=MultinomialNB(alpha=0.00000000001)\n",
    "# m.fit(X_train,y_train)\n",
    "# p=m.predict(X_val)\n",
    "\n",
    "# {2:1,1:4,0:7}\n",
    "m=OneVsRestClassifier(LogisticRegression(class_weight='balanced',random_state=1994))\n",
    "# m=OneVsRestClassifier(SGDClassifier(class_weight='balanced',loss='log'))\n",
    "m.fit(X_train,y_train)\n",
    "p=m.predict(X_val)\n",
    "\n",
    "# m=RandomForestClassifier(class_weight='balanced',random_state=1994,max_depth=17,max_features=50000)\n",
    "# m.fit(X_train,y_train)\n",
    "# p=m.predict(X_val)\n",
    "\n",
    "print(f1_score(y_val,p,average='macro'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 362,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 71
    },
    "colab_type": "code",
    "id": "89lPJq7vdbmX",
    "outputId": "a9466825-2c72-4806-c6ef-bf2d63069dd3"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "m.fit(X,y)\n",
    "pp=m.predict(final_featurest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 363,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "5YRKU6Nwddme",
    "outputId": "4b5b893c-d0cb-409a-eb55-df6b7d9656ba"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    2103\n",
       "1     483\n",
       "0     338\n",
       "Name: sentiment, dtype: int64"
      ]
     },
     "execution_count": 363,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test[\"sentiment\"] = pp\n",
    "\n",
    "test_common['sentiment']=2\n",
    "sub=test[[\"unique_hash\", \"sentiment\"]]\n",
    "sub=sub.append(test_common[[\"unique_hash\", \"sentiment\"]],ignore_index=True)\n",
    "\n",
    "\n",
    "# sub=test[[\"unique_hash\", \"sentiment\"]]\n",
    "sub[\"sentiment\"] = sub[\"sentiment\"].astype(int)\n",
    "sub.to_csv(\"subLocalOvO_LR7_colab.csv\", index=False)\n",
    "sub.sentiment.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bIqbRMjae77v"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "LR INN.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}