Week-1-homework.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 448,
   "id": "719ea670",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Special module written for this class\n",
    "#This provides access to data and to helper functions from previous weeks\n",
    "import lucem_illud #pip install git+git://github.com/UChicago-Computational-Content-Analysis/lucem_illud.git\n",
    "\n",
    "#All these packages need to be installed from pip\n",
    "import requests #for http requests\n",
    "import bs4 #called `beautifulsoup4`, an html parser\n",
    "import pandas #gives us DataFrames\n",
    "import docx #reading MS doc files, install as `python-docx`\n",
    "\n",
    "#Stuff for pdfs\n",
    "#Install as `pdfminer2`\n",
    "import pdfminer.pdfinterp\n",
    "import pdfminer.converter\n",
    "import pdfminer.layout\n",
    "import pdfminer.pdfpage\n",
    "\n",
    "#These come with Python\n",
    "import re #for regexs\n",
    "import urllib.parse #For joining urls\n",
    "import io #for making http requests look like files\n",
    "import json #For Tumblr API responses\n",
    "import os.path #For checking if files exist\n",
    "import os \n",
    "import csv#For making directories"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dddf5d1",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Exercise 1</font>\n",
    "<font color=\"red\">Construct cells immediately below this that describe and download webcontent relating to your anticipated final project. Use beautiful soup and at least five regular expressions to extract relevant, nontrivial *chunks* of that content (e.g., cleaned sentences, paragraphs, etc.) to a pandas `Dataframe`.</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa12f495",
   "metadata": {},
   "source": [
    "For this exercise I scrape off data from the website of one of the largest newspapers in northern India - The Tribune. I scrape off data of matrimonial ads, which in India, are published in newspapers by people seeking prospective brides and grooms. This paper is published in all major states of northern India, and for exercise 1, I scrape off the matrimonial ads for 'brides' and 'grooms' for 25th october 1998 and store them in a data frame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 449,
   "id": "b161bec9",
   "metadata": {},
   "outputs": [],
   "source": [
    "tribune_request = requests.get('https://www.tribuneindia.com/1998/98oct25/class.htm')\n",
    "\n",
    "tribune_content_Soup = bs4.BeautifulSoup(tribune_request.text, 'html.parser')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 454,
   "id": "d83a21f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "contentPTags = tribune_content_Soup.body.findAll('tr')\n",
    "\n",
    "bride_info = []\n",
    "groom_info = []\n",
    "for trTag in contentPTags:\n",
    "    for tdTag in trTag.findAll('td'):\n",
    "        if tdTag.text == 'Brides Wanted':\n",
    "            for pTag in trTag.findAll('p'):\n",
    "                new_s = re.sub(r'[\\n,\".—-]','',pTag.text) \n",
    "                new_s1 = (re.sub(r'\\r  {2,}|\\bBox|\\bTribune|\\b([A-Z]*\\d[A-Z]*){5,}\\b|\\bChandigarh|\\bPh|\\b-OL|\\s+$'\\\n",
    "                                        ,' ',new_s))\n",
    "                bride_info.append(re.sub(r'\\(\\d+\\)|\\s*$','',new_s1))\n",
    "                with open(file, 'a') as f:\n",
    "                    writer = csv.writer(f, delimiter = ',')\n",
    "                    writer.writerow([re.sub(r'\\(\\d+\\)|\\s*$','',new_s1)] )\n",
    "         \n",
    "                \n",
    "        if tdTag.text == 'Grooms Wanted':\n",
    "            for pTag in trTag.findAll('p'):\n",
    "                new_s = re.sub(r'[\\n,.—-]','',pTag.text) \n",
    "                new_s1 = (re.sub(r'\\r  {2,}|\\bBox|\\bTribune|\\b([A-Z]*\\d[A-Z]*){5,}\\b|\\bChandigarh|\\bPh|\\b-OL|\\s+$'\\\n",
    "                                        ,' ',new_s))\n",
    "                groom_info.append(re.sub(r'\"\\(\\d+\\)|\\s*$','',new_s1))\n",
    "                "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 446,
   "id": "0b29450e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Brides-wanted</th>\n",
       "      <th>Grooms-wanted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alliance invited by well established and resp...</td>\n",
       "      <td>Well settled industrialist businessman match f...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Match for Delhi based Ramgarhia Sikh boy 28/6'...</td>\n",
       "      <td>Match for Punjabi Sarswat Brahmin slim girl 23...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MBBS bride for Hindu Arora educated boy 25 5'7...</td>\n",
       "      <td>Vegetarian professional match for Jat Sikh bea...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Professionally qualified beautiful girl prefer...</td>\n",
       "      <td>Suitable match for 5'2\"/27 convent educated   ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>USA/Canada citizen match for very handsome tee...</td>\n",
       "      <td>Professionals MCAs clean shaven match for well...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Highly cultured beautiful wellqualified girl f...</td>\n",
       "      <td>Large 4 Star Hotel with International collabo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>US Citizen Jat Sikh parents seek MBBS MBA elec...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Graceful match for young handsomeSikh clean sh...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                       Brides-wanted  \\\n",
       "0   Alliance invited by well established and resp...   \n",
       "1  Match for Delhi based Ramgarhia Sikh boy 28/6'...   \n",
       "2  MBBS bride for Hindu Arora educated boy 25 5'7...   \n",
       "3  Professionally qualified beautiful girl prefer...   \n",
       "4  USA/Canada citizen match for very handsome tee...   \n",
       "5  Highly cultured beautiful wellqualified girl f...   \n",
       "6  US Citizen Jat Sikh parents seek MBBS MBA elec...   \n",
       "7  Graceful match for young handsomeSikh clean sh...   \n",
       "\n",
       "                                       Grooms-wanted  \n",
       "0  Well settled industrialist businessman match f...  \n",
       "1  Match for Punjabi Sarswat Brahmin slim girl 23...  \n",
       "2  Vegetarian professional match for Jat Sikh bea...  \n",
       "3  Suitable match for 5'2\"/27 convent educated   ...  \n",
       "4  Professionals MCAs clean shaven match for well...  \n",
       "5   Large 4 Star Hotel with International collabo...  \n",
       "6                                                NaN  \n",
       "7                                                NaN  "
      ]
     },
     "execution_count": 446,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "matrimonials = pandas.DataFrame({'Brides-wanted' : pandas.Series(bride_info), 'Grooms-wanted':pandas.Series(groom_info)})\n",
    "matrimonials"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d141642",
   "metadata": {},
   "source": [
    "\n",
    "# <font color=\"red\">Exercise 2</font>\n",
    "<font color=\"red\">Construct cells immediately below this that spider webcontent from another site with content relating to your anticipated final project. Specifically, identify urls on a core page, then follow and extract content from them into a pandas `Dataframe`. In addition, demonstrate a *recursive* spider, which follows more than one level of links (i.e., follows links from a site, then follows links on followed sites to new sites, etc.), making sure to define a reasonable endpoint so that you do not wander the web forever :-).</font>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "961c9832",
   "metadata": {},
   "source": [
    "For exercise 2, I scrape off ads from the website of the newspaper for the years 2002, and 2014, and put them in sepateare dataframes with 'brides-wanted' and 'grooms-wanted' as columns. I also save the data into csv files for the purpose of reusing it if need be."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 455,
   "id": "a298f6c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "tribune_url = 'https://www.tribuneindia.com/archive'\n",
    "base_webpage_content = requests.get(tribune_url)\n",
    "web_page_content_soup = bs4.BeautifulSoup(base_webpage_content.text, 'html.parser')\n",
    "base_url = 'https://www.tribuneindia.com'\n",
    "\n",
    "\n",
    "tagLinks = web_page_content_soup.findAll('a', href=re.compile('/2014/form'), class_=False)\n",
    "final_urls_1998 = set()\n",
    "for aTag in tagLinks[:2]:\n",
    "        relurl = aTag.get('href')\n",
    "        level_1_url = base_url+relurl \n",
    "        level_1_request = requests.get(level_1_url)\n",
    "        level_1_soup = bs4.BeautifulSoup(level_1_request.text, 'html.parser')\n",
    "        level_1_tagLinks = level_1_soup.body.findAll('a',href = re.compile(aTag.text), class_=False)\n",
    "       \n",
    "        for a1Tag in level_1_tagLinks:\n",
    "            level_2_url = a1Tag.get('href')\n",
    "            level_2_request = requests.get(level_2_url)\n",
    "            level_2_soup = bs4.BeautifulSoup(level_2_request.text, 'html.parser')\n",
    "            level_2_tagLinks = level_2_soup.body.findAll('a',href = re.compile('class.htm'), class_=False)\n",
    "            for a2Tag in level_2_tagLinks:\n",
    "                final_urls_1998.add(base_url+'/'+aTag.text+re.sub(r'^..','',a2Tag.get('href')))\n",
    "                \n",
    "        break\n",
    "                "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 532,
   "id": "4a3d04e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "def getTextFromTribunePage(targetURL, file_1, file_2):\n",
    "    r = requests.get(targetURL)\n",
    "    soup = bs4.BeautifulSoup(r.text, 'html.parser')\n",
    "    lst_1 = []\n",
    "    lst_2 = []\n",
    "    contentPTags = soup.findAll('p')\n",
    "    \n",
    "   \n",
    "    if 'class-f' in targetURL:\n",
    "         for ptag in contentPTags:\n",
    "          \n",
    "            new_s = re.sub(r'[\\n,\".—-]','',ptag.text) \n",
    "            new_s1 = (re.sub(r'\\r  {2,}|\\bBox|\\bTribune|\\b([A-Z]*\\d[A-Z]*){5,}\\b|\\bChandigarh|\\bPh|\\b-OL|\\s+$'\\\n",
    "                                        ,' ',new_s))\n",
    "                \n",
    "            lst_2.append(re.sub(r'\\(\\d+\\)|\\s*$', '', new_s1))\n",
    "            with open(file_1, 'a') as f:\n",
    "                writer = csv.writer(f)\n",
    "                writer.writerow([re.sub(r'\\(\\d+\\)|\\s*$', '', new_s1)] )\n",
    "    if 'class-m' in targetURL:\n",
    "        for ptag in contentPTags:\n",
    "            new_s = re.sub(r'[\\n,\".—-]','',ptag.text) \n",
    "            new_s1 = (re.sub(r'\\r  {2,}|\\bBox|\\bTribune|\\b([A-Z]*\\d[A-Z]*){5,}\\b|\\bChandigarh|\\bPh|\\b-OL|\\s+$'\\\n",
    "                                        ,' ',new_s))\n",
    "                \n",
    "          \n",
    "            lst_1.append(re.sub(r'\\(\\d+\\)|\\s*$', '', new_s1))\n",
    "            with open(file_2, 'a') as f:\n",
    "                writer = csv.writer(f)\n",
    "                writer.writerow([re.sub(r'\\(\\d+\\)|\\s*$', '', new_s1)] )\n",
    "        \n",
    "                \n",
    "    return pandas.DataFrame({'Brides-wanted' : pandas.Series(lst_1), 'Grooms-wanted':pandas.Series(lst_2)})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 465,
   "id": "c4c818a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Starting with level 0\n",
    "base_url = 'https://www.tribuneindia.com/2014/forms/archive2.htm'\n",
    "\n",
    "url_request = requests.get(base_url)\n",
    "web_page_soup = bs4.BeautifulSoup(url_request.text,'html.parser')\n",
    "tagLinks = web_page_soup.findAll('a', href=re.compile('2002'), class_=False)\n",
    "final_urls_2002 = set()\n",
    "\n",
    "# getting all relevant links and this is the next level\n",
    "for aTag in tagLinks:\n",
    "        relurl = aTag.get('href')\n",
    "        level_1_request = requests.get(relurl)\n",
    "        level_1_soup = bs4.BeautifulSoup(level_1_request.text, 'html.parser')\n",
    "        level_1_tagLinks = level_1_soup.findAll('a',href = re.compile('class'), class_=False)\n",
    "        \n",
    "        # going further down another level and getting all relevant links for the next level \n",
    "        # and creating final set of urls\n",
    "        for a1Tag in level_1_tagLinks:\n",
    "            relurl = a1Tag.get('href')\n",
    "            level_2_url = 'https://www.tribuneindia.com/2002'+re.sub(r'^..','',relurl)\n",
    "            final_urls_2002.add(level_2_url)\n",
    "       "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 533,
   "id": "114a4b5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# getting all info for 2002 and storing in dataframe\n",
    "final_df_2002 = pandas.DataFrame()\n",
    "for url in final_urls_2002:\n",
    "    try:\n",
    "        final_df_2002 = final_df_2002.append(getTextFromTribunePage(url,'brides-wanted_2002.csv','grooms-wanted_2002.csv'),ignore_index = True)\n",
    "    except:\n",
    "        continue"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 561,
   "id": "0b4f1f15",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Brides-wanted</th>\n",
       "      <th>Grooms-wanted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>300</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Medico/nonMedico/professionally qualified wel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>301</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Wanted educated professional Jat Sikh boy for ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Alliance invited from NRI vegetarian  preferab...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>303</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Well settled Jat Sikh parents invite correspon...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>304</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Jat Khatri Sikh Doc/S/W/Engr match for Jat Sik...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8478</th>\n",
       "      <td>Wanted well educated smart girl from reputed f...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8479</th>\n",
       "      <td>Suitable beautiful tall professionally qualifi...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8480</th>\n",
       "      <td>Delhi based small Saraswat Punjabi Bhardwaj Br...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8481</th>\n",
       "      <td>Match for Brahmin handsome boy 26/58 BA D  a...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8482</th>\n",
       "      <td>Suitable match for businessman widower 48/54...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8183 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          Brides-wanted  \\\n",
       "300                                                 NaN   \n",
       "301                                                 NaN   \n",
       "302                                                 NaN   \n",
       "303                                                 NaN   \n",
       "304                                                 NaN   \n",
       "...                                                 ...   \n",
       "8478  Wanted well educated smart girl from reputed f...   \n",
       "8479  Suitable beautiful tall professionally qualifi...   \n",
       "8480  Delhi based small Saraswat Punjabi Bhardwaj Br...   \n",
       "8481  Match for Brahmin handsome boy 26/58 BA D  a...   \n",
       "8482  Suitable match for businessman widower 48/54...   \n",
       "\n",
       "                                          Grooms-wanted  \n",
       "300    Medico/nonMedico/professionally qualified wel...  \n",
       "301   Wanted educated professional Jat Sikh boy for ...  \n",
       "302   Alliance invited from NRI vegetarian  preferab...  \n",
       "303   Well settled Jat Sikh parents invite correspon...  \n",
       "304   Jat Khatri Sikh Doc/S/W/Engr match for Jat Sik...  \n",
       "...                                                 ...  \n",
       "8478                                                NaN  \n",
       "8479                                                NaN  \n",
       "8480                                                NaN  \n",
       "8481                                                NaN  \n",
       "8482                                                NaN  \n",
       "\n",
       "[8183 rows x 2 columns]"
      ]
     },
     "execution_count": 561,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_df_2002[300:]# There are several missing values, which occur because the \n",
    "                    #number of ads for men and women are not equal and can be cleaned, and \n",
    "                    #should not be an issue if a csv file is used for further analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68f1d564",
   "metadata": {},
   "source": [
    "I tried creating a general spider function, but the structure of the website has changed that the tags have changed and I had to wwrite another spider for getting data for 2014."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 462,
   "id": "7a3a99cd",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Starting with level 0\n",
    "base_url = 'https://www.tribuneindia.com/2014/forms/archive.htm'\n",
    "final_urls_2014 = []\n",
    "url_request = requests.get(base_url)\n",
    "web_page_soup = bs4.BeautifulSoup(url_request.text,'html.parser')\n",
    "tagLinks = web_page_soup.findAll('a', href=re.compile('2014'), class_=False)\n",
    "final_urls = set()\n",
    "# getting all relevant links and this is the next level\n",
    "for aTag in tagLinks:\n",
    "        relurl = aTag.get('href')\n",
    "       \n",
    "        level_1_request = requests.get(relurl)\n",
    "        level_1_soup = bs4.BeautifulSoup(level_1_request.text, 'html.parser')\n",
    "        level_1_tagLinks = level_1_soup.body.findAll('area',href = re.compile('class'), class_=False)\n",
    "         # going further down another level and getting all relevant links for the next level \n",
    "        # and creating final set of urls\n",
    "        for a1Tag in level_1_tagLinks:\n",
    "            relurl = a1Tag.get('href')\n",
    "            level_2_url = 'https://www.tribuneindia.com/2014'+re.sub(r'^..','',a1Tag.get('href'))\n",
    "            final_urls_2014.append(level_2_url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 556,
   "id": "87e4880b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://www.tribuneindia.com/2014/20141026/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141026/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141026/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141102/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141102/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141102/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141102/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141102/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141102/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141109/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141109/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141109/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141116/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141116/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141116/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141123/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141123/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141123/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140928/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140928/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140928/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140928/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140928/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140928/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141005/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141005/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141005/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141012/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141012/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141012/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141012/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141012/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141012/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141019/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141019/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141019/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20141019/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20141019/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20141019/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140907/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140907/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140907/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140907/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140907/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140907/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140914/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140914/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140914/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140928/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140928/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140928/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140727/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140727/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140727/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140727/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140727/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140727/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140810/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140810/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140810/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140817/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140817/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140817/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140817/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140817/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140817/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140824/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140824/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140824/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140824/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140824/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140824/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140629/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140629/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140629/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140629/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140629/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140629/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140706/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140706/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140706/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140706/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140706/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140706/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140706/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140706/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140706/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140713/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140713/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140713/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140713/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140713/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140713/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140720/class-m.htm\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140727/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140727/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140727/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140601/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140601/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140601/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140601/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140601/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140601/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140601/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140601/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140601/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140608/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140608/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140608/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140608/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140608/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140608/class-f.htm#2\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140615/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140615/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140615/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140615/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140615/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140615/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140615/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140615/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140615/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140622/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140622/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140622/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140622/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140622/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140622/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140622/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140622/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140622/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140622/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140622/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140622/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140629/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140629/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140629/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140504/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140504/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140504/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140504/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140504/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140504/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140504/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140504/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140504/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140504/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140504/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140504/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140511/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140511/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140511/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140511/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140511/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140511/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140511/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140511/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140511/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140511/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140511/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140511/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140518/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140518/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140518/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140518/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140518/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140518/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140518/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140518/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140518/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140518/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140518/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140518/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140525/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140525/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140525/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140525/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140525/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140525/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140525/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140525/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140525/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140525/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140525/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140525/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140406/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140406/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140406/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140413/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140413/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140413/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140413/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140413/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140413/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140413/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140413/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140413/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140420/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140420/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140420/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140420/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140420/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140420/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140420/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140420/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140420/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140420/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140420/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140420/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140330/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140330/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140330/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140223/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140302/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140302/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140302/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140302/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140302/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140302/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140309/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140309/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140309/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140309/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140309/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140309/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140126/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140202/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140202/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140202/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140202/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140202/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140202/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140202/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140209/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140209/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140209/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140209/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140209/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140209/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140209/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140216/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140216/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140216/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140216/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140223/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140223/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140223/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140223/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140223/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140223/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-m.htm\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-f.htm\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-m.htm\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-f.htm\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-m.htm\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-f.htm\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-m.htm\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-f.htm\n",
      "https://www.tribuneindia.com/2014/../2013/20131229/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140105/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140105/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140105/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140105/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140105/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140105/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140105/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140112/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140112/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140112/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140112/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140112/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140112/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140216/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140216/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140119/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140119/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140119/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140119/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140119/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140119/class-f.htm\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://www.tribuneindia.com/2014/20140119/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014ass-m.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm\n",
      "https://www.tribuneindia.com/2014ass-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140126/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140126/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140126/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm#2\n",
      "https://www.tribuneindia.com/2014/20140126/class-m.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm\n",
      "https://www.tribuneindia.com/2014/20140126/class-f.htm#2\n"
     ]
    }
   ],
   "source": [
    "# getting all info for 2014 and storing in dataframe\n",
    "final_df_2014 = pandas.DataFrame()\n",
    "for url in final_urls_2014:\n",
    "    try:\n",
    "        print(url)\n",
    "        final_df_2014 = final_df_2014.append(getTextFromTribunePage(url,'brides-wanted_2014.csv','grooms-wanted_2014.csv'),ignore_index = True)\n",
    "    except:\n",
    "        continue\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 594,
   "id": "a86bd189",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100      Saini (Pabla) beautiful fair   5'1 KV backgrou...\n",
       "101      Professionally qualified suitable match for be...\n",
       "102      Suitable match for fair beautiful Hindu Punjab...\n",
       "103      Prof qulified boy for MCA MPhil fair slim beau...\n",
       "104      Punjab Doaba Karnana Parhars educated smart gi...\n",
       "                               ...                        \n",
       "26581    LAND FOR SALELand 2 5 10 25 acres on and aroun...\n",
       "26582    Land 2 5 10 25 acres on and around LandranKhar...\n",
       "26583                                                     \n",
       "26584                                                     \n",
       "26585                                                     \n",
       "Name: Grooms-wanted, Length: 26486, dtype: object"
      ]
     },
     "execution_count": 594,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_df_2014['Grooms-wanted'][100:] #will have similar problems of missing values but can be \n",
    "                                    #used for analysis if it's a csv file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bea9ba77",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Exercise 3</font>\n",
    "<font color=\"red\">Construct cells immediately below this that extract and organize textual content from text, PDF or Word into a pandas dataframe.</font>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8971723a",
   "metadata": {},
   "source": [
    "For this exercise, I extract data from a pdf about internet censorship 'https://geography.fullerton.edu/taylor/550/internetcensorship.pdf'. In the last step, I select a few pages whose data might be relevant to me, and organize the page wise data in a data frame. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "0cf6b383",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "%PDF-1.3\n",
      "%verypdf.com\n",
      "532 0 obj\n",
      "<<\n",
      "/Linearized 1\n",
      "/O 534\n",
      "/H [ 1056413 143    ]\n",
      "/L 1155272        \n",
      "/E 1056413        \n",
      "/N 23\n",
      "/T 1144512        \n",
      ">>\n",
      "endobj\n",
      "xref\n",
      "532 44\n",
      "0000000022 00000 n\r\n",
      "0001055770 00000 n\r\n",
      "0001055928 00000 n\r\n",
      "0001056413 00000 n\r\n",
      "0001056557 00000 n\r\n",
      "0001056880 00000 n\r\n",
      "0001061662 00000 n\r\n",
      "0001062097 00000 n\r\n",
      "0001062150 00000 n\r\n",
      "0001062560 00000 n\r\n",
      "0001080168 00000 n\r\n",
      "0001080207 00000 n\r\n",
      "0001082907 00000 n\r\n",
      "0001083000 00000 n\r\n",
      "0001083497 00000 n\r\n",
      "0001083813 00000 n\r\n",
      "0001083998 00000 n\r\n",
      "0001087302 00000 n\r\n",
      "0001087481 00000 n\r\n",
      "0001087551 00000 n\r\n",
      "0001087746 00000 n\r\n",
      "0001092558 00000 n\r\n",
      "0001092847 00000 n\r\n",
      "0001093538 00000 n\r\n",
      "0001093651 00000 n\r\n",
      "0001093856 00000 n\r\n",
      "0001105624 00000 n\r\n",
      "0001105932 00000 n\r\n",
      "0001106889 00000 n\r\n",
      "0001107083 00000 n\r\n",
      "0001107289 00000 n\r\n",
      "0001117753 00000 n\r\n",
      "0001118189 00000 n\r\n",
      "0001118267 00000 n\r\n",
      "0001118472 00000 n\r\n",
      "0001131986 00000 n\r\n",
      "0001132277 00000 n\r\n",
      "0001133200 00000 n\r\n",
      "0001133407 00000 n\r\n",
      "0001143776 00000 n\r\n",
      "0001143846 00000 n\r\n",
      "0001143983 00000 \n"
     ]
    }
   ],
   "source": [
    "information_extraction_pdf = 'https://geography.fullerton.edu/taylor/550/internetcensorship.pdf'\n",
    "infoExtractionRequest = requests.get(information_extraction_pdf, stream=True)\n",
    "print(infoExtractionRequest.text[:1000])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 262,
   "id": "da9853a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "def readPDF(pdfFile, pageno):\n",
    "    codec = 'utf-8'\n",
    "    rsrcmgr = pdfminer.pdfinterp.PDFResourceManager()\n",
    "    retstr = io.StringIO()\n",
    "    layoutParams = pdfminer.layout.LAParams()\n",
    "    device = pdfminer.converter.TextConverter(rsrcmgr, retstr, laparams = layoutParams, codec = codec)\n",
    "    #We need a device and an interpreter\n",
    "    interpreter = pdfminer.pdfinterp.PDFPageInterpreter(rsrcmgr, device)\n",
    "    password = ''\n",
    "    maxpages = 0\n",
    "    caching = True\n",
    "    pagenos=set([21,22])\n",
    "    #countries()\n",
    "    for page in pdfminer.pdfpage.PDFPage.get_pages(pdfFile, [pageno], maxpages=maxpages, password=password,caching=caching, check_extractable=True):\n",
    "       \n",
    "        interpreter.process_page(page)\n",
    "        \n",
    "    device.close()\n",
    "    returnedString = retstr.getvalue()\n",
    "    returnedString = returnedString.replace(' \\n ','')\n",
    "    retstr.close()\n",
    "    \n",
    "  \n",
    "    return returnedString"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "id": "ab601c92",
   "metadata": {},
   "outputs": [],
   "source": [
    "infoExtractionBytes = io.BytesIO(infoExtractionRequest.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 270,
   "id": "77f04e53",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "pages, content = [],[]\n",
    "for page in range(14,23):\n",
    "    pages.append('page no '+str(page)) \n",
    "    content.append(re.sub(r'\\n',' ',readPDF(infoExtractionBytes,page)))\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "id": "158850a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "final_df = pandas.DataFrame({'pages':pandas.Series(pages),'content':pandas.Series(content)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "id": "93d2dfbc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pages</th>\n",
       "      <th>content</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>page no 14</td>\n",
       "      <td>GeoJournal  newspaper was arrested for posting...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>page no 15</td>\n",
       "      <td>been shut down a number of times (http://www. ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>page no 16</td>\n",
       "      <td>GeoJournal  Internet have been more explicit a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>page no 17</td>\n",
       "      <td>(Kreimer 2006). Thus, Congress has mandated th...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>page no 18</td>\n",
       "      <td>GeoJournal  The only criterion that remains fo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>page no 19</td>\n",
       "      <td>The information technology revolution, however...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>page no 20</td>\n",
       "      <td>GeoJournal  Cunningham, K. (2002). Factors inﬂ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>page no 21</td>\n",
       "      <td>MacKinnon, R. (2009). China’s censorship 2.0: ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>page no 22</td>\n",
       "      <td>GeoJournal  Zook, M. (2005a). The geography of...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        pages                                            content\n",
       "0  page no 14  GeoJournal  newspaper was arrested for posting...\n",
       "1  page no 15  been shut down a number of times (http://www. ...\n",
       "2  page no 16  GeoJournal  Internet have been more explicit a...\n",
       "3  page no 17  (Kreimer 2006). Thus, Congress has mandated th...\n",
       "4  page no 18  GeoJournal  The only criterion that remains fo...\n",
       "5  page no 19  The information technology revolution, however...\n",
       "6  page no 20  GeoJournal  Cunningham, K. (2002). Factors inﬂ...\n",
       "7  page no 21  MacKinnon, R. (2009). China’s censorship 2.0: ...\n",
       "8  page no 22  GeoJournal  Zook, M. (2005a). The geography of..."
      ]
     },
     "execution_count": 272,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8c8e6d7",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Exercise 4</font>\n",
    "\n",
    "<font color=\"red\">In the two cells immediately following, describe a possible project (e.g., it might end up being your final project, but need not be if you are still searching): **WHAT** you will analyze--the texts you will select and the social game, world and actors you intend to learn about through your analysis (<100 words); **WHY** you will analyze these texts to learn about that context--justify the rationale behind your proposed sample design for this project, based on the readings. What is the social game, social work, or social actors about whom you are seeking to make inferences? What are the virtues of your proposed sample with respect to your research questions? What are its limitations? What are alternatives? What would be a reasonable path to \"scale up\" your sample for further analysis (i.e., high-profile publication)? (<150 words)? [**Note**: your individual or collective project will change over the course of the quarter as new data and/or analysis opportunities arise or if old ones fade away.] \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a77d15fe",
   "metadata": {},
   "source": [
    "## ***What?*** \n",
    "<100 words"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8f5fbcc3",
   "metadata": {},
   "source": [
    "I am currently interested in understanding how matrimonial advertisements— advertisements that are published seeking matches with prospective brides and grooms— in newspapers have changed over the years, in northern India.  I plan on looking at archived digital ads from 1998 to 2014 with ‘the Tribune’ the largest selling English language paper in 6 major north Indian states. Some of these ads mention traits such as facial complexion, willingness to migrate, caste etc. and I want to see how the characteristics of the social game of seeking partners has changed, and how this might align with developments in and around the states."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "962ad35a",
   "metadata": {},
   "source": [
    "## ***Why?***\n",
    "<150 words"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "84804aa1",
   "metadata": {},
   "source": [
    "The tradition of seeking matrimony through advertisements has been prevalent in India for several decades. Since these advertisements elicit alliances and aim to act as a filter for prospective partners, they tend to be very descriptive and can be a good way of understanding several social dynamics such as age at which men and women get married, social desirability of a caste, importance of education in matrimony etc, and looking at them can give a great understanding of the social fabric of these states and how it has changed in tandem with other developments. Even though there are several websites that now post these ads, I stick to newspapers since I want to study how things have changed over time, from a point when there were no websites. I believe the limitations for this could be lack of representativeness of the sample, sparse nature of the data, and nuanced language used. This study can be scaled up if it is combined with the e-advertisements and multiple newspapers, to have a more comprehensive understanding of the social landscape."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2628a754",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}