05_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 5: Link Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this assignment, we will use network-based algorithms, such as PageRank, to improve our search results. After the last assignment on evaluating IR systems, we go back to our PubMed dataset of scientific papers. In this dataset, we look at two graphs in particular: the co-authorship network and the citation network.\n",
    "\n",
    "The citation network is similar to the link network of the web: Citations are like web links pointing to other documents. We can therefore apply the same network-based ranking methods."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code from previous assignments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle, bz2\n",
    "from collections import defaultdict, namedtuple, Counter\n",
    "from math import log10, sqrt\n",
    "from IPython.display import display, HTML\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# show plots inline within the notebook\n",
    "%matplotlib inline\n",
    "# set plots' resolution\n",
    "plt.rcParams['savefig.dpi'] = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "Ids_file = 'data/allergy_Ids.pkl.bz2'\n",
    "Summaries_file = 'data/allergy_Summaries.pkl.bz2'\n",
    "Citations_file = 'data/allergy_Citations.pkl.bz2'\n",
    "Abstracts_file = 'data/allergy_Abstracts.pkl.bz2'\n",
    "\n",
    "Ids = pickle.load( bz2.BZ2File( Ids_file, 'rb' ) )\n",
    "Summaries = pickle.load( bz2.BZ2File( Summaries_file, 'rb' ) )\n",
    "Citations = pickle.load( bz2.BZ2File( Citations_file, 'rb' ) )\n",
    "Abstracts = pickle.load( bz2.BZ2File( Abstracts_file, 'rb' ) )\n",
    "\n",
    "paper = namedtuple( 'paper', ['title', 'authors', 'year', 'doi'] )\n",
    "\n",
    "for (id, paper_info) in Summaries.items():\n",
    "    Summaries[id] = paper( *paper_info )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def display_summary( id, show_abstract=False, show_id=True, extra_text='' ):\n",
    "    \"\"\"\n",
    "    Function for printing a paper's summary through IPython's Rich Display System.\n",
    "    Trims long author lists, and adds a link to the paper's DOI (when available).\n",
    "    \"\"\"\n",
    "    s = Summaries[id]\n",
    "    lines = []\n",
    "    title = s.title\n",
    "    if s.doi != '':\n",
    "        title = '<a href=http://dx.doi.org/{:s}>{:s}</a>'.format(s.doi, title)\n",
    "    title = '<strong>' + title + '</strong>'\n",
    "    lines.append(title)\n",
    "    authors = ', '.join( s.authors[:20] ) + ('' if len(s.authors) <= 20 else ', ...')\n",
    "    lines.append(str(s.year) + '. ' + authors)\n",
    "    if (show_abstract):\n",
    "        lines.append('<small><strong>Abstract:</strong> <em>{:s}</em></small>'.format(Abstracts[id]))\n",
    "    if (show_id):\n",
    "        lines.append('[ID: {:d}]'.format(id))\n",
    "    if (extra_text != ''):\n",
    "         lines.append(extra_text)\n",
    "    display( HTML('<br>'.join(lines)) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def tokenize(text):\n",
    "    return text.split(' ')\n",
    "\n",
    "def preprocess(tokens):\n",
    "    result = []\n",
    "    for token in tokens:\n",
    "        result.append(token.lower())\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "inverted_index = defaultdict(set)\n",
    "\n",
    "for (id, abstract) in Abstracts.items():\n",
    "    for term in preprocess(tokenize(abstract)):\n",
    "        inverted_index[term].add(id)\n",
    "\n",
    "tf_matrix = defaultdict(Counter)\n",
    "\n",
    "for (doc_id, abstract) in Abstracts.items():\n",
    "    tokens = preprocess(tokenize(abstract))\n",
    "    tf_matrix[doc_id] = Counter(tokens)\n",
    "\n",
    "def tf(t,d):\n",
    "    return float(tf_matrix[d][t])\n",
    "\n",
    "def df(t):\n",
    "    return float(len(inverted_index[t]))\n",
    "\n",
    "num_documents = float(len(Abstracts))\n",
    "\n",
    "def idf(t):\n",
    "    return log10((num_documents + 1)/(df(t) + 1))\n",
    "\n",
    "def tfidf(t,d):\n",
    "    return tf(t,d) * idf(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Co-authorship network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by building a mapping from authors to the set of identifiers of papers they authored.  We'll be using Python sets again for that purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "papers_of_author = defaultdict(set)\n",
    "\n",
    "for (id, p) in Summaries.items():\n",
    "    for a in p.authors:\n",
    "        papers_of_author[a].add(id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try it out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{21436399, 24037376}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "papers_of_author['Chawla A']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1038/nature12526>Type 2 innate lymphoid cells control eosinophil homeostasis.</a></strong><br>2013. Nussbaum JC, Van Dyken SJ, von Moltke J, Cheng LE, Mohapatra A, Molofsky AB, Thornton EE, Krummel MF, Chawla A, Liang HE, Locksley RM<br>[ID: 24037376]"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1126/science.1201475>Eosinophils sustain adipose alternatively activated macrophages associated with glucose homeostasis.</a></strong><br>2011. Wu D, Molofsky AB, Liang HE, Ricardo-Gonzalez RR, Jouihan HA, Bando JK, Chawla A, Locksley RM<br>[ID: 21436399]"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for id in papers_of_author['Chawla A']:\n",
    "    display_summary(id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now build a co-authorship network, that is a graph linking authors to the set of co-authors they have published with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "coauthors = defaultdict(set)\n",
    "\n",
    "for p in Summaries.values():\n",
    "    for a in p.authors:\n",
    "        coauthors[a].update(p.authors)\n",
    "\n",
    "# The code above results in each author being listed as having co-authored with himself/herself.\n",
    "# We remove these self-references here:\n",
    "for (a, ca) in coauthors.items():\n",
    "    ca.remove(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's try it out again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bando JK, Wu D, Mohapatra A, Nussbaum JC, Molofsky AB, Thornton EE, Jouihan HA, Cheng LE, von Moltke J, Locksley RM, Van Dyken SJ, Krummel MF, Ricardo-Gonzalez RR, Liang HE\n"
     ]
    }
   ],
   "source": [
    "print(', '.join( coauthors['Chawla A'] ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unlike a citation or link network, the edges of this co-authorship network are not directed: There is no direction (no arrow) in the link between author 'Chawla A' and 'Thornton EE', for example. With our chosen implementation, each of these links in fact appears twice in our data, as we also get 'Chawla A' as co-author when we look for 'Thornton EE':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mohapatra A, Nussbaum JC, Molofsky AB, Chawla A, Cheng LE, von Moltke J, Locksley RM, Van Dyken SJ, Krummel MF, Liang HE\n"
     ]
    }
   ],
   "source": [
    "print(', '.join( coauthors['Thornton EE'] ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With that in mind, we can calculate some basic statistics about our graph:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of nodes (authors):  137755\n",
      "Number of links (co-authorship relations):  961501\n"
     ]
    }
   ],
   "source": [
    "print('Number of nodes (authors): ', len(coauthors))\n",
    "\n",
    "# We divide by two here to account for the fact that each edge is represented twice (see above):\n",
    "coauthor_rel_count = int(sum( len(c) for c in coauthors.values() ) / 2)\n",
    "print('Number of links (co-authorship relations): ', coauthor_rel_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this data at hand, we can plot the [degree distribution](https://en.wikipedia.org/wiki/Degree_distribution) by showing the number of collaborators a scientist has published with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEGCAYAAACkQqisAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcdElEQVR4nO3dfZRU1Znv8e8voPgSFXnRIUDSGDsvmjG+cNVMnFyjuYovV8wajTom6RgmrCRGTZzMCGbl4phrFt7JVcfMRIeIER0jEhMjMSaIbzGZKApIQCWGHiHSgyM4IHLFQcHn/nF2SdFUd58+3VVFdf0+a9Wqc/bZp85TB7qfPnufs7ciAjMzsyLeUe8AzMyscTmJmJlZYU4iZmZWmJOImZkV5iRiZmaFDa53ALU2YsSIaGlpqXcYZmYNZdGiRS9HxMjO5U2XRFpaWli4cGG9wzAzayiS/lip3M1ZZmZWmJOImZkV5iRiZmaFOYmYmVlhTiJmZlaYk4iZmRXmJGJmZoU5iZiZWWFOImZmVljTPbFeKy1Tft7ltlXTT6thJGZm1eMrETMzK8xXIn3Q3dWGmVkz8JWImZkV5iRiZmaFOYmYmVlhTiJmZlaYk4iZmRXmJGJmZoVVLYlIulnSWklPl5X9vaTfS1oq6W5JQ8u2TZXULuk5SSeXlU9IZe2SppSVj5O0QNIKSXdK2r1a38XMzCqr5pXILcCETmXzgQ9FxGHAH4CpAJIOAc4FDk37fE/SIEmDgH8CTgEOAc5LdQGuBq6NiFZgAzCpit/FzMwqqNrDhhHxqKSWTmX3l60+DpyVlicCsyNiC7BSUjtwdNrWHhHPA0iaDUyUtBw4AfjLVGcWcAVwQ/9/k/7X1UOKHg7FzBpNPftEPg/8Ii2PBlaXbetIZV2VDwdeiYitncorkjRZ0kJJC9etW9dP4ZuZWV2SiKRvAFuB20tFFapFgfKKImJGRIyPiPEjR47sbbhmZtaFmo+dJakNOB04MSJKv/g7gLFl1cYAa9JypfKXgaGSBqerkfL6ZmZWIzW9EpE0AbgMOCMiNpdtmgucK2mIpHFAK/AE8CTQmu7E2p2s831uSj4Ps71PpQ24p1bfw8zMMtW8xfcO4DHg/ZI6JE0C/hHYB5gvaYmkGwEi4hlgDvAs8EvgwojYlq4yvgLMA5YDc1JdyJLRpakTfjgws1rfxczMKqvm3VnnVSju8hd9RFwFXFWh/D7gvgrlz7P9Di4zM6sDP7FuZmaFOYmYmVlhTiJmZlaYp8fNwdPgmplV5isRMzMrzEnEzMwKcxIxM7PCnETMzKwwJxEzMyvMScTMzApzEjEzs8KcRMzMrDAnETMzK8xJxMzMCnMSMTOzwpxEzMysMCcRMzMrzEnEzMwKcxIxM7PCnETMzKwwJxEzMyvMScTMzApzEjEzs8KqlkQk3SxpraSny8qGSZovaUV63z+VS9L1ktolLZV0ZNk+ban+CkltZeVHSVqW9rlekqr1XczMrLJqXoncAkzoVDYFeDAiWoEH0zrAKUBrek0GboAs6QDTgGOAo4FppcST6kwu26/zsczMrMp6TCKS3itpSFo+XtLFkob2tF9EPAqs71Q8EZiVlmcBZ5aV3xqZx4GhkkYBJwPzI2J9RGwA5gMT0rZ9I+KxiAjg1rLPMjOzGslzJfJjYJukg4GZwDjghwWPd2BEvAiQ3g9I5aOB1WX1OlJZd+UdFcorkjRZ0kJJC9etW1cwdDMz6yxPEnkrIrYCnwSui4ivAaP6OY5K/RlRoLyiiJgREeMjYvzIkSMLhmhmZp3lSSJvSjoPaAPuTWW7FTzeS6kpivS+NpV3AGPL6o0B1vRQPqZCuZmZ1dDgHHUuAL4IXBURKyWNA/6l4PHmkiWj6en9nrLyr0iaTdaJvjEiXpQ0D/h2WWf6ScDUiFgvaZOkY4EFwGeB7xaMaZfRMuXnXW5bNf20GkZiZpZPt0lE0iDg8oj4dKksIlaSJYFuSboDOB4YIamD7C6r6cAcSZOAF4CzU/X7gFOBdmAzWeIiJYtvAU+meldGRKmz/ktkd4DtCfwivczMrIa6TSIRsU3SSEm7R8QbvfngiDivi00nVqgbwIVdfM7NwM0VyhcCH+pNTGZm1r/yNGetAv5V0lzgtVJhRFxTraDMzKwx5Ekia9LrHcA+1Q3HzMwaSY9JJCL+DkDS3hHxWk/1zcyseeR5Yv0jkp4Flqf1D0v6XtUjMzOzXV6e50SuIxt+5D8BIuJ3wMeqGZSZmTWGPH0iRMTqToPkbqtOOPXT3TMaZmZWWZ4kslrSnwEhaXfgYlLTlpmZNbc8zVlfJHuGozTo4eF08UyHmZk1lzx3Z70MnF+DWMzMrMH0mEQkjQS+ALSU14+Iz1cvLDMzawR5+kTuAX4NPMAA7FA3M7Pi8iSRvSLisqpHYmZmDSdPx/q9kk6teiRmZtZwurwSkbSJ7bMIXi5pC/BmWo+I2Lc2IZqZ2a6qyyQSER5s0czMupVn7KxPStqvbH2opDOrG5aZmTWCPH0i0yJiY2klIl4hm6XQzMyaXJ4kUqlOrjG3zMxsYMuTRBZKukbSeyUdJOlaYFG1AzMzs11fniRyEfAGcCcwB3gdj51lZmb00CwlaRBwRUT8TY3iMTOzBtLtlUhEbAOOqlEsZmbWYPJ0kD8laS7wI+DtOdYj4idVi8rMzBpCniQyjGxq3BPKygJwEqmhrmZeXDX9tBpHYma2XZ75RC7o74NK+hrwV2TJaBlwATAKmE2WtBYDn4mINyQNAW4la1b7T+CciFiVPmcqMIlsdOGLI2Jef8dqZmZdyzOfyB5kv6gPBfYolRedT0TSaLIpdg+JiNclzQHOBU4Fro2I2ZJuTMe8Ib1viIiDJZ0LXA2cI+mQtN+hwLuAByS9L/XjmJlZDeS5xfc24E+Ak4FfAWOATX087mBgT0mDgb2AF8may+5K22cBpaFVJqZ10vYTJSmVz46ILRGxEmgHju5jXGZm1gt5ksjBEfFN4LWImAWcBvxp0QNGxL8D3wFeIEseG8keXnwlIramah1kc7qT3lenfbem+sPLyyvsswNJkyUtlLRw3bp1RUM3M7NO8iSRN9P7K5I+BOxHNlVuIZL2J7uKGEfWDLU3cEqFqlHapYttXZXvXBgxIyLGR8T4kSNH9j5oMzOrKE8SmZF+8X8TmAs8C/yfPhzzE8DKiFgXEW+S3eX1Z8DQ1LwFWZPZmrTcAYwFSNv3A9aXl1fYx8zMaqDHJBIRN0XEhoj4VUQcFBEHRMSNfTjmC8CxkvZKfRsnkiWmh4GzUp02srndIUtcbWn5LOChiIhUfq6kIZLGAa3AE32Iy8zMeinPfCIHSpop6Rdp/RBJk4oeMCIWkHWQLya7vfcdwAzgMuBSSe1kfR4z0y4zgeGp/FJgSvqcZ8jG8noW+CVwoe/MMjOrrTwPG94C/AD4Rlr/A9lgjDO72qEnETGNneckeZ4Kd1dFxH8BZ3fxOVcBVxWNw8zM+iZPn8iIiJgDvAVv3yHlv/jNzCxXEnlN0nDSnU+SjiW7zdbMzJpcnuasS8k6sd8r6V+BkWzvADczsyaWZ+ysxZL+O/B+smcznku35pqZWZPLc3fW2cCe6W6oM4E7JR1Z9cjMzGyXl6dP5JsRsUnScWTjZ80iGxjRzMyaXJ4kUroT6zTghoi4B9i9eiGZmVmjyJNE/l3SPwOfAu5L83vk2c/MzAa4PMngU8A8YEJEvEI2adTfVDUqMzNrCHnGztoMrAWOS0VbgRXVDMrMzBpDnruzppGNazU1Fe0G/Es1gzIzs8aQpznrk8AZwGsAEbEG2KeaQZmZWWPIk0TeSEOvl4Y92bu6IZmZWaPIk0TmpLuzhkr6AvAA8P3qhmVmZo0gz7An35H0P4BXyYY++V8RMb/qkZmZ2S6v2yQiaRAwLyI+AThxmJnZDrptzkozBW6WtF+N4jEzswaSZyj4/wKWSZpPukMLICIurlpUZmbWEPIkkZ+nl5mZ2Q6U3b3bPIaMao1RbdfVO4yaWDX9tHqHYGYDhKRFETG+c7kHUjQzs8KcRMzMrLAuk4ik29L7JbULx8zMGkl3VyJHSXoP8HlJ+0saVv7qy0ElDZV0l6TfS1ou6SPpc+dLWpHe9091Jel6Se2SlpZPzSupLdVfIamtLzGZmVnvdZdEbgR+CXwAWNTptbCPx/0H4JcR8QHgw8ByYArwYES0Ag+mdYBTgNb0mkyamjclsmnAMcDRwLRS4jEzs9roMolExPUR8UHg5og4KCLGlb0OKnpASfsCHwNmpuO8kSa7mkg2fzvp/cy0PBG4NTKPk43hNYpsvvf5EbE+IjaQPVE/oWhcZmbWe3nGzvqSpA8Df56KHo2IpX045kHAOuAH6XMXAZcAB0bEi+mYL0o6INUfDawu278jlXVVvhNJk8muYhi078g+hG5mZuXyTEp1MXA7cEB63S7poj4cczBwJHBDRBxB9hT8lG7qq0JZdFO+c2HEjIgYHxHjB+3lEVzMzPpLnifW/wo4JiJeA5B0NfAY8N2Cx+wAOiJiQVq/iyyJvCRpVLoKGUU2JW+p/tiy/ccAa1L58Z3KHykY04DUMqXyQAN+CNHM+kue50QEbCtb30blq4BcIuI/gNWS3p+KTgSeBeYCpTus2oB70vJc4LPpLq1jgY2p2WsecFK6c2x/4KRUZmZmNZLnSuQHwAJJd6f1M0md4n1wEVmz2O7A88AFZAltjqRJwAvA2anufcCpQDuwOdUlItZL+hbwZKp3ZUSs72NcZmbWC7nGzkrPZhxHdgXyaEQ8Ve3AqqWZxs7qipuzzKy3uho7K8+VCBGxGFjc71GZmVlD89hZZmZWmJOImZkV1m0SkTRI0gO1CsbMzBqL51g3M7PCPMe6mZkV5jnWzcyssDwDMM6StCfw7oh4rgYxmZlZg8gzAOP/BJaQzS2CpMMlza12YGZmtuvLc4vvFWSTPr0CEBFLgHFVjMnMzBpEniSyNSI2dirreawUMzMb8PJ0rD8t6S+BQZJagYuB31Y3LDMzawR5rkQuAg4FtgB3AK8CX61mUGZm1hjy3J21GfhGmowqImJT9cMyM7NGkOfurP8maRmwlOyhw99JOqr6oZmZ2a4uT5/ITODLEfFrAEnHkU1UdVg1AzMzs11fnj6RTaUEAhARvwHcpGVmZl1fiaTZDAGekPTPZJ3qAZwDPFL90MzMbFfXXXPW/+20Pq1s2c+JmJlZ10kkIj5ey0DMzKzx9NixLmko8Fmgpby+h4I3M7M8d2fdBzwOLAPeqm44ZmbWSPIkkT0i4tKqR2JmZg0nTxK5TdIXgHvJhj4BICLWVy0qq6qWKV3PMbZq+mk1jMTMGl2e50TeAP4eeAxYlF4L+3pgSYMkPSXp3rQ+TtICSSsk3Slp91Q+JK23p+0tZZ8xNZU/J+nkvsZkZma9kyeJXAocHBEtETEuvQ7qh2NfAiwvW78auDYiWoENwKRUPgnYEBEHA9emekg6BDiXbHDICcD3JA3qh7jMzCynPEnkGWBzfx5U0hjgNOCmtC7gBOCuVGUWcGZanpjWSdtPTPUnArMjYktErATaySbPMjOzGsnTJ7INWCLpYXbsE+nLLb7XAX8L7JPWhwOvRMTWtN4BjE7Lo4HV6ZhbJW1M9UeT3TVGhX12IGkyMBlg0L4j+xC2mZmVy5NEfppe/ULS6cDaiFgk6fhScYWq0cO27vbZsTBiBjADYMioVj9tb2bWT/LMJzKrpzq99FHgDEmnAnsA+5JdmQyVNDhdjYwB1qT6HcBYoEPSYGA/YH1ZeUn5PmZmVgN5nlhfSYW/8It2rkfEVGBq+uzjga9HxPmSfgScBcwG2oB70i5z0/pjaftDERGS5gI/lHQN8C6gFXiiSEy2XVe3//rWXzOrJE9z1viy5T2As4FhVYjlMmC2pP8NPEU2jwnp/TZJ7WRXIOcCRMQzkuYAzwJbgQsjYlsV4jIzsy4oovddBJJ+ExHHVSGeqhsyqjVGtV1X7zAajq9EzJqbpEURMb5zeZ7mrCPLVt9BdmWyTxfVzcysieRpziqfV2QrsAr4VFWiMTOzhpLn7izPK2JmZhXlac4aAvwFO88ncmX1wjIzs0aQpznrHmAj2cCLW3qoa2ZmTSRPEhkTEROqHomZmTWcPAMw/lbSn1Y9EjMzazh5rkSOAz6XnlzfQjZmVUTEYVWNzMzMdnl5ksgpVY/CzMwaUp5bfP9Yi0DMzKzx5LkSMfO87GZWUZ6OdTMzs4qcRMzMrDAnETMzK8xJxMzMCnMSMTOzwpxEzMysMN/ia33m23/NmpevRMzMrDAnETMzK8xJxMzMCnMSMTOzwtyxblXVVae7O9zNBoaaX4lIGivpYUnLJT0j6ZJUPkzSfEkr0vv+qVySrpfULmmppCPLPqst1V8hqa3W38XMrNnVozlrK/DXEfFB4FjgQkmHAFOAByOiFXgwrUM2n0lrek0GboAs6QDTgGOAo4FppcRjZma1UfMkEhEvRsTitLwJWA6MBiYCs1K1WcCZaXkicGtkHgeGShoFnAzMj4j1EbEBmA94Lngzsxqqa8e6pBbgCGABcGBEvAhZogEOSNVGA6vLdutIZV2VVzrOZEkLJS3ctnljf34FM7OmVreOdUnvBH4MfDUiXpXUZdUKZdFN+c6FETOAGQBDRrVWrGO15afczQaGulyJSNqNLIHcHhE/ScUvpWYq0vvaVN4BjC3bfQywpptyMzOrkXrcnSVgJrA8Iq4p2zQXKN1h1QbcU1b+2XSX1rHAxtTcNQ84SdL+qUP9pFRmZmY1Uo/mrI8CnwGWSVqSyi4HpgNzJE0CXgDOTtvuA04F2oHNwAUAEbFe0reAJ1O9KyNifW2+glWTny0xaxw1TyIR8Rsq92cAnFihfgAXdvFZNwM39190ZmbWGx72xMzMCnMSMTOzwjx2ljUM3xZstutxErEBwZ3xZvXh5iwzMyvMScTMzApzc5YNaO5HMasuX4mYmVlhTiJmZlaYk4iZmRXmPhFrWu4vMes7JxGzCvzciVk+bs4yM7PCnETMzKwwN2eZ9UJ3/ShdcROYDWROImZV5sRjA5mbs8zMrDBfiZjtgorefuy7yqzWnETMmpwTj/WFk4hZgynSx1JkH7M8nETMrN94FIDmo4iodww1NWRUa4xqu67eYZhZHzkp1ZakRRExvnO5r0TMrCHVuy/HV10ZJxEzG1Dc/1NbDZ9EJE0A/gEYBNwUEdPrHJKZNblmSmQN/bChpEHAPwGnAIcA50k6pL5RmZk1j4ZOIsDRQHtEPB8RbwCzgYl1jsnMrGk0enPWaGB12XoHcEznSpImA5PT6pY/Xn360zWIbVc2Ani53kHsAnweMj4PPgcl3Z2H91QqbPQkogplO92zHBEzgBkAkhZWuk2tmfgcZHweMj4PPgclRc5DozdndQBjy9bHAGvqFIuZWdNp9CTyJNAqaZyk3YFzgbl1jsnMrGk0dHNWRGyV9BVgHtktvjdHxDM97Daj+pHt8nwOMj4PGZ8Hn4OSXp+Hphv2xMzM+k+jN2eZmVkdOYmYmVlhTZNEJE2Q9JykdklT6h1PrUi6WdJaSU+XlQ2TNF/SivS+fz1jrDZJYyU9LGm5pGckXZLKm+087CHpCUm/S+fh71L5OEkL0nm4M92kMqBJGiTpKUn3pvVmPAerJC2TtETSwlTW65+JpkgiTT48yi3AhE5lU4AHI6IVeDCtD2Rbgb+OiA8CxwIXpn//ZjsPW4ATIuLDwOHABEnHAlcD16bzsAGYVMcYa+USYHnZejOeA4CPR8ThZc+G9PpnoimSCE08PEpEPAqs71Q8EZiVlmcBZ9Y0qBqLiBcjYnFa3kT2y2M0zXceIiL+X1rdLb0COAG4K5UP+PMgaQxwGnBTWhdNdg660eufiWZJIpWGRxldp1h2BQdGxIuQ/YIFDqhzPDUjqQU4AlhAE56H1IyzBFgLzAf+DXglIramKs3ws3Ed8LfAW2l9OM13DiD7A+J+SYvS0FBQ4GeioZ8T6YVcw6PYwCbpncCPga9GxKvZH6DNJSK2AYdLGgrcDXywUrXaRlU7kk4H1kbEIknHl4orVB2w56DMRyNijaQDgPmSfl/kQ5rlSsTDo+zoJUmjANL72jrHU3WSdiNLILdHxE9ScdOdh5KIeAV4hKyPaKik0h+UA/1n46PAGZJWkTVrn0B2ZdJM5wCAiFiT3teS/UFxNAV+JpoliXh4lB3NBdrSchtwTx1jqbrU5j0TWB4R15RtarbzMDJdgSBpT+ATZP1DDwNnpWoD+jxExNSIGBMRLWS/Bx6KiPNponMAIGlvSfuUloGTgKcp8DPRNE+sSzqV7C+O0vAoV9U5pJqQdAdwPNkQzy8B04CfAnOAdwMvAGdHROfO9wFD0nHAr4FlbG8Hv5ysX6SZzsNhZJ2lg8j+gJwTEVdKOojsr/JhwFPApyNiS/0irY3UnPX1iDi92c5B+r53p9XBwA8j4ipJw+nlz0TTJBEzM+t/zdKcZWZmVeAkYmZmhTmJmJlZYU4iZmZWmJOImZkV5iRiVkbSI5LG91yzz8e5OI0qfHu1j9VDHJ+T9K6y9VWSRtQzJmsszTLsiVnVSRpcNv5ST74MnBIRK6sZUw6fI3vIrM9PaPfy+9sA4SsRaziSWtJf8d9P82Lcn57A3uFKQtKINLxF6S/un0r6maSVkr4i6dI0p8TjkoaVHeLTkn4r6WlJR6f991Y2N8uTaZ+JZZ/7I0k/A+6vEOul6XOelvTVVHYjcBAwV9LXOtUfJOk7aZ6HpZIuSuUnpuMuS3EMqXCsd0p6UNLiVK8UY4t2nE/m65KukHQWMB64Pc0psWeqclHZZ3wg7TMsnb+l6XwdlsqvkDRD0v3ArZIOVTZnyZJUt7VX/7jWeCLCL78a6gW0kM0Rcnhan0P2hDFk40GNT8sjgFVp+XNAO7APMBLYCHwxbbuWbFDG0v7fT8sfA55Oy98uO8ZQ4A/A3ulzO4BhFeI8iuwp+b2BdwLPAEekbauAERX2+RLZGF+D0/owYA+yUajfl8puLcXbad/BwL5l372dbHDBltL3SNu+DlzR+XyVxXVRWv4ycFNa/i4wLS2fACxJy1cAi4A9y+qdn5Z3L5X7NXBfvhKxRrUyIpak5UVkvyh78nBEbIqIdWRJ5GepfFmn/e+At+di2TeNN3USMCUNo/4I2S/2d6f686Py0BDHAXdHxGuRzePxE+DPe4jxE8CNkZqF0ue+P33fP6Q6s8gSXGcCvi1pKfAA2XDmB/ZwvEpKA1SWn9fjgNtSTA8BwyXtl7bNjYjX0/JjwOWSLgPeU1ZuA5STiDWq8nGNtrG9f28r2/9f79HNPm+Vrb/Fjv2DnccCCrJf0H8R2Sxwh0fEuyOiNDPea13EWGSseVU4fsXPkXRMajZaIukM4Hyyq6yjIuJwsrHS9mDHcwI7n5fOSuel/Lx2N1z6298/In4InAG8DsyTdEIPx7IG5yRiA80qsmYk2D4qa2+dA28P3LgxIjYC88j6CpS2HZHjcx4FzpS0Vxop9ZNkA0F2537gi6VhyVNfze+BFkkHpzqfAX4VEQvKktpcYD+yuTLelPRx4D2p/kvAAZKGp76U08uOt4msiS/Pdzk/xXQ88HJEvNq5UhrY7/mIuJ5sRNjDcny2NTDfnWUDzXeAOZI+AzxU8DM2SPotsC/w+VT2LbJRoJemRLKKHX8Z7yQiFku6BXgiFd0UEU/1cOybgPel47xJ1j/zj5IuAH6UksuTwI0V9r0d+JmkhcASsuRDSipXko1avLJUntwC3CjpdeAj3cR1BfCD1FS2me3DhXd2DtmNCW8C/wFc2cP3tQbnUXzNzKwwN2eZmVlhTiJmZlaYk4iZmRXmJGJmZoU5iZiZWWFOImZmVpiTiJmZFfb/AdL7myARJa2+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist( x=[ len(ca) for ca in coauthors.values() ], bins=range(60) )\n",
    "plt.xlabel('number of co-authors')\n",
    "plt.ylabel('number of researchers')\n",
    "plt.xlim(0,51);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have explored this network, let's move to the citation network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Citations network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can look at the citation network. In contrast to the co-authorship network, the citation network is a _directed_ network, where edges can be drawn as arrows. We'll start by expanding the our data about citations into two mappings: \n",
    "\n",
    "* `papers_citing[id]`: papers citing a given paper\n",
    "* `cited_by[id]`: papers cited by a given paper (in other words: its list of references)\n",
    "\n",
    "`papers_citing` will give us the list of a node's incoming links, whereas `cited_by` will give us the list of its outgoing links."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "papers_citing = Citations  # no changes needed, this is what we are storing already in the Citations dataset\n",
    "\n",
    "cited_by = defaultdict(list)\n",
    "\n",
    "for ref, papers_citing_ref in papers_citing.items():\n",
    "    for id in papers_citing_ref:\n",
    "        cited_by[ id ].append( ref )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we are dealing with a subset of the data (the papers mentioning \"allergy\"), `papers_citing` can contain references to papers outside of our subset. On the other hand, the way we created `cited_by`, it will only contain backward references from within our dataset, meaning that it is incomplete with respect to the whole dataset. Nethertheless, we can use this citation network on our subset of allergy-related papers to implement link analysis techniques.\n",
    "\n",
    "Let us now look at an exemplary paper, let's say the one with identifier 23917667. We can now use the `cited_by` mapping to retrieve its (incomplete) list of references:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12 references found for paper 23917667\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{21771119: 'The role of genetics and environment in the rise of childhood food allergy.',\n",
       " 21624746: 'Association between genes encoding components of the IL-10/IL-0 receptor pathway and asthma in children.',\n",
       " 18440625: 'Dust mite exposure modifies the effect of functional IL10 polymorphisms on allergy and asthma exacerbations.',\n",
       " 17935766: \"The natural history of IgE-mediated cow's milk allergy.\",\n",
       " 17121586: 'Influence of SNPs in cytokine-related genes on the severity of food allergy and atopic eczema in children.',\n",
       " 16373958: \"Cow's milk allergy: a complex disorder.\",\n",
       " 16310526: 'Adverse reactions to foods.',\n",
       " 15864084: 'Genetic and environmental risk factors for the development of food allergy.',\n",
       " 15131577: \"Cow's milk-specific T-cell reactivity of children with and without persistent cow's milk allergy: key role for IL-10.\",\n",
       " 14748015: 'IL10 gene polymorphisms are associated with asthma phenotypes in children.',\n",
       " 12884306: 'Reduced transforming growth factor-beta1-producing T cells in the duodenal mucosa of children with food allergy.',\n",
       " 12592300: '9. Food allergy.'}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper_id = 23917667\n",
    "refs = { id : Summaries[id].title for id in cited_by[paper_id] }\n",
    "print(len(refs), 'references found for paper', paper_id)\n",
    "refs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we lookup the same paper in `papers_citing`, we now see that some of the cited papers are themselves in our dataset, but others are not (shown below as `'??'`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{27070332: 'Are genetic tests informative in predicting food allergy?',\n",
       " 26516307: '??',\n",
       " 25721048: '??',\n",
       " 25238536: '??'}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{ id : Summaries.get(id,['??'])[0]  for id in papers_citing[paper_id] }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Paper 26516307, for example, is not in our dataset and we do not have any direct information about it, but its repeated occurrence in other papers' citation lists does allow us to reconstruct some of its references. Below is the list of papers in our dataset cited by that paper:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4 references identified for the paper with id 26516307\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{23917667: \"Interleukin 10 (IL10) and transforming growth factor β1 (TGFβ1) gene polymorphisms in persistent IgE-mediated cow's milk allergy.\",\n",
       " 23108822: 'Interleukin-13 gene polymorphisms at -1055 C/T and +2044 G/A positions in patients with squamous cell carcinoma of head and neck.',\n",
       " 15711639: 'IL-13 R130Q, a common variant associated with allergy and asthma, enhances effector mechanisms essential for human allergic inflammation.',\n",
       " 15548720: 'Reduced immunoglobulin E and allergy among adults with glioma compared with controls.'}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper_id2 = 26516307\n",
    "refs2 = { id : Summaries[id].title for id in cited_by[paper_id2] }\n",
    "print(len(refs2), 'references identified for the paper with id', paper_id2)\n",
    "refs2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a better understanding about the data we're dealing with, let us obtain again some basic statistics about our graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of papers in our subset: 73740 (100.00 %)\n",
      "Number of papers cited at least once: 37449 (50.79 %)\n",
      "Number of isolated nodes:  34435 (46.70 %)\n"
     ]
    }
   ],
   "source": [
    "n = len(Ids)\n",
    "print('Number of papers in our subset: {:d} ({:.2f} %)'.format(n, 100.0) )\n",
    "\n",
    "with_citation = [ id for id in Ids if papers_citing[id] != [] ]\n",
    "with_citation_rel = 100. * len(with_citation) / n\n",
    "print('Number of papers cited at least once: {:d} ({:.2f} %)'.format(len(with_citation), with_citation_rel) )\n",
    "\n",
    "isolated = set( id for id in Ids if papers_citing[id] == [] and id not in cited_by )\n",
    "isolated_rel = 100. * len(isolated) / n\n",
    "print('Number of isolated nodes:  {:d} ({:.2f} %)'.format(len(isolated), isolated_rel) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overall number of nodes: 198687 (100.00 %)\n",
      "Number of non-isolated nodes: 164252 (82.67 %)\n",
      "Number of nodes outside our subset: 124947 (62.89 %)\n"
     ]
    }
   ],
   "source": [
    "id_set = set( Ids )\n",
    "citing_set = set( cited_by.keys() )\n",
    "\n",
    "outsiders = citing_set - id_set   # set difference\n",
    "nodes = citing_set | id_set   # set union\n",
    "non_isolated = nodes - isolated   # set difference\n",
    "\n",
    "print('Overall number of nodes: {:d} ({:.2f} %)'.format(len(nodes), 100.0) )\n",
    "\n",
    "non_isolated_rel = 100. * len(non_isolated) / len(nodes)\n",
    "print('Number of non-isolated nodes: {:d} ({:.2f} %)'.format(len(non_isolated), non_isolated_rel) )\n",
    "\n",
    "outsiders_rel = 100. * len(outsiders) / len(nodes)\n",
    "print('Number of nodes outside our subset: {:d} ({:.2f} %)'.format( len(outsiders), outsiders_rel ) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overall number of links (citations): 309385 (100.00 %)\n",
      "Citations outside the subset: 221506 (71.60 %)\n"
     ]
    }
   ],
   "source": [
    "all_citations = [ c for citing in papers_citing.values() for c in citing ]\n",
    "outsider_citations = [ c for citing in papers_citing.values() for c in citing if c in outsiders ]\n",
    "\n",
    "print('Overall number of links (citations): {:d} ({:.2f} %)'.format(len(all_citations), 100.0) )\n",
    "\n",
    "outsider_citations_rel = 100. * len(outsider_citations) / len(all_citations)\n",
    "print('Citations outside the subset: {:d} ({:.2f} %)'.format(len(outsider_citations), outsider_citations_rel) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now find our which 10 papers are the most cited in our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1016/j.cell.2006.02.015>Pathogen recognition and innate immunity.</a></strong><br>2006. Akira S, Uematsu S, Takeuchi O<br>[ID: 16497588]<br>Citation count: 2655"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1056/NEJMoa1105243>Prevention of HIV-1 infection with early antiretroviral therapy.</a></strong><br>2011. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, Hakim JG, Kumwenda J, Grinsztejn B, Pilotto JH, Godbole SV, Mehendale S, Chariyalertsak S, Santos BR, Mayer KH, Hoffman IF, Eshleman SH, Piwowar-Manning E, Wang L, Makhema J, ...<br>[ID: 21767103]<br>Citation count: 2519"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1038/35047123>A Toll-like receptor recognizes bacterial DNA.</a></strong><br>2000. Hemmi H, Takeuchi O, Kawai T, Kaisho T, Sato S, Sanjo H, Matsumoto M, Hoshino K, Wagner H, Takeda K, Akira S<br>[ID: 11130078]<br>Citation count: 1357"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1038/nri3073>Protective and pathogenic functions of macrophage subsets.</a></strong><br>2011. Murray PJ, Wynn TA<br>[ID: 21997792]<br>Citation count: 1073"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1038/nri2515>The gut microbiota shapes intestinal immune responses during health and disease.</a></strong><br>2009. Round JL, Mazmanian SK<br>[ID: 19343057]<br>Citation count: 1049"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1126/science.1198469>Induction of colonic regulatory T cells by indigenous Clostridium species.</a></strong><br>2011. Atarashi K, Tanoue T, Shima T, Imaoka A, Kuwahara T, Momose Y, Cheng G, Yamasaki S, Saito T, Ohba Y, Taniguchi T, Takeda K, Hori S, Ivanov II, Umesaki Y, Itoh K, Honda K<br>[ID: 21205640]<br>Citation count: 916"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1086/588660>Revised definitions of invasive fungal disease from the European Organization for Research and Treatment of Cancer/Invasive Fungal Infections Cooperative Group and the National Institute of Allergy and Infectious Diseases Mycoses Study Group (EORTC/MSG) Consensus Group.</a></strong><br>2008. De Pauw B, Walsh TJ, Donnelly JP, Stevens DA, Edwards JE, Calandra T, Pappas PG, Maertens J, Lortholary O, Kauffman CA, Denning DW, Patterson TF, Maschmeyer G, Bille J, Dismukes WE, Herbrecht R, Hope WW, Kibbler CC, Kullberg BJ, Marr KA, ...<br>[ID: 18462102]<br>Citation count: 866"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1146/annurev.immunol.21.120601.141122>Naturally arising CD4+ regulatory t cells for immunologic self-tolerance and negative control of immune responses.</a></strong><br>2004. Sakaguchi S<br>[ID: 15032588]<br>Citation count: 800"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1146/annurev.immunol.25.022106.141623>Regulatory T cells: mechanisms of differentiation and function.</a></strong><br>2012. Josefowicz SZ, Lu LF, Rudensky AY<br>[ID: 22224781]<br>Citation count: 736"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1056/NEJMoa1506816>Initiation of Antiretroviral Therapy in Early Asymptomatic HIV Infection.</a></strong><br>2015. INSIGHT START Study Group., Lundgren JD, Babiker AG, Gordin F, Emery S, Grund B, Sharma S, Avihingsanon A, Cooper DA, Fätkenheuer G, Llibre JM, Molina JM, Munderi P, Schechter M, Wood R, Klingman KL, Collins S, Lane HC, Phillips AN, Neaton JD<br>[ID: 26192873]<br>Citation count: 685"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "citation_count_per_paper = [ (id, len(citations)) for (id,citations) in papers_citing.items() ]\n",
    "sorted_by_citation_count = sorted(citation_count_per_paper, key=lambda i:i[1], reverse=True)\n",
    "\n",
    "for (id, c) in sorted_by_citation_count[:10]:\n",
    "    display_summary(id, extra_text = 'Citation count: ' + str(c))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we can start calculating some interesting network metrics, we will first have a closer look at the Python package that we are going to use for that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Link Analysis for Search Engines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to use the citation network, we need to be able to perform some complex graph algorithms on it. To make our lives easier, we will use [NetworkX](https://pypi.python.org/pypi/networkx), a Python package for dealing with complex networks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: networkx in /Users/andrewphilipharrison/anaconda3/lib/python3.7/site-packages (2.4)\n",
      "Requirement already satisfied: decorator>=4.3.0 in /Users/andrewphilipharrison/anaconda3/lib/python3.7/site-packages (from networkx) (4.4.0)\n"
     ]
    }
   ],
   "source": [
    "! pip install --user networkx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "\n",
    "G = nx.DiGraph(cited_by)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have a NetworkX Directed Graph stored in `G`, where a node represents a paper, and an edge represents a citation. This means we can now apply the [algorithms](http://networkx.github.io/documentation/networkx-1.10/reference/algorithms.html) and [functions](http://networkx.github.io/documentation/networkx-1.10/reference/functions.html) of NetworkX to our graph:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: \n",
      "Type: DiGraph\n",
      "Number of nodes: 164252\n",
      "Number of edges: 309385\n",
      "Average in degree:   1.8836\n",
      "Average out degree:   1.8836\n",
      "Directed graph: True\n",
      "Density of graph: 1.14678120125452e-05\n"
     ]
    }
   ],
   "source": [
    "print(nx.info(G))\n",
    "print('Directed graph:', nx.is_directed(G))\n",
    "print('Density of graph:', nx.density(G))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As this graph was generated from citations only, we need to add all isolated nodes (nodes that are not cited and do not cite other papers) as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "G.add_nodes_from(isolated)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we get slightly different values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: \n",
      "Type: DiGraph\n",
      "Number of nodes: 198687\n",
      "Number of edges: 309385\n",
      "Average in degree:   1.5571\n",
      "Average out degree:   1.5571\n",
      "Directed graph: True\n",
      "Density of graph: 7.837228966726296e-06\n"
     ]
    }
   ],
   "source": [
    "print(nx.info(G))\n",
    "print('Directed graph:', nx.is_directed(G))\n",
    "print('Density of graph:', nx.density(G))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to use this package for our tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tasks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Your name:** Andrew Harrison"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 1\n",
    "\n",
    "Plot the in-degree distribution (that is, the distribution of the number of incoming links; see [here](https://en.wikipedia.org/wiki/Degree_distribution) and [here](http://mathinsight.org/degree_distribution) for more detailed explanations) for the citation network. What can you tell about the shape of this distribution, and what does this tell us about the network?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'number of nodes')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEGCAYAAACpXNjrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAfYUlEQVR4nO3df5QdZZ3n8ffHRH74AxIgOJhEE8aWFRjF0ELwB4tkDAFdgrMwBnVpMTs5Yvi17MwQlj2GxeEI44xo/IFG0xI8DAEZkewYDFl+yHqEkAYCIURMG5C0iaQxMaKsMInf/aOehuKmbnd1d9177ebzOqfOrfrWU1XPvd19v/1UPfWUIgIzM7MqvarVFTAzs9HHycXMzCrn5GJmZpVzcjEzs8o5uZiZWeXGtroCfyoOOuigmDJlSqurYWY2ojzwwAPPRMSE2riTSzJlyhS6urpaXQ0zsxFF0i+K4j4tZmZmlXNyMTOzyjm5mJlZ5ZxczMysck4uZmZWOScXMzOrnJOLmZlVrmHJRVKnpG2SHq2JnyfpcUnrJf1jLn6JpO607qRcfFaKdUtakItPlbRa0kZJN0raK8X3Tsvdaf2URr1HMzMr1siWy7XArHxA0vuB2cDbI+II4J9S/HBgDnBE2uZrksZIGgN8FTgZOBw4M5UFuAq4OiLagB3A3BSfC+yIiLcAV6dyZmbWRA27Qz8i7iloNZwDXBkRz6cy21J8NrAsxZ+Q1A0ck9Z1R8QmAEnLgNmSNgAnAh9NZZYClwHXpH1dluI3A1+RpGjgU9GmLPjBkLd98soPVlgTM7M/Dc2+5vJW4H3pdNWPJL0rxScCm3PlelKsXvxA4DcRsasm/rJ9pfU7U/k9SJonqUtSV29v77DfnJmZZZqdXMYC44HpwN8BN0kSoIKyMYQ4A6x7eTBicUS0R0T7hAl7jLtmZmZD1Ozk0gN8LzL3A38EDkrxyblyk4At/cSfAcZJGlsTJ79NWr8/sL0h78bMzAo1O7l8n+xaCZLeCuxFliiWA3NST6+pQBtwP7AGaEs9w/Yiu+i/PF0/uQs4Pe23A7g1zS9Py6T1dzbyeouZme2pYRf0Jd0AnAAcJKkHWAh0Ap2pe/ILQEf64l8v6SbgMWAXMD8idqf9nAusBMYAnRGxPh3iYmCZpH8AHgKWpPgS4DupU8B2soRkZmZN1MjeYmfWWfXxOuWvAK4oiK8AVhTEN/FSj7J8/A/AGYOqrJmZVcp36JuZWeWcXMzMrHJOLmZmVjknFzMzq5yTi5mZVc7JxczMKufkYmZmlXNyMTOzyjm5mJlZ5ZxczMysck4uZmZWOScXMzOrnJOLmZlVzsnFzMwq5+RiZmaVc3IxM7PKObmYmVnlGpZcJHVK2pYeaVy77m8lhaSD0rIkLZLULekRSdNyZTskbUxTRy5+tKR1aZtFkpTiB0halcqvkjS+Ue/RzMyKNbLlci0wqzYoaTLwAeCpXPhkoC1N84BrUtkDgIXAsWSPNF6YSxbXpLJ92/UdawFwR0S0AXekZTMza6KGJZeIuAfYXrDqauDvgcjFZgPXReY+YJykQ4CTgFURsT0idgCrgFlp3X4RcW9EBHAdcFpuX0vT/NJc3MzMmqSp11wknQr8MiIerlk1EdicW+5Jsf7iPQVxgDdExFaA9HpwP/WZJ6lLUldvb+8Q3pGZmRVpWnKR9BrgUuAzRasLYjGE+KBExOKIaI+I9gkTJgx2czMzq6OZLZc/B6YCD0t6EpgEPCjpz8haHpNzZScBWwaITyqIAzydTpuRXrdV/k7MzKxfTUsuEbEuIg6OiCkRMYUsQUyLiF8By4GzUq+x6cDOdEprJTBT0vh0IX8msDKte1bS9NRL7Czg1nSo5UBfr7KOXNzMzJqkkV2RbwDuBQ6T1CNpbj/FVwCbgG7gm8CnASJiO/BZYE2aLk8xgHOAb6Vtfg7cluJXAh+QtJGsV9qVVb4vMzMb2NhG7Tgizhxg/ZTcfADz65TrBDoL4l3AkQXxXwMzBlldMzOrkO/QNzOzyjm5mJlZ5ZxczMysck4uZmZWOScXMzOrnJOLmZlVzsnFzMwq5+RiZmaVc3IxM7PKObmYmVnlnFzMzKxyTi5mZlY5JxczM6uck4uZmVXOycXMzCrn5GJmZpVzcjEzs8o18jHHnZK2SXo0F/u8pJ9KekTSLZLG5dZdIqlb0uOSTsrFZ6VYt6QFufhUSaslbZR0o6S9UnzvtNyd1k9p1Hs0M7NijWy5XAvMqomtAo6MiLcDPwMuAZB0ODAHOCJt8zVJYySNAb4KnAwcDpyZygJcBVwdEW3ADmBuis8FdkTEW4CrUzkzM2uihiWXiLgH2F4Tuz0idqXF+4BJaX42sCwino+IJ4Bu4Jg0dUfEpoh4AVgGzJYk4ETg5rT9UuC03L6WpvmbgRmpvJmZNUkrr7l8ErgtzU8ENufW9aRYvfiBwG9yiaov/rJ9pfU7U/k9SJonqUtSV29v77DfkJmZZVqSXCRdCuwCru8LFRSLIcT729eewYjFEdEeEe0TJkzov9JmZlba2GYfUFIH8CFgRkT0fen3AJNzxSYBW9J8UfwZYJyksal1ki/ft68eSWOB/ak5PWdmZo3V1JaLpFnAxcCpEfFcbtVyYE7q6TUVaAPuB9YAbaln2F5kF/2Xp6R0F3B62r4DuDW3r440fzpwZy6JmZlZEwyYXCS9VtKr0vxbJZ0q6dUltrsBuBc4TFKPpLnAV4DXA6skrZX0dYCIWA/cBDwG/BCYHxG7U6vkXGAlsAG4KZWFLEldJKmb7JrKkhRfAhyY4hcBL3ZfNjOz5ihzWuwe4H2SxgN3AF3AR4CP9bdRRJxZEF5SEOsrfwVwRUF8BbCiIL6JrDdZbfwPwBn91c3MzBqrzGkxpVNYfwV8OSI+THbPiZmZWaFSyUXScWQtlR+kWNM7ApiZ2chRJrlcSHYn/S0RsV7SoWQX083MzAoN2AKJiB8BP5L02rS8CTi/0RUzM7ORq0xvseMkPUbWWwtJ75D0tYbXzMzMRqwyp8W+CJwE/BogIh4Gjm9kpczMbGQrdRNlRGyuCe1uQF3MzGyUKNPra7OkdwOR7pI/n3SKzMzMrEiZlsungPlkow33AEelZTMzs0Jleos9wwB345uZmeXVTS6SvkydoeoBIsLdkc3MrFB/p8W6gAeAfYBpwMY0HYUv6JuZWT/qtlwiYimApE8A74+If0/LXwdub0rtzMxsRCpzQf+NZMPk93ldipmZmRUq0xX5SuAhSX3jif1H4LKG1cjMzEa8Mr3Fvi3pNuBYsgv8CyLiVw2vmZmZjVhlH3N8DPA+smFf3lVmA0mdkrZJejQXO0DSKkkb0+v4FJekRZK6JT0iaVpum45UfqOkjlz8aEnr0jaLJKm/Y5iZWfOUGbjySuACskcQPwacL+lzJfZ9LTCrJrYAuCMi2sieatn3COKTgbY0zQOuScc+AFhI1mo6BliYSxbXpLJ9280a4BhmZtYkZVoupwAfiIjOiOgk+xL/4EAbRcQ9wPaa8GxgaZpfCpyWi18XmfuAcZIOIRswc1VEbI+IHcAqYFZat19E3BsRAVxXs6+iY5iZWZOUPS02Lje//zCO94aI2AqQXg9O8YlAfnDMnhTrL95TEO/vGHuQNE9Sl6Su3t7eIb8pMzN7uTK9xT7HS73FRHbd5ZKK66GCWAwhPigRsRhYDNDe3j7o7c3MrNiALZeIuAGYDnwvTcdFxLIhHu/pdEqL9LotxXuAyblyk4AtA8QnFcT7O4aZmTVJ2dNirwKeAXYAb5U01IeFLQf6enx1ALfm4melXmPTgZ3plNZKYKak8elC/kxgZVr3rKTpqZfYWTX7KjqGmZk1yYCnxSRdBXwEWA/8MYUDuGeA7W4ATgAOktRD1uvrSuAmSXOBp4AzUvEVZB0HuoHngLMBImK7pM8Ca1K5yyOir5PAOWQ90vYFbksT/RzDzMyapMw1l9OAwyLi+cHsOCLOrLNqRkHZoM4zYlIPtc6CeBdwZEH810XHMDOz5ilzWmwT8OpGV8TMzEaPMi2X54C1ku4AXmy9+HkuZmZWT5nksjxNZmZmpZQZuHLpQGXMzMzyynZFNjMzK83JxczMKlc3uUj6Tnq9oHnVMTOz0aC/lsvRkt4MfDLdIX9AfmpWBc3MbOTp74L+14EfAocCD/DywSIjxc3MzPZQt+USEYsi4m1AZ0QcGhFTc5MTi5mZ1VWmK/I5kt5B9phjgHsi4pHGVsvMzEayMo85Ph+4nuyhWwcD10s6r9EVMzOzkavMHfr/FTg2In4PL46SfC/w5UZWzMzMRq4y97kI2J1b3k3xkyDNzMyAci2XbwOrJd2Slk8DljSuSmZmNtKVuaD/BUl3A+8la7GcHREPNbpiZmY2cpVpuRARDwIPNrguZmY2SrRkbDFJ/03SekmPSrpB0j6SpkpaLWmjpBsl7ZXK7p2Wu9P6Kbn9XJLij0s6KReflWLdkhY0/x2amb2yNT25SJoInA+0R8SRwBhgDnAVcHVEtAE7gLlpk7nAjoh4C3B1Koekw9N2RwCzgK9JGiNpDPBV4GTgcODMVNbMzJqk3+SSvqz/TwOOOxbYV9JY4DXAVuBE4Oa0filZxwGA2WmZtH6GJKX4soh4PiKeALqBY9LUHRGbIuIFYFkqa2ZmTdJvcomI3cBzkvav6oAR8Uvgn4CnyJLKTrKxy34TEbtSsR5gYpqfCGxO2+5K5Q/Mx2u2qRffg6R5krokdfX29g7/zZmZGVDugv4fgHWSVgG/7wtGxPlDOaCk8WQtianAb4Dvkp3CqhV9m9RZVy9elDCjIEZELAYWA7S3txeWMTOzwSuTXH6Qpqr8JfBERPQCSPoe8G5gnKSxqXUyCdiSyvcAk4GedBptf2B7Lt4nv029uJmZNUGZ+1yWStoXeFNEPF7BMZ8Cpkt6DfD/gBlAF3AXcDrZNZIO4NZUfnlavjetvzMiQtJy4F8kfQF4I9AG3E/WommTNBX4JdlF/49WUG8zMyupzMCV/wlYS/ZsFyQdlb7YhyQiVpNdmH8QWJfqsBi4GLhIUjfZNZW+UQCWAAem+EXAgrSf9cBNwGOpbvMjYndq+ZwLrAQ2ADelsmZm1iRlTotdRtYD626AiFibWgVDFhELgYU14U3pOLVl/wCcUWc/VwBXFMRXACuGU0czMxu6Mve57IqInTUxX/w2M7O6yrRcHpX0UWCMpDayGyB/0thqmZnZSFam5XIe2V3wzwM3AL8FLmxkpczMbGQr01vsOeDS9JCwiIhnG18tMzMbycr0FnuXpHXAI2Q3Uz4s6ejGV83MzEaqMtdclgCfjoj/CyDpvWQPEHt7IytmZmYjV5lrLs/2JRaAiPgx4FNjZmZWV92Wi6RpafZ+Sd8gu5gfwEdI97yYmZkV6e+02D/XLOdvevR9LmZmVlfd5BIR729mRczMbPQY8IK+pHHAWcCUfPmhDrlvZmajX5neYiuA+8gGmfxjY6tjZmajQZnksk9EXNTwmpiZ2ahRpivydyT9jaRDJB3QNzW8ZmZmNmKVabm8AHweuJSXeokFcGijKmVmZiNbmeRyEfCWiHim0ZUxM7PRocxpsfXAc42uiJmZjR5lkstuYK2kb0ha1DcN56CSxkm6WdJPJW2QdFy6lrNK0sb0Oj6VVTpmt6RHciMHIKkjld8oqSMXP1rSurTNIkkaTn3NzGxwyiSX75M9SvgnwAO5aTi+BPwwIv4D8A6yZ90vAO6IiDbgjrQMcDLQlqZ5wDUAqVPBQuBYsscjL+xLSKnMvNx2s4ZZXzMzG4Qyz3NZWuUBJe0HHA98Iu3/BeAFSbOBE1KxpWTjl10MzAaui4gA7kutnkNS2VURsT3tdxUwS9LdwH4RcW+KXwecBtxW5fswM7P6ytyh/wQFY4lFxFB7ix0K9ALflvQOslbQBcAbImJr2vdWSQen8hOBzbnte1Ksv3hPQXwPkuaRtXB405veNMS3Y2Zmtcr0FmvPze8DnAEM5z6XscA04LyIWC3pS7x0CqxI0fWSGEJ8z2DEYmAxQHt7uwfjNDOryIDXXCLi17nplxHxReDEYRyzB+iJiNVp+WayZPN0Ot1Fet2WKz85t/0kYMsA8UkFcTMza5IyjzmelpvaJX0KeP1QDxgRvwI2SzoshWYAjwHLgb4eXx3ArWl+OXBW6jU2HdiZTp+tBGZKGp8u5M8EVqZ1z0qannqJnZXbl5mZNUGZ02L557rsAp4E/nqYxz0PuF7SXsAm4GyyRHeTpLnAU2Sn3yAbOPMUoJvsfpuzASJiu6TPAmtSucv7Lu4D5wDXAvuSXcj3xXwzsyYq01us8ue6RMRaXn4tp8+MgrIBzK+zn06gsyDeBRw5zGqamdkQlekttjfwn9nzeS6XN65aZmY2kpU5LXYrsJOsy/Dzja2OmZmNBmWSy6SI8B3uZmZWWpnhX34i6S8aXhMzMxs1yrRc3gt8It2p/zzZTYoREW9vaM3MzGzEKpNcTm54LczMbFQp0xX5F82oiJmZjR5lrrmYmZkNipOLmZlVzsnFzMwq5+RiZmaVc3IxM7PKObmYmVnlnFzMzKxyTi5mZlY5JxczM6tcy5KLpDGSHpL0b2l5qqTVkjZKujE9pRJJe6fl7rR+Sm4fl6T445JOysVnpVi3pAXNfm9mZq90rWy5XABsyC1fBVwdEW3ADmBuis8FdkTEW4CrUzkkHQ7MAY4AZgFfSwlrDPBVsjHRDgfOTGXNzKxJWpJcJE0CPgh8Ky0LOBG4ORVZCpyW5menZdL6Gan8bGBZRDwfEU8A3cAxaeqOiE0R8QKwLJU1M7MmaVXL5YvA3wN/TMsHAr+JiF1puQeYmOYnApsB0vqdqfyL8Zpt6sX3IGmepC5JXb29vcN9T2ZmljQ9uUj6ELAtIh7IhwuKxgDrBhvfMxixOCLaI6J9woQJ/dTazMwGo8zzXKr2HuBUSacA+wD7kbVkxkkam1onk4AtqXwPMBnokTQW2B/Ynov3yW9TL25mZk3Q9JZLRFwSEZMiYgrZBfk7I+JjwF3A6alYB3Brml+elknr74yISPE5qTfZVKANuB9YA7Sl3md7pWMsb8JbMzOzpBUtl3ouBpZJ+gfgIWBJii8BviOpm6zFMgcgItZLugl4DNgFzI+I3QCSzgVWAmOAzohY39R3Ymb2CtfS5BIRdwN3p/lNZD29asv8ATijzvZXAFcUxFcAKyqsqpmZDYLv0Dczs8o5uZiZWeWcXMzMrHJOLmZmVjknFzMzq5yTi5mZVc7JxczMKufkYmZmlXNyMTOzyjm5mJlZ5ZxczMysck4uZmZWOScXMzOrnJOLmZlVzsnFzMwq5+RiZmaVc3IxM7PKNT25SJos6S5JGyStl3RBih8gaZWkjel1fIpL0iJJ3ZIekTQtt6+OVH6jpI5c/GhJ69I2iySp2e/TzOyVrBUtl13Af4+ItwHTgfmSDgcWAHdERBtwR1oGOBloS9M84BrIkhGwEDiW7PHIC/sSUiozL7fdrCa8LzMzS5qeXCJia0Q8mOafBTYAE4HZwNJUbClwWpqfDVwXmfuAcZIOAU4CVkXE9ojYAawCZqV1+0XEvRERwHW5fZmZWRO09JqLpCnAO4HVwBsiYitkCQg4OBWbCGzObdaTYv3FewriRcefJ6lLUldvb+9w346ZmSUtSy6SXgf8K3BhRPy2v6IFsRhCfM9gxOKIaI+I9gkTJgxUZTMzK6klyUXSq8kSy/UR8b0Ufjqd0iK9bkvxHmBybvNJwJYB4pMK4mZm1iSt6C0mYAmwISK+kFu1HOjr8dUB3JqLn5V6jU0HdqbTZiuBmZLGpwv5M4GVad2zkqanY52V25eZmTXB2BYc8z3AfwHWSVqbYv8DuBK4SdJc4CngjLRuBXAK0A08B5wNEBHbJX0WWJPKXR4R29P8OcC1wL7AbWkyM7MmaXpyiYgfU3xdBGBGQfkA5tfZVyfQWRDvAo4cRjXNzGwYfIe+mZlVzsnFzMwq5+RiZmaVc3IxM7PKObmYmVnlnFzMzKxyTi5mZla5VtxEaTlTFvxgWNs/eeUHK6qJmVl13HIxM7PKObmYmVnlnFzMzKxyTi5mZlY5X9Af4YbTIcCdAcysUdxyMTOzyjm5mJlZ5Xxa7BXMp9TMrFGcXGxInJjMrD+jNrlImgV8CRgDfCsirmxxlSxxYjIb/UZlcpE0Bvgq8AGgB1gjaXlEPNbamtlwDXe4nFZxUrRXmlGZXIBjgO6I2AQgaRkwG3BysZYYqUnRXhka8c/PaE0uE4HNueUe4NjaQpLmAfPS4u8kPT7E4x0EPDPEbRvJ9Roc12twXK/B+VOtF7pqWHV7c1FwtCYXFcRij0DEYmDxsA8mdUVE+3D3UzXXa3Bcr8FxvQbnT7Ve0Ji6jdb7XHqAybnlScCWFtXFzOwVZ7QmlzVAm6SpkvYC5gDLW1wnM7NXjFF5Wiwidkk6F1hJ1hW5MyLWN/CQwz611iCu1+C4XoPjeg3On2q9oAF1U8QelyLMzMyGZbSeFjMzsxZycjEzs8o5uQyCpFmSHpfULWlBwfq9Jd2Y1q+WNKUJdZos6S5JGyStl3RBQZkTJO2UtDZNn2l0vdJxn5S0Lh2zq2C9JC1Kn9cjkqY1oU6H5T6HtZJ+K+nCmjJN+bwkdUraJunRXOwASaskbUyv4+ts25HKbJTU0YR6fV7ST9PP6RZJ4+ps2+/PvAH1ukzSL3M/q1PqbNvv324D6nVjrk5PSlpbZ9tGfl6F3w1N+x2LCE8lJrKOAT8HDgX2Ah4GDq8p82ng62l+DnBjE+p1CDAtzb8e+FlBvU4A/q0Fn9mTwEH9rD8FuI3svqTpwOoW/Ex/Bby5FZ8XcDwwDXg0F/tHYEGaXwBcVbDdAcCm9Do+zY9vcL1mAmPT/FVF9SrzM29AvS4D/rbEz7nfv92q61Wz/p+Bz7Tg8yr8bmjW75hbLuW9OKRMRLwA9A0pkzcbWJrmbwZmSCq6obMyEbE1Ih5M888CG8hGKBgJZgPXReY+YJykQ5p4/BnAzyPiF0085osi4h5ge004/zu0FDitYNOTgFURsT0idgCrgFmNrFdE3B4Ru9LifWT3jjVVnc+rjDJ/uw2pV/r7/2vghqqOV1Y/3w1N+R1zcimvaEiZ2i/xF8ukP8SdwIFNqR2QTsO9E1hdsPo4SQ9Luk3SEU2qUgC3S3pA2VA7tcp8po00h/p/9K34vADeEBFbIftyAA4uKNPqz+2TZC3OIgP9zBvh3HS6rrPOKZ5Wfl7vA56OiI111jfl86r5bmjK75iTS3llhpQpNexMI0h6HfCvwIUR8dua1Q+Snfp5B/Bl4PvNqBPwnoiYBpwMzJd0fM36Vn5eewGnAt8tWN2qz6usVn5ulwK7gOvrFBnoZ161a4A/B44CtpKdgqrVss8LOJP+Wy0N/7wG+G6ou1lBbFCfmZNLeWWGlHmxjKSxwP4MrRk/KJJeTfbLc31EfK92fUT8NiJ+l+ZXAK+WdFCj6xURW9LrNuAWstMTea0cpudk4MGIeLp2Ras+r+TpvlOD6XVbQZmWfG7pou6HgI9FOjFfq8TPvFIR8XRE7I6IPwLfrHO8Vn1eY4G/Am6sV6bRn1ed74am/I45uZRXZkiZ5UBfr4rTgTvr/RFWJZ3TXQJsiIgv1CnzZ33XfiQdQ/Zz/3WD6/VaSa/vmye7IPxoTbHlwFnKTAd29jXXm6Duf5St+Lxy8r9DHcCtBWVWAjMljU+ngWamWMMoe/jexcCpEfFcnTJlfuZV1yt/je7DdY7XquGg/hL4aUT0FK1s9OfVz3dDc37HGtFLYbROZL2bfkbW8+TSFLuc7A8OYB+y0yzdwP3AoU2o03vJmquPAGvTdArwKeBTqcy5wHqyXjL3Ae9uQr0OTcd7OB277/PK10tkD3X7ObAOaG/Sz/E1ZMli/1ys6Z8XWXLbCvw72X+Kc8mu0d0BbEyvB6Sy7WRPVO3b9pPp96wbOLsJ9eomOwff9zvW1yvyjcCK/n7mDa7Xd9LvziNkX5qH1NYrLe/xt9vIeqX4tX2/U7myzfy86n03NOV3zMO/mJlZ5XxazMzMKufkYmZmlXNyMTOzyjm5mJlZ5ZxczMysck4uZhWQ9JNhbn+tpNOrqo9Zqzm5mFUgIt7d6jrAi3eFm7Wck4tZBST9Lr2eIOluSTcre/7J9UUjY6dRCb4i6TFJPyA3eKCkoyX9KA1muDI3VMe70gCN9yp7vsqjKf4JSd+V9L+B21Ps7yStSeX/V27fH5d0v7Lnh3xD0pjGfjL2SuXkYla9dwIXkj0741DgPQVlPgwcBvwF8DfAu+HFsaC+DJweEUcDncAVaZtvk93xfRywu2Z/xwEdEXGipJlAG9k4VUcBR0s6XtLbgI+QDZZ4VNrHx6p5y2Yv5ya0WfXujzSelLInEE4BflxT5njghojYDWyRdGeKHwYcCaxKDZ4xwFZlT358fUT0Xdv5F7JBJPusioi+QVJnpumhtPw6smTzduBoYE3a974UD1poNmxOLmbVez43vxsYK+lY4Bsp1vfY5KKxlwSsT62Tl4J1HkWb8/uafXwuIr6RLyDpPGBpRFwywL7Mhs2nxcyaICJWR8RRaVoO3APMkTQmXVN5fyr6ODBB0nGQnSaTdERkTwN8No0eDdnIvvWsBD6ZnuOBpImSDiYbpPD0NN/3LPU3V/5mzXDLxaxVbgFOJBvR92fAjwAi4oXUJXmRpP3J/ka/SDZq7lzgm5J+D9xN9qTTPUTE7en6yr3p9NfvgI9HxGOS/ifZkw9fRTaK73ygJY95ttHNoyKbjRCSXhfpIWaSFpANL39Bi6tlVsgtF7OR44OSLiH7u/0F8InWVsesPrdczMyscr6gb2ZmlXNyMTOzyjm5mJlZ5ZxczMysck4uZmZWuf8PE3KDy8qiNqEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import statistics as stats\n",
    "degrees = [G.in_degree(n) for n in G.nodes()]\n",
    "# plt.hist(nx.in_degree_histogram(G), bins=range(10))\n",
    "plt.hist(degrees, bins=range(21))\n",
    "# for n in range(0, 10):\n",
    "#     print(degrees.count(n))\n",
    "# print(\"max is: \",max(degrees))\n",
    "# print(\"min is: \",min(degrees))\n",
    "# print(\"mean is: \",stats.mean(degrees))\n",
    "# print(\"median is: \",stats.median(degrees))\n",
    "\n",
    "# # plt.hist( x=[ len(ca) for ca in coauthors.values() ], bins=range(60) )\n",
    "plt.xlabel('in-degree')\n",
    "plt.ylabel('number of nodes')\n",
    "# plt.xlim(0,51);\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** _The histogram appears to follow a decreasing power law, albeit with a huge spike at zero, which is a floor effect as you cannot have less than 0 in_nodes. It's skewed to the right obviously, with the mean larger than the median. What it tells us about the network is that the majority of nodes have no papers citing them at all. Then it follows a decreasing power law, with each extra citation (in_degree) reducing significantly the number of nodes. There are also some outlier super-connector nodes, with high in_degrees, thus important papers cited a lot (they're not visible on the above plot). Because of these super-connected nodes, you could say that it's a scale free network (a form of small world network)_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 2\n",
    "\n",
    "Using the [Link Analysis](https://networkx.github.io/documentation/networkx-1.10/reference/algorithms.link_analysis.html) algorithms provided by NetworkX, calculate the PageRank score for all nodes in the citation network, and store the result in a variable called `pageranks`. Print out the PageRank values for the two example papers given below.\n",
    "You can also use the `pagerank_scipy` implementation, which tends to be considerably faster than its regular `pagerank` counterpart, for which you have to install the [SciPy package](http://scipy.org/):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: scipy in /Users/andrewphilipharrison/anaconda3/lib/python3.7/site-packages (1.3.1)\r\n"
     ]
    }
   ],
   "source": [
    "! pip install --user scipy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To print and compare PageRank values, you might want to use commands like `print('{:.6f}'.format(var))` to use regular decimal notation with a fixed number of decimal places."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PageRank for paper 26968958 is 0.000004\n",
      "PageRank for paper 20923692 is 0.000059\n"
     ]
    }
   ],
   "source": [
    "# Add your code here:\n",
    "pageranks = nx.pagerank_scipy(G)\n",
    "\n",
    "# print PageRank for paper 26968958\n",
    "print('PageRank for paper 26968958 is {:.6f}'.format(pageranks[26968958]))\n",
    "\n",
    "# print PageRank for paper 20923692\n",
    "print('PageRank for paper 20923692 is {:.6f}'.format(pageranks[20923692]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 3\n",
    "\n",
    "Why do the two papers above have such different PageRank values? Write code below to investigate and show the cause of this, and then explain the cause of this difference based on the results generated by your code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Papers in_degree [(26968958, 10), (20923692, 10)]\n",
      "Papers out_degree [(26968958, 0), (20923692, 0)] \n",
      "\n",
      "Papers citing paper 26968958 [31426616, 31172718, 30828329, 30744105, 30646188, 30263035, 30231558, 30206453, 30151118, 28677354]\n",
      "Papers citing paper 20923692 [30081920, 29728628, 28889999, 25422088, 24037376, 23378838, 23316260, 22732592, 22139413, 21878358] \n",
      "\n",
      "Paper 26968958: in_degree of nodes citing  [(31426616, 0), (31172718, 1), (30828329, 0), (30744105, 3), (30646188, 2), (30263035, 1), (30231558, 6), (30206453, 0), (30151118, 1), (28677354, 4)]\n",
      "Paper 26968958: out_degree of nodes citing  [(31426616, 112), (31172718, 29), (30828329, 75), (30744105, 98), (30646188, 31), (30263035, 40), (30231558, 48), (30206453, 23), (30151118, 24), (28677354, 37)] \n",
      "\n",
      "Paper 20923692: in_degree of nodes citing  [(30081920, 0), (29728628, 0), (28889999, 2), (25422088, 0), (24037376, 209), (23378838, 0), (23316260, 4), (22732592, 0), (22139413, 0), (21878358, 0)]\n",
      "Paper 20923692: out_degree of nodes citing  [(30081920, 2), (29728628, 2), (28889999, 4), (25422088, 2), (24037376, 4), (23378838, 17), (23316260, 1), (22732592, 4), (22139413, 2), (21878358, 3)]\n"
     ]
    }
   ],
   "source": [
    "# Add your code here\n",
    "print(\"Papers in_degree\",G.in_degree([26968958, 20923692]))\n",
    "print(\"Papers out_degree\",G.out_degree([26968958, 20923692]),'\\n')\n",
    "\n",
    "print(\"Papers citing paper 26968958\",papers_citing[26968958])\n",
    "\n",
    "print(\"Papers citing paper 20923692\",papers_citing[20923692],'\\n')\n",
    "\n",
    "print(\"Paper 26968958: in_degree of nodes citing \",G.in_degree(papers_citing[26968958]))\n",
    "print(\"Paper 26968958: out_degree of nodes citing \",G.out_degree(papers_citing[26968958]),'\\n')\n",
    "\n",
    "print(\"Paper 20923692: in_degree of nodes citing \",G.in_degree(papers_citing[20923692]))\n",
    "print(\"Paper 20923692: out_degree of nodes citing \",G.out_degree(papers_citing[20923692]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Answer:** [Pagerank is based on the number of papers citing you, and from these papers, they're more important if they themselves are cited often. However, this importance is then shared amongst all of the papers it then cites.\n",
    "We can see two effects above.\n",
    "Paper 26968958 with page rank = 0.000004 is cited by 10 papers, and paper 20923692 with page rank = 0.000059 is also cited by 10 papers. So it must be some difference between these papers citing them, if it's not just sheer number of citations differing. Hence, we need to check the importance of each of these groups of 10 papers citing our papers of interest, and then how much that importance is diluted.\n",
    "For paper 26968958 we can see that the importance of nodes citing it, is not very high, ranging from 0 to 6 with 3 papers having nothing citing. For paper 20923692, 7 papers have nothing citing them, however one paper citing our paper (24037376, 209) is fairly important, as it's cited 209 times.\n",
    "Then, we look at how diluted the importance of cited papers are. For paper 26968958, we can see that all the papers citing it, cite a lot of papers, ranging from 24 to 112, meaning a strong dilution in the importance it receives. Whereas for paper 20923692, from the papers citing it, they don't cite a lot of papers, ranging from 1 to 17.\n",
    "It seems the main differtiator, is that Paper 20923692 receives 1/4 of the importance of (24037376, 4), which is an important paper as it's cited 209 times (24037376, 209). This would explain the difference in PageRank values.]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 4\n",
    "\n",
    "Copy the scoring function `score_ntn` from Task 4 of assignment 3. Rename it to `score_ntn_pagerank` and change its code to incorporate a paper's PageRank score in it's final score, in addition to tf-idf. In other words, the new function should accept a `list` of query tokens and a document ID, and should return a single `float` value that is calculated based on both scores (PageRank and tf-idf). Note that a `tf-idf` function is already provided above. Explain your decision on how to combine the two scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def score_ntn_pagerank(query_words, doc_id):\n",
    "    score = 0\n",
    "    for word in query_words:\n",
    "        score += tfidf(word, doc_id)\n",
    "    score*=pageranks[doc_id]\n",
    "    return float(score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Answer:** _Methods I read about online, involved returning n best documents (e.g. 10 best) from tf-idf, and then sorting this n by their pagerank. However, the structure of this assignment precludes that. Simple summing wouldn't work well, as the values are so different in scale that one can dominate the other. Therefore, I've chosen to use a weighted scoring that after calculating the sum of tf-idfs for each word in a document, multiplies this score by the pagerank. You could of course adjust this simple weighting, if you wanted to emphasise relevance or importance of documents respectively. However I've no further understanding of information needs at this point to justify that._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 5\n",
    "\n",
    "Copy the query function `query_ntn` from Task 4 of assignment 3. Rename it to `query_ntn_pagerank` and change the code to use our new scoring function `score_ntn_pagerank` from task 4 above. Demonstrate these functions with an example query that returns paper 19343057 as the top result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "doc id: 19343057 score: 0.030573494712968788\n",
      "doc id: 12462124 score: 0.010039051862298995\n",
      "doc id: 16497588 score: 0.008734186859012845\n",
      "doc id: 15032588 score: 0.005545104293061366\n",
      "doc id: 21997792 score: 0.0053350522869381875\n",
      "doc id: 15790845 score: 0.00465109904742401\n",
      "doc id: 22318520 score: 0.004001171557028238\n",
      "doc id: 22224781 score: 0.0036162594596571584\n",
      "doc id: 15608205 score: 0.003503549438092533\n",
      "doc id: 16602946 score: 0.0033459559723715343\n"
     ]
    }
   ],
   "source": [
    "def query_ntn_pagerank(query_string):\n",
    "    PRINT_TOP_X = 10\n",
    "    query_words = []\n",
    "    docs_ranked = {}\n",
    "    query_words = preprocess(tokenize(query_string))\n",
    "    doc_matches = set(inverted_index[query_words[0]])\n",
    "    for i in query_words:\n",
    "        temp_matches = set()\n",
    "        for j in inverted_index[i]:\n",
    "            temp_matches.add(j)\n",
    "        doc_matches = doc_matches.union(temp_matches)\n",
    "    for doc in doc_matches:\n",
    "        docs_ranked[doc] = score_ntn_pagerank(query_words, doc)\n",
    "    top = 0\n",
    "    for k, v in sorted(docs_ranked.items(), key=lambda kv: kv[1], reverse=True):\n",
    "        print(\"doc id:\",k, \"score:\", v)\n",
    "        top+=1\n",
    "        if top == PRINT_TOP_X:\n",
    "            break\n",
    "            \n",
    "query_ntn_pagerank(\"immunological dysregulation and gut microbiota\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Submission"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Submit the answers to the assignment via Canvas as a modified version of this Notebook file (file with `.ipynb` extension) that includes your code and your answers.\n",
    "\n",
    "Before submitting, restart the kernel and re-run the complete code (**Kernel > Restart & Run All**), and then check whether your assignment code still works as expected.\n",
    "\n",
    "Don't forget to add your name, and remember that the assignments have to be done individually and group submissions are **not allowed**."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}