03-Quering-AOpenAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d59d527f-1100-45ff-b051-5f7c9029d94d",
   "metadata": {},
   "source": [
    "# Queries with and without Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb9a9444-dc90-4fc3-aea7-8ee918301aba",
   "metadata": {},
   "source": [
    "So far, you have your Search Engine loaded **from two different data sources in two diferent text-based indexes**, on this notebook we are going to try some example queries and then use Azure OpenAI service to see if we can get even better results.\n",
    "\n",
    "The idea is that a user can ask a question about Computer Science (first datasource/index) or about Covid (second datasource/index), and the engine will respond accordingly.\n",
    "This **Multi-Index** demo, mimics the scenario where a company loads multiple type of documents of different types and about completly different topics and the search engine must respond with the most relevant results."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f6c7e3-9037-4b1e-ae17-1deaa27b9c08",
   "metadata": {},
   "source": [
    "## Set up variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8e50b404-a061-49e7-a3c7-c6eabc98ff0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import urllib\n",
    "import requests\n",
    "import random\n",
    "import json\n",
    "from collections import OrderedDict\n",
    "from IPython.display import display, HTML, Markdown\n",
    "from langchain.chains import LLMChain\n",
    "from langchain.prompts import PromptTemplate\n",
    "from langchain.chat_models import AzureChatOpenAI\n",
    "from langchain.docstore.document import Document\n",
    "from langchain.chains.qa_with_sources import load_qa_with_sources_chain\n",
    "from langchain.embeddings import AzureOpenAIEmbeddings\n",
    "\n",
    "from common.prompts import COMBINE_QUESTION_PROMPT, COMBINE_PROMPT, COMBINE_PROMPT_TEMPLATE\n",
    "from common.utils import (\n",
    "    get_search_results,\n",
    "    model_tokens_limit,\n",
    "    num_tokens_from_docs,\n",
    "    num_tokens_from_string\n",
    ")\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2f2c22f8-79ab-405c-95e8-77a1978e53bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup the Payloads header\n",
    "headers = {'Content-Type': 'application/json','api-key': os.environ['AZURE_SEARCH_KEY']}\n",
    "params = {'api-version': os.environ['AZURE_SEARCH_API_VERSION']}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9297d29b-1f61-4dce-858e-bf4272172dba",
   "metadata": {},
   "source": [
    "## Multi-Index Search queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5a46e2d3-298a-4708-83de-9e108b1a117a",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Text-based Indexes that we are going to query (from Notebook 01 and 02)\n",
    "index1_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index1_name, index2_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c62ebb2-d7be-4bfb-b1ba-4db86c11839a",
   "metadata": {},
   "source": [
    "Try questions that you think might be answered or addressed in computer science papers in 2020-2021 or that can be addressed by medical publications about COVID in 2020-2021. Try comparing the results with the open version of ChatGPT.<br>\n",
    "The idea is that the answers using Azure OpenAI only looks at the information contained on these publications.\n",
    "\n",
    "**Example Questions you can ask**:\n",
    "- What is CLP?\n",
    "- How Markov chains work?\n",
    "- What are some examples of reinforcement learning?\n",
    "- What are the main risk factors for Covid-19?\n",
    "- What medicine reduces inflamation in the lungs?\n",
    "- Why Covid doesn't affect kids that much compared to adults?\n",
    "- Does chloroquine really works against covid?\n",
    "- Who won the 1994 soccer world cup? # This question should yield no answer if the system is correctly grounded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b9b53c14-19bd-451f-aa43-7ad27ccfeead",
   "metadata": {},
   "outputs": [],
   "source": [
    "QUESTION = \"What is CLP?\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d925eb-7f9c-429e-a62a-4c37d7702caf",
   "metadata": {},
   "source": [
    "### Search on both indexes individually and aggragate results\n",
    "\n",
    "#### **Note**: \n",
    "In order to standarize the indexes, **there must be 8 mandatory fields present on each text-based index**: `id, title, content, chunks, language, name, location, vectorized`. This is so that each document can be treated the same along the code. Also, **all indexes must have a semantic configuration**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "faf2e30f-e71f-4533-ab52-27d048b80a89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "Index: cogsrch-index-files Results Found: 9172, Results Returned: 10\n",
      "200\n",
      "Index: cogsrch-index-csv Results Found: 48638, Results Returned: 10\n"
     ]
    }
   ],
   "source": [
    "agg_search_results = dict()\n",
    "\n",
    "for index in indexes:\n",
    "    search_payload = {\n",
    "        \"search\": QUESTION,\n",
    "        \"select\": \"id, title, chunks, name, location\",\n",
    "        \"queryType\": \"semantic\",\n",
    "        \"semanticConfiguration\": \"my-semantic-config\",\n",
    "        \"count\": \"true\",\n",
    "        \"speller\": \"lexicon\",\n",
    "        \"queryLanguage\": \"en-us\",\n",
    "        \"captions\": \"extractive\",\n",
    "        \"answers\": \"extractive\",\n",
    "        \"top\": \"10\"\n",
    "    }\n",
    "\n",
    "    r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + index + \"/docs/search\",\n",
    "                     data=json.dumps(search_payload), headers=headers, params=params)\n",
    "    print(r.status_code)\n",
    "\n",
    "    search_results = r.json()\n",
    "    agg_search_results[index]=search_results\n",
    "    print(\"Index:\", index, \"Results Found: {}, Results Returned: {}\".format(search_results['@odata.count'], len(search_results['value'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7fd0fe5-4ee0-42e2-a920-72b93a407389",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Display the top results (from both searches) based on the score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9e938337-602d-4b61-8141-b8c92a5d91da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Top Answers</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5>Answer - score: 0.97</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The key notions of CLP are those of an algebra and an associated constraint solver over a class of constraints, namely a set of first order formulas including the always satisfiable constraint true, the un- satisfiable constraint false, and closed under variable renaming, conjunction and existential quantification."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5>Answer - score: 0.93</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Isotype-specific antibody responses to rotavirus and virus proteins in cows inoculated with subunit vaccines composed of recombinant SA11 rotavirus core-like particles (CLP) or virus-like particles (VLP)."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<h4>Top Results</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0508/0508108v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">0508108v1.pdf</a> - score: 3.55</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "CLP(FD) is an extension of logic programming. In CLP(FD) pro- grams, logical variables are assigned a domain and relations between vari- ables are described with constraints. A solution to a CLP(FD) program is a valuation of every variable in its own domain such that no constraint is falsified."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0701/0701082v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">0701082v1.pdf</a> - score: 3.46</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The key notions of CLP are those of an algebra and an associated constraint solver over a class of constraints, namely a set of first order formulas including the always satisfiable constraint true, the un- satisfiable constraint false, and closed under variable renaming, conjunction and existential quantification."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0508/0508106v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">arXiv:cs/0508106v1  [cs.PL]  24 Aug 2005</a> - score: 3.13</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "A CLP(C) program is a finite set of rules. A rule has the form H ← c⋄B where H and B are atoms and c is a finite conjunction of primitive constraints such that DC |= ∃c. A query has the form 〈A | d〉 where A is an atom and d is a finite conjunction of primitive constraints. Given an atom A := p(t̃), we write rel(A) to denote the predicate symbol p."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0011/0011030v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">arXiv:cs/0011030v1  [cs.AI]  21 Nov 2000</a> - score: 3.12</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "A solution is an instantiation of the variables of X which satisfies all the constraints in R.  2.1 Constraint Logic Programming  Constraint logic programming (CLP) [7] is an extension of logic programming where some of the predicate and function symbols have a fixed interpretation over some subdomain (e.g. finite trees or real numbers)."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0106/0106008v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">arXiv:cs/0106008v1  [cs.PL]  7 Jun 2001</a> - score: 3.11</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "A distinguishing feature of CLP(Intervals) is that it decomposes equations, or other composite expressions, into primitive constraints. These primitive con- straints are the relational versions of the building blocks of expressions, which are admissible functions."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0506/0506005v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">0506005v1.pdf</a> - score: 3.1</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "A CLP(FD) system provides primitives for  accessing and updating attribute values. A CLP(FD) system provides equality (=), disequality (6=), and inequality con-  straints. In addition, a CLP(FD) system also provides some other constraints such  as global constraints."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0408/0408056v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">0408056v1.pdf</a> - score: 3.07</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "CLP(FD) languages have been suc-  cessfully used for solving a variety of industrial and academic problems. However,  in some constraint problems, where domain elements need to be acquired, it may  not be wise to perform the acquisition of the whole domains of variables before the  beginning of the constraint propagation process."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0310/0310042v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">()</a> - score: 3.02</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "A CLP(FD) program searches a solution for a set of variables which take values over finite domains and which must verify a set of constraints. The evolution of the domains can be viewed as a sequence of applications of reduction operators attached to the constraints."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0003/0003026v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">arXiv:cs/0003026v1  [cs.LO]  8 Mar 2000</a> - score: 2.97</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "A proof procedure for CLP is defined as an extension of standard resolution. A state is defined as a pair 〈← a, A || C〉 of a goal and a set of constraints. At each step of the computation, some literal a is selected from the current goal according to some selection function."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0008/0008036v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">arXiv:cs/0008036v1  [cs.CL]  30 Aug 2000</a> - score: 2.96</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "To this end,  we discuss the formal basics of Constraint Logic Programming (CLP), which is used here  to provide an operational treatment of various declarative constraint-based grammars. This  is done by an embedding of the logical description languages of such grammars into a CLP  scheme, yielding Constraint Logic Grammars (CLGs)."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/10403670/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">Quantification of recombinant core-like particles of bluetongue virus using immunosorbent electron microscopy.</a> - score: 2.79</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Immunosorbent electron microscopy was used to quantify recombinant baculovirus-generated bluetongue virus (BTV) core-like particles (CLP) in either purified preparations or lysates of recombinant baculovirus-infected cells. The capture antibody was an anti-BTV VP7 monoclonal antibody."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/17047515/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">Increased susceptibility to septic and endotoxic shock in monocyte chemoattractant protein 1/cc chemokine ligand 2-deficient mice correlates with reduced interleukin 10 and enhanced macrophage migration inhibitory factor production.</a> - score: 2.55</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The chemokine monocyte chemoattractant protein 1/CC chemokine ligand 2 (MCP-1/CCL2) is a potent chemoattractant of mononuclear cells and a regulatory mediator involved in a variety of inflammatory diseases."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7103146/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">Length of encapsidated cargo impacts stability and structure of in vitro assembled alphavirus core-like particles</a> - score: 2.54</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "In vitro assembly of alphavirus nucleocapsid cores, called core-like particles (CLPs), requires a polyanionic cargo. There are no sequence or structure requirements to encapsidate single-stranded nucleic acid cargo. In this work, we wanted to determine how the length of the cargo impacts the stability and structure of the assembled CLPs."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://api.elsevier.com/content/article/pii/S0033318220301420; https://www.sciencedirect.com/science/article/pii/S0033318220301420?v=s5?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">The educational value of outpatient CL rotations- a white paper from the ACLP residency education subcommittee</a> - score: 2.46</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The Academy of Consultation-Liaison Psychiatry (ACLP) residency education subcommittee convened a writing group with the goal of summarizing the current evidence about outpatient consultation-liaison psychiatry (CLP) training and providing a framework for CLP educators who are interested in developing outpatient CLP rotations within their programs."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/16286234/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">What is the contribution of respiratory viruses and lung proteases to airway remodelling in asthma and chronic obstructive pulmonary disease?</a> - score: 2.29</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Chronic obstructive pulmonary disease (COPD), by definition, involves structural changes to the airways. However, very little is known about what role virus infections play in the development of this remodelling."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7131174/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">Isotype-specific antibody responses to rotavirus and virus proteins in cows inoculated with subunit vaccines composed of recombinant SA11 rotavirus core-like particles (CLP) or virus-like particles (VLP)</a> - score: 2.26</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Isotype-specific antibody responses to rotavirus and virus proteins in cows inoculated with subunit vaccines composed of recombinant SA11 rotavirus core-like particles (CLP) or virus-like particles (VLP)."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5809586/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">A cold-inducible RNA-binding protein (CIRP)-derived peptide attenuates inflammation and organ injury in septic mice</a> - score: 2.11</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Cold-inducible RNA-binding protein (CIRP) is a novel sepsis inflammatory mediator and C23 is a putative CIRP competitive inhibitor. Therefore, we hypothesized that C23 can ameliorate sepsis-associated injury to the lungs and kidneys."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/17414124/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">Community-acquired pneumonia: what is relevant and what is not?</a> - score: 1.88</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The mainstay of community-acquired pneumonia prevention is influenza and pneumococcal immunization. Promotion of smoking cessation will also help curtail the incidence of pneumococcal disease..\u0000"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7131086/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">Passive immunity to bovine rotavirus in newborn calves fed colostrum supplements from cows immunized with recombinant SA11 rotavirus core-like particle (CLP) or virus-like particle (VLP) vaccines</a> - score: 1.82</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Passive immunity to bovine rotavirus in newborn calves fed colostrum supplements from cows immunized with recombinant SA11 rotavirus core-like particle (CLP) or virus-like particle (VLP) vaccines. Heterotypic passive immunity to IND (P[5]G6) bovine rotavirus (BRV) was evaluated."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1111/bjh.14134; https://www.ncbi.nlm.nih.gov/pubmed/27173746/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">Adherence to long-term anticoagulation treatment, what is known and what the future might hold.</a> - score: 1.78</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "utilizing the com-b (capability, opportunity, motivation and behaviour) psychological model of non-adherence, we present the available evidence, not only in terms of describing the extent of the non-adherence problem, but also describing why patients do not adhere, offering theory-driven and evidence-based solutions to improve long-term adherence …"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML('<h4>Top Answers</h4>'))\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['@search.answers']:\n",
    "        if result['score'] > 0.5: # Show answers that are at least 50% of the max possible score=1\n",
    "            display(HTML('<h5>' + 'Answer - score: ' + str(round(result['score'],2)) + '</h5>'))\n",
    "            display(HTML(result['text']))\n",
    "            \n",
    "print(\"\\n\\n\")\n",
    "display(HTML('<h4>Top Results</h4>'))\n",
    "\n",
    "content = dict()\n",
    "ordered_content = OrderedDict()\n",
    "\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['value']:\n",
    "        if result['@search.rerankerScore'] > 1:# Show answers that are at least 25% of the max possible score=4\n",
    "            content[result['id']]={\n",
    "                                    \"title\": result['title'],\n",
    "                                    \"chunks\": result['chunks'], \n",
    "                                    \"chunks_vectors\": [],\n",
    "                                    \"name\": result['name'], \n",
    "                                    \"location\": result['location'] ,\n",
    "                                    \"caption\": result['@search.captions'][0]['text'],\n",
    "                                    \"score\": result['@search.rerankerScore'],\n",
    "                                    \"index\": index\n",
    "                                    }\n",
    "    \n",
    "#After results have been filtered we will Sort and add them as an Ordered list\\n\",\n",
    "for id in sorted(content, key= lambda x: content[x][\"score\"], reverse=True):\n",
    "    ordered_content[id] = content[id]\n",
    "    url = str(ordered_content[id]['location']) + os.environ['BLOB_SAS_TOKEN']\n",
    "    title = str(ordered_content[id]['title']) if (ordered_content[id]['title']) else ordered_content[id]['name']\n",
    "    score = str(round(ordered_content[id]['score'],2))\n",
    "    display(HTML('<h5><a href=\"'+ url + '\">' + title + '</a> - score: '+ score + '</h5>'))\n",
    "    display(HTML(ordered_content[id]['caption']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a6d3e6-afb2-4fa7-96d3-69bc2373ded5",
   "metadata": {},
   "source": [
    "## Comments on Query results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84e02227-6a92-4944-86f8-6c1e38d90fe4",
   "metadata": {},
   "source": [
    "As seen above the semantic search feature of Azure Cognitive Search service is good. It gives answers (sometimes) and also the top results with the corresponding file and the paragraph where the answers is possible located.\n",
    "\n",
    "Let's see if we can make this better with Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df3e6d4-9a09-4b0f-b328-238738ccfaec",
   "metadata": {},
   "source": [
    "# Using Azure OpenAI\n",
    "\n",
    "To use OpenAI to get a better answer to our question, the thought process is simple: let's **give the answer and the content of the documents from the search result to the GPT model as context and let it provide a better response**.\n",
    "\n",
    "Now, before we do this, we need to understand a few things first:\n",
    "\n",
    "1) Chainning and Prompt Engineering\n",
    "2) Embeddings\n",
    "\n",
    "We will use a library call **LangChain** that wraps a lot of boiler plate code.\n",
    "Langchain is one library that does a lot of the prompt engineering for us under the hood, for more information see [here](https://python.langchain.com/en/latest/index.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "eea62a7d-7e0e-4a93-a89c-20c96560c665",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
    "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "325d9138-2250-4f6b-bc88-50d7957f8d33",
   "metadata": {},
   "source": [
    "**Important Note**: Starting now, we will utilize OpenAI models. Please ensure that you have deployed the following models within the Azure OpenAI portal using these precise deployment names:\n",
    "\n",
    "- text-embedding-ada-002\n",
    "- gpt-35-turbo\n",
    "- gpt-35-turbo-16k\n",
    "- gpt-4\n",
    "- gpt-4-32k\n",
    "\n",
    "Should you have deployed the models under different names, the code provided below will not function as expected. To resolve this, you would need to modify the variable names throughout all the notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e7c720e-ece1-45ad-9d01-2dfd15c182bb",
   "metadata": {},
   "source": [
    "## A gentle intro to chaining LLMs and prompt engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bcd7028-5a6c-4296-8c85-4f420d408d69",
   "metadata": {},
   "source": [
    "Chains are what you get by connecting one or more large language models (LLMs) in a logical way. (Chains can be built of entities other than LLMs but for now, let’s stick with this definition for simplicity).\n",
    "\n",
    "Azure OpenAI is a type of LLM (provider) that you can use but there are others like Cohere, Huggingface, etc.\n",
    "\n",
    "Chains can be simple (i.e. Generic) or specialized (i.e. Utility).\n",
    "\n",
    "* Generic — A single LLM is the simplest chain. It takes an input prompt and the name of the LLM and then uses the LLM for text generation (i.e. output for the prompt).\n",
    "\n",
    "Here’s an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "13df9247-e784-4e04-9475-55e672efea47",
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL = \"gpt-35-turbo\" # options: gpt-35-turbo, gpt-35-turbo-16k, gpt-4, gpt-4-32k\n",
    "COMPLETION_TOKENS = 1000\n",
    "llm = AzureChatOpenAI(deployment_name=MODEL, temperature=0, max_tokens=COMPLETION_TOKENS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7b0520b9-83b2-49fd-ad84-624cb0f15ce1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer the following question: \"What is CLP?\". Give your response in French\n"
     ]
    }
   ],
   "source": [
    "# Now we create a simple prompt template\n",
    "prompt = PromptTemplate(\n",
    "    input_variables=[\"question\", \"language\"],\n",
    "    template='Answer the following question: \"{question}\". Give your response in {language}',\n",
    ")\n",
    "\n",
    "print(prompt.format(question=QUESTION, language=\"French\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "dcc7dae3-6b88-4ea6-be43-b178ebc559dc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'question': 'What is CLP?',\n",
       " 'language': 'French',\n",
       " 'text': \"CLP, ou Classification, Labelling and Packaging, est un système de classification, d'étiquetage et d'emballage des produits chimiques utilisé dans l'Union européenne. Il vise à informer les utilisateurs sur les dangers des produits chimiques et à promouvoir une utilisation sûre. Le CLP repose sur des critères de classification harmonisés au niveau international et utilise des pictogrammes, des mentions de danger et des conseils de prudence pour communiquer les informations de manière claire et compréhensible.\"}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# And finnaly we create our first generic chain\n",
    "chain_chat = LLMChain(llm=llm, prompt=prompt)\n",
    "chain_chat({\"question\": QUESTION, \"language\": \"French\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd8539d0-a538-4368-82c3-5f91d8370f1e",
   "metadata": {},
   "source": [
    "**Note**: this is the first time you use OpenAI in this Accelerator, so if you get a Resource not found error, is most likely because the name of your OpenAI model deployment is different than the variable MODEL set above"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ed014c-0c6b-448c-b995-fe7970b92ad5",
   "metadata": {},
   "source": [
    "Great!!, now you know how to create a simple prompt and use a chain in order to answer a general question using ChatGPT knowledge!. \n",
    "\n",
    "It is important to note that we rarely use generic chains as standalone chains. More often they are used as building blocks for Utility chains (as we will see next). Also important to notice is that we are NOT using our documents or the result of the Azure Search yet, just the knowledge of ChatGPT on the data it was trained on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c48038-b1af-4228-8ffb-720e554fd3b2",
   "metadata": {
    "tags": []
   },
   "source": [
    "**The second type of Chains are Utility:**\n",
    "\n",
    "* Utility — These are specialized chains, comprised of many LLMs to help solve a specific task. For example, LangChain supports some end-to-end chains (such as QA_WITH_SOURCES for QnA Doc retrieval, Summarization, etc) and some specific ones (such as GraphQnAChain for creating, querying, and saving graphs). \n",
    "\n",
    "We will look at one specific chain called **qa_with_sources** in this workshop for digging deeper and solve our use case of enhancing the results of Azure Cognitive Search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0454ddb-44d8-4fa9-929a-5e5563dd28f8",
   "metadata": {},
   "source": [
    "\n",
    "But before dealing with the utility chain needed, we need to deal first with this problem: **the content of the search result files is or can be very lengthy, more than the allowed tokens allowed by the GPT Azure OpenAI models**. \n",
    "\n",
    "This is where the concept of embeddings/vectors come into place.\n",
    "\n",
    "## Embeddings and Vector Search\n",
    "\n",
    "From the Azure OpenAI documentation ([HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=python)), An embedding is a special format of data representation that can be easily utilized by machine learning models and algorithms. The embedding is an information dense representation of the semantic meaning of a piece of text. Each embedding is a vector of floating point numbers, such that the distance between two embeddings in the vector space is correlated with semantic similarity between two inputs in the original format. For example, if two texts are similar, then their vector representations should also be similar. \n",
    "\n",
    "To address the challenge of accommodating context within the token limit of a Language Model (LLM), the solution involves the following steps:\n",
    "\n",
    "1. **Segmenting Documents**: Divide the documents into smaller segments or chunks.\n",
    "2. **Vectorization of Chunks**: Transform these chunks into vectors using appropriate techniques.\n",
    "3. **Vector Semantic Search**: Execute a semantic search using vectors to identify the top chunks similar to the given question.\n",
    "4. **Optimal Context Provision**: Provide the LLM with the most relevant and concise context, thereby achieving an optimal balance between comprehensiveness and lengthiness.\n",
    "\n",
    "\n",
    "Notice that **the documents chunks are already done in Azure Search**. *ordered_content* dictionary (created a few cells above) contains the chunks of each document. So we don't really need to chunk them again, but we still need to make sure that we can be as fast as possible and that we are below the max allowed input token limits of our selected OpenAI model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80e79235-3d8b-4713-9336-5004cc4a1556",
   "metadata": {},
   "source": [
    "Our ultimate goal is to rely solely on vector indexes. While it is possible to manually code parsers with OCR for various file types and develop a scheduler to synchronize data with the index, there is a more efficient alternative: **Azure Cognitive Search is soon going to release automated chunking strategies and vectorization within the next months**, so we have three options: \n",
    "1. Wait for this functionality while in the meantime manually push chunks and its vectors to the vector-based indexes \n",
    "2. Fill up the vector-based indexes on-demand, as documents are discovered by users\n",
    "3. Use custom skills (for chunking and vectorization) and use knowledge stores in order to create a vector-base index from a text-based-ai-enriched index at ingestion time. See [HERE](https://github.com/Azure/cognitive-search-vector-pr/blob/main/demo-python/code/azure-search-vector-ingestion-python-sample.ipynb) for instructions on how to do this.\n",
    "\n",
    "In this notebook we are going to implement Option 2: **Create vector-based indexes per each text-based indexes and fill them up on-demand as documents are discovered**. Why? because is simpler and quick to implement, while we wait for Option 1 to become a feature of Azure Search Engine (which is the automation of Option 3 inside the search engine).\n",
    "\n",
    "As observed in Notebooks 1 and 2, each text-based index contains a field named `vectorized` that we have not utilized yet. We will now harness this field. The objective is to avoid vectorizing all documents at the time of ingestion (Option 3). Instead, we can vectorize the chunks as users search for or discover documents. This approach ensures that we allocate funds and resources only when the documents are actually required. Typically, in an organization with a vast repository of documents in a data lake, only 20% of the documents are frequently accessed, while the rest remain untouched. This phenomenon mirrors the [Pareto Principle](https://en.wikipedia.org/wiki/Pareto_principle) found in nature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "12682a1b-df92-49ce-a638-7277103f6cb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "index_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index_name, index2_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78a6d6a7-18ef-45b2-a216-3c1f50006593",
   "metadata": {},
   "source": [
    "In order to not duplicate code, we have put many of the code used above into functions. These functions are in the `common/utils.py` and `common/prompts.py` files. This way we can use these functios in the app that we will build later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3bccca45-d1dd-476f-b109-a528b857b6b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 20\n"
     ]
    }
   ],
   "source": [
    "k = 10 # Number of results per each text_index\n",
    "ordered_results = get_search_results(QUESTION, indexes, k=10, reranker_threshold=1)\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7714f38a-daaa-4fc5-a95a-dd025d153216",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the below line if you want to inspect the ordered results\n",
    "# ordered_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da70e7a8-7536-4688-b30c-01ba28e9b9f8",
   "metadata": {},
   "source": [
    "Now we can fill up the vector-based index as users lookup documents using the text-based index. This approach although it requires two searches per user query (one on the text-based indexes and the other one on the vector-based indexes), it is simpler to implement and will be incrementatly faster as user use the system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2937ba3b-098d-43f8-8498-3534882a5cc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "embedder = AzureOpenAIEmbeddings(model=\"text-embedding-ada-002\", skip_empty=True) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f664df30-99c3-4a30-8cb0-42ba3044e5b0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vectorizing 7 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0508/0508108v1.pdf\n",
      "Vectorizing 5 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0701/0701082v1.pdf\n",
      "Vectorizing 8 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0508/0508106v1.pdf\n",
      "Vectorizing 8 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0011/0011030v1.pdf\n",
      "Vectorizing 7 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0106/0106008v1.pdf\n",
      "Vectorizing 14 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0506/0506005v1.pdf\n",
      "Vectorizing 13 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0408/0408056v1.pdf\n",
      "Vectorizing 11 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0310/0310042v1.pdf\n",
      "Vectorizing 8 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0003/0003026v1.pdf\n",
      "Vectorizing 66 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0008/0008036v1.pdf\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pubmed/10403670/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pubmed/17047515/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7103146/\n",
      "Vectorizing 1 chunks from Document: https://api.elsevier.com/content/article/pii/S0033318220301420; https://www.sciencedirect.com/science/article/pii/S0033318220301420?v=s5\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pubmed/16286234/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7131174/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5809586/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pubmed/17414124/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7131086/\n",
      "Vectorizing 1 chunks from Document: https://doi.org/10.1111/bjh.14134; https://www.ncbi.nlm.nih.gov/pubmed/27173746/\n",
      "CPU times: user 8.23 s, sys: 208 ms, total: 8.44 s\n",
      "Wall time: 47.8 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "for key,value in ordered_results.items():\n",
    "    if value[\"vectorized\"] != True: # If the document has not been vectorized yet\n",
    "        i = 0\n",
    "        print(\"Vectorizing\",len(value[\"chunks\"]),\"chunks from Document:\",value[\"location\"])\n",
    "        for chunk in value[\"chunks\"]: # Iterate over the document's text chunks\n",
    "            try:\n",
    "                upload_payload = {  # Insert the chunk and its vector in the vector-based index\n",
    "                    \"value\": [\n",
    "                        {\n",
    "                            \"id\": key + \"_\" + str(i),\n",
    "                            \"title\": f\"{value['title']}_chunk_{str(i)}\",\n",
    "                            \"chunk\": chunk,\n",
    "                            \"chunkVector\": embedder.embed_query(chunk if chunk!=\"\" else \"-------\"),\n",
    "                            \"name\": value[\"name\"],\n",
    "                            \"location\": value[\"location\"],\n",
    "                            \"@search.action\": \"upload\"\n",
    "                        },\n",
    "                    ]\n",
    "                }\n",
    "\n",
    "                r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + value[\"index\"]+\"-vector\" + \"/docs/index\",\n",
    "                                     data=json.dumps(upload_payload), headers=headers, params=params)\n",
    "                \n",
    "                if r.status_code != 200:\n",
    "                    print(r.status_code)\n",
    "                    print(r.text)\n",
    "                else:\n",
    "                    i = i + 1 # increment chunk number\n",
    "                    \n",
    "            except Exception as e:\n",
    "                print(\"Exception:\",e)\n",
    "                print(chunk)\n",
    "                continue\n",
    "                    \n",
    "        # Update document in text-based index and mark it as \"vectorized\"\n",
    "        upload_payload = {\n",
    "            \"value\": [\n",
    "                {\n",
    "                    \"id\": key,\n",
    "                    \"vectorized\": True,\n",
    "                    \"@search.action\": \"merge\"\n",
    "                },\n",
    "            ]\n",
    "        }\n",
    "\n",
    "        r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + value[\"index\"]+ \"/docs/index\",\n",
    "                         data=json.dumps(upload_payload), headers=headers, params=params)\n",
    "                    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f490b7fe-eec2-4c96-a2f2-f8ab0a1b2098",
   "metadata": {},
   "source": [
    "**Note**: How the text-based and the vector-based indexes stay in sync?\n",
    "For document changes, the problem is already taken care of, since Azure Engine will update the text-based index automatically if a file has a new version. This puts the vectorized field in None and the next time that the file is searched it will be vectorized again into the vector-based index.\n",
    "\n",
    "However for deletion of files, the problem is half solved. Azure Search engine would delete the documents in the text-based index if the file is deleted on the source, however you will need to code a script that runs on a fixed schedule that looks for deleted ids in the text-based index and deletes the corresponding chunks in the vector-based index."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f67f3a2-0023-4f5a-b52f-3fb071cfd8e1",
   "metadata": {},
   "source": [
    "Now we search on the vector-based indexes and get the top k most similar chunks to our question:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "61098bb4-33da-4eb4-94cf-503587337aca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 3\n"
     ]
    }
   ],
   "source": [
    "vector_indexes = [index+\"-vector\" for index in indexes]\n",
    "\n",
    "k = 10\n",
    "similarity_k = 3\n",
    "ordered_results = get_search_results(QUESTION, vector_indexes,\n",
    "                                        k=k, # Number of results per vector index\n",
    "                                        reranker_threshold=1,\n",
    "                                        vector_search=True, \n",
    "                                        similarity_k=similarity_k,\n",
    "                                        query_vector = embedder.embed_query(QUESTION)\n",
    "                                        )\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a98a974-0633-499f-a8f0-29bf6242e737",
   "metadata": {},
   "source": [
    "For vector search is not recommended to give more than k=5 chunks (of max 5000 characters each) to the LLM as context. Otherwise you can have issues later with the token limit trying to have a conversation with memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7dfb9e39-2542-469d-8f64-4c0c26d79535",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of chunks: 3\n"
     ]
    }
   ],
   "source": [
    "top_docs = []\n",
    "for key,value in ordered_results.items():\n",
    "    location = value[\"location\"] if value[\"location\"] is not None else \"\"\n",
    "    top_docs.append(Document(page_content=value[\"chunk\"], metadata={\"source\": location}))\n",
    "        \n",
    "print(\"Number of chunks:\",len(top_docs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "880885fe-16bd-44bb-9556-7cb3d4989993",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "System prompt token count: 1669\n",
      "Max Completion Token count: 1000\n",
      "Combined docs (context) token count: 2423\n",
      "--------\n",
      "Requested token count: 5092\n",
      "Token limit for gpt-35-turbo : 4096\n",
      "Chain Type selected: map_reduce\n"
     ]
    }
   ],
   "source": [
    "# Calculate number of tokens of our docs\n",
    "if(len(top_docs)>0):\n",
    "    tokens_limit = model_tokens_limit(MODEL) # this is a custom function we created in common/utils.py\n",
    "    prompt_tokens = num_tokens_from_string(COMBINE_PROMPT_TEMPLATE) # this is a custom function we created in common/utils.py\n",
    "    context_tokens = num_tokens_from_docs(top_docs) # this is a custom function we created in common/utils.py\n",
    "    \n",
    "    requested_tokens = prompt_tokens + context_tokens + COMPLETION_TOKENS\n",
    "    \n",
    "    chain_type = \"map_reduce\" if requested_tokens > 0.9 * tokens_limit else \"stuff\"  \n",
    "    \n",
    "    print(\"System prompt token count:\",prompt_tokens)\n",
    "    print(\"Max Completion Token count:\", COMPLETION_TOKENS)\n",
    "    print(\"Combined docs (context) token count:\",context_tokens)\n",
    "    print(\"--------\")\n",
    "    print(\"Requested token count:\",requested_tokens)\n",
    "    print(\"Token limit for\", MODEL, \":\", tokens_limit)\n",
    "    print(\"Chain Type selected:\", chain_type)\n",
    "        \n",
    "else:\n",
    "    print(\"NO RESULTS FROM AZURE SEARCH\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e232424-c7ba-4153-b23b-fb1fa2ebc64b",
   "metadata": {},
   "source": [
    "Now we will use our Utility Chain from LangChain `qa_with_sources`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "511273b3-256d-4e60-be72-ccd4a74cb885",
   "metadata": {},
   "outputs": [],
   "source": [
    "if chain_type == \"stuff\":\n",
    "    chain = load_qa_with_sources_chain(llm, chain_type=chain_type, \n",
    "                                       prompt=COMBINE_PROMPT)\n",
    "elif chain_type == \"map_reduce\":\n",
    "    chain = load_qa_with_sources_chain(llm, chain_type=chain_type, \n",
    "                                       question_prompt=COMBINE_QUESTION_PROMPT,\n",
    "                                       combine_prompt=COMBINE_PROMPT,\n",
    "                                       return_intermediate_steps=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "b99a0c19-d48c-41e9-8d6c-6d9f13d29da3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 49.6 ms, sys: 8.8 ms, total: 58.4 ms\n",
      "Wall time: 3.69 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Try with other language as well\n",
    "response = chain({\"input_documents\": top_docs, \"question\": QUESTION, \"language\": \"English\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "37f7fa67-f67b-402e-89e3-266d5d6d21d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "CLP can refer to different things depending on the context. In the context of Consultation-Liaison Psychiatry, CLP stands for Consultation-Liaison Psychiatry<sup><a href=\"https://api.elsevier.com/content/article/pii/S0033318220301420\" target=\"_blank\">[1]</a></sup>. In the context of Constraint Logic Programming, CLP stands for Constraint Logic Programming<sup><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/arxivcs/0008/0008036v1.pdf\" target=\"_blank\">[2]</a></sup>."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(response['output_text']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05e27c75-bfd9-4304-b2fd-c8e30bcc0558",
   "metadata": {},
   "source": [
    "**Please Note**: There are some instances where, despite the answer's high accuracy and quality, the references are not done according to the instructions provided in the COMBINE_PROMPT. This behavior is anticipated when dealing with GPT-3.5 models. We will provide a more detailed explanation of this phenomenon towards the conclusion of Notebook 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "11345374-6420-4b36-b061-795d2a804c85",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment if you want to inspect the results from map_reduce chain type, each top similar chunk summary (k=4 by default)\n",
    "\n",
    "# if chain_type == \"map_reduce\":\n",
    "#     for step in response['intermediate_steps']:\n",
    "#         display(HTML(\"<b>Chunk Summary:</b> \" + step))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f347373a-a5be-473d-b64e-0f6b6dbcd0e0",
   "metadata": {},
   "source": [
    "# Summary\n",
    "##### This answer is way better than taking just the result from Azure Cognitive Search. So the summary is:\n",
    "- Utilizing Azure Cognitive Search, we conduct a multi-index text-based search that identifies the top documents from each index.\n",
    "- Utilizing Azure Cognitive Search's vector search, we extract the most relevant chunks of information.\n",
    "- Subsequently, Azure OpenAI utilizes these extracted chunks as context, comprehends the content, and employs it to deliver optimal answers.\n",
    "- Best of two worlds!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdc6e2fe-1c34-4952-99ad-14940f022379",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "In the next notebook, we are going to see how we can treat complex and large documents separately, also using Vector Search"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}