chatbot.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AI Chatbot Search PDF Files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Packages\n",
    "\n",
    "If you are using MacOS, please use `pip3`.\n",
    "\n",
    "`-qU` means `quiet` and `Upgrade`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -qU \\\n",
    "    langchain==0.0.276 \\\n",
    "    openai==0.27.10 \\\n",
    "    tiktoken==0.4.0 \\\n",
    "    pinecone-client==2.2.2 \\\n",
    "    wikipedia==1.4.0 \\\n",
    "    pypdf==3.15.4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.embeddings import OpenAIEmbeddings   \n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.vectorstores import Pinecone\n",
    "from langchain.chains import ConversationalRetrievalChain\n",
    "from langchain.memory import ConversationSummaryMemory\n",
    "from langchain.retrievers import WikipediaRetriever\n",
    "\n",
    "import pinecone\n",
    "import time\n",
    "\n",
    "from config import OPENAI_API_KEY, PINECONE_API_KEY, PINECONE_ENVIRONMENT, PINECONE_INDEX_NAME, EMBEDDING_MODEL"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper Functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_source(result):\n",
    "    sources = result[\"source_documents\"]\n",
    "    for i in range(min(3, len(sources))):\n",
    "        print(\"=\"*60)\n",
    "        print(f\"Source [{i+1}] \\t File: [{sources[i].metadata['source']}] \\t Page: [{int(sources[i].metadata['page'])}]\")\n",
    "        print(\"=\"*60)\n",
    "        print(sources[i].page_content)\n",
    "        print(\"=\"*60)\n",
    "        print()\n",
    "\n",
    "\n",
    "\n",
    "def print_wiki_source(result):\n",
    "    sources = result[\"source_documents\"]\n",
    "    for i in range(min(3, len(sources))):\n",
    "        print(\"=\"*60)\n",
    "        print(f\"Source [{i+1}] \\t Title: [{sources[i].metadata['title']}]\")\n",
    "        print(f\"URL: [{sources[i].metadata['source']}]\")\n",
    "        print(\"=\"*60)\n",
    "        print(sources[i].page_content)\n",
    "        print(\"=\"*60)\n",
    "        print()\n",
    "\n",
    "\n",
    "\n",
    "def print_answer(result):\n",
    "    print(\"=\"*30)\n",
    "    print(\" \"*10 + \"Question\")\n",
    "    print(\"=\"*30)\n",
    "    print(result[\"question\"])\n",
    "    print(\"=\"*30)\n",
    "    print()\n",
    "\n",
    "    print(\"=\"*30)\n",
    "    print(\" \"*10 + \"Answer\")\n",
    "    print(\"=\"*30)\n",
    "    print(result[\"answer\"])\n",
    "    print(\"=\"*30)\n",
    "    print()\n",
    "    \n",
    "\n",
    "\n",
    "def if_existed(query, vectorstore):\n",
    "    is_existed = True\n",
    "    try:\n",
    "        res = vectorstore.max_marginal_relevance_search(\n",
    "            query=query,\n",
    "            k=4,\n",
    "            fetch_k=20,\n",
    "            lambda_mult=0.5\n",
    "        )\n",
    "        if len(res) == 0:\n",
    "            is_existed = False\n",
    "    except:\n",
    "        is_existed = False\n",
    "    \n",
    "    return is_existed\n",
    "\n",
    "\n",
    "\n",
    "def search(query, vector_chain, vectorstore, wiki_chain):\n",
    "    if if_existed(query, vectorstore):\n",
    "        res = vector_chain({\"question\": query})\n",
    "        print_answer(res)\n",
    "        print_source(res)\n",
    "    else:\n",
    "        print(\"Let me grab Wikipedia to answer your question......\")\n",
    "        res = wiki_chain({\"question\": query})\n",
    "        print_answer(res)\n",
    "        print_wiki_source(res)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize OpenAI Chat Model\n",
    "\n",
    "Langchain offers LLMs and Chat models. For the purpose of conversation, [chat model](https://python.langchain.com/docs/modules/model_io/models/chat/) is used. By default, `gpt-3.5-turbo` [OpenAI model](https://platform.openai.com/docs/models/gpt-3-5) is used. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "OpenAI initialization: OK\n",
      "==============================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY)\n",
    "\n",
    "embedding_model = OpenAIEmbeddings(\n",
    "    openai_api_key=OPENAI_API_KEY, \n",
    "    model=EMBEDDING_MODEL\n",
    ")\n",
    "\n",
    "print(\"=\"*30)\n",
    "print(\"OpenAI initialization: OK\")\n",
    "print(\"=\"*30)\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize Pinecone\n",
    "\n",
    "If the index does not exist in your Pinecone, it will automatically create a new one. \n",
    "\n",
    "- `metric='cosine'`: This is often used to find similarities between different documents. The advantage is that the scores are normalized to [-1,1] range. You can choose other options listed [here](https://docs.pinecone.io/docs/indexes#distance-metrics).\n",
    "- `dimension=1536`: The OpenAI `text-embedding-ada-002` embedding has a dimension of 1536\n",
    "- There is a limitation for the free plan for Pinecone. Please refer to the [starter plan](https://docs.pinecone.io/docs/indexes#starter-plan) for more details"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "Pinecone initialization: OK\n",
      "{'dimension': 1536,\n",
      " 'index_fullness': 0.0223,\n",
      " 'namespaces': {'': {'vector_count': 2230}},\n",
      " 'total_vector_count': 2230}\n",
      "==============================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "pinecone.init(\n",
    "    api_key=PINECONE_API_KEY,\n",
    "    environment=PINECONE_ENVIRONMENT\n",
    ")\n",
    "\n",
    "if PINECONE_INDEX_NAME not in pinecone.list_indexes():\n",
    "    # we create a new index if it doesn't exist\n",
    "    pinecone.create_index(\n",
    "        name=PINECONE_INDEX_NAME,\n",
    "        metric='cosine',\n",
    "        dimension=1536  # 1536 dim of text-embedding-ada-002\n",
    "    )\n",
    "    # wait for index to be initialized\n",
    "    time.sleep(1)\n",
    "\n",
    "pinecone_index = pinecone.Index(PINECONE_INDEX_NAME)\n",
    "pinecone_stats = pinecone_index.describe_index_stats()\n",
    "print(\"=\"*30)\n",
    "print(\"Pinecone initialization: OK\")\n",
    "print(pinecone_stats)\n",
    "print(\"=\"*30)\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retriever\n",
    "\n",
    "### Pinecone retriever\n",
    "\n",
    "A Pinecone vector store is used as a retriever.\n",
    "\n",
    "- `search_type=\"mmr\"`: [Maximal Marginal Relevance (MMR)](https://medium.com/tech-that-works/maximal-marginal-relevance-to-rerank-results-in-unsupervised-keyphrase-extraction-22d95015c7c5) is used for search method. MMR can include diversity in the search results while maintaining query relevance at the same time. Other search types can be found [here](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.pinecone.Pinecone.html#langchain.vectorstores.pinecone.Pinecone.as_retriever).\n",
    "  - `\"lambda_mult\": 0.5`: This offers the optimal mix of diversity and accuracy in the result set\n",
    "- `\"k\": 5`: Include top 5 search result in order to give the model more context\n",
    "\n",
    "\n",
    "### Wikipedia Retriever\n",
    "\n",
    "Include a [Wikipedia Retriever](https://python.langchain.com/docs/integrations/document_loaders/wikipedia) to handle the situation where the search target does not exist in the Pinecone vector store. \n",
    "\n",
    "I think there is bug where the [Langchain Pinecone Search function](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.pinecone.Pinecone.html#langchain.vectorstores.pinecone.Pinecone.max_marginal_relevance_search) raises a Validation Error when it cannot find the vector from the Pinecone vector store. As a result, I created a [GitHub Issue](https://github.com/langchain-ai/langchain/issues/10111) to the Langchain community. However, this bug seems only happen occasionally. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "Pinecone retriever: OK\n",
      "==============================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "vectorstore = Pinecone(pinecone_index, embedding_model, \"text\")\n",
    "retriever = vectorstore.as_retriever(\n",
    "    search_type=\"mmr\", \n",
    "    search_kwargs={\n",
    "                    \"k\": 5,\n",
    "                    \"lambda_mult\": 0.5, # the optimal mix of diversity and accuracy in the result set\n",
    "                    }\n",
    ")\n",
    "\n",
    "wiki_retriever = WikipediaRetriever()\n",
    "\n",
    "print(\"=\"*30)\n",
    "print(\"Pinecone retriever: OK\")\n",
    "print(\"=\"*30)\n",
    "print()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chat Memory\n",
    "\n",
    "Conversation Summary Memory is used based on [this article](https://www.pinecone.io/learn/series/langchain/langchain-conversational-memory/). This type of memory generates less tokens than the Conversation Buffer Memory, so it can keep the chat history in a more efficient way. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "Chat memory: OK\n",
      "==============================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "memory = ConversationSummaryMemory(\n",
    "    llm=llm, \n",
    "    memory_key=\"chat_history\", \n",
    "    input_key='question', \n",
    "    output_key='answer', \n",
    "    return_messages=True\n",
    ")\n",
    "print(\"=\"*30)\n",
    "print(\"Chat memory: OK\")\n",
    "print(\"=\"*30)\n",
    "print()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conversational Retrieval Chain\n",
    "\n",
    "Create two Conversational Retrieval Chains for both Pinecone vector store and Wikipedia retriever. \n",
    "\n",
    "- `chain_type=\"stuff\"`: For demo purpose, `\"stuff\"` can be used to include all relevant documents. However, `\"map-reduce\"` or `\"refine\"` can be used as the number of PDFs grows. For more details, please refer [here](https://www.youtube.com/watch?v=DXmiJKrQIvg&t=300s&ab_channel=SophiaYang).\n",
    "\n",
    "For more information, please refer [here](https://python.langchain.com/docs/use_cases/chatbots). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "Conversational Retrieval Chain: OK\n",
      "==============================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "conversation_qa_chain = ConversationalRetrievalChain.from_llm(\n",
    "    llm=llm,\n",
    "    chain_type=\"stuff\",\n",
    "    retriever=retriever,\n",
    "    memory=memory,\n",
    "    return_source_documents=True, \n",
    "    verbose=False\n",
    ")\n",
    "\n",
    "wiki_qa_chain = ConversationalRetrievalChain.from_llm(\n",
    "    llm=llm,\n",
    "    retriever=wiki_retriever,\n",
    "    memory=memory,\n",
    "    return_source_documents=True,\n",
    "    verbose=False\n",
    ")\n",
    "\n",
    "print(\"=\"*30)\n",
    "print(\"Conversational Retrieval Chain: OK\")\n",
    "print(\"=\"*30)\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test 1: Ask a query to the Pinecone vector store"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "          Question\n",
      "==============================\n",
      "How to diagnose Resistant Hypertension?\n",
      "==============================\n",
      "\n",
      "==============================\n",
      "          Answer\n",
      "==============================\n",
      "The process for diagnosing Resistant Hypertension involves several steps. First, proper office and out-of-office blood pressure measurements are taken into account. This includes ambulatory blood pressure monitoring (ABPM). Second, optimization of pharmacotherapy is considered, taking into account clinical inertia. This means ensuring that the patient is on three or more drugs at optimally tolerated dosages. Finally, other possible causes of uncontrolled blood pressure are ruled out. It is important to note that the specific diagnostic algorithm may vary, and it is best to consult with a healthcare professional for an individualized assessment and diagnosis.\n",
      "==============================\n",
      "\n",
      "============================================================\n",
      "Source [1] \t File: [data/paper3.pdf] \t Page: [4]\n",
      "============================================================\n",
      "R\n",
      "C\n",
      "Conﬁrm diagnosis of true resistant hypertension\n",
      "Figure 1. Diagnostic algorithm for a patient with suspected resistant hypertension. ABPM, ambulatory blood pressure monitoring; BP, blood\n",
      "pressure; HT, hypertension. *Three or more drugs, at optimally tolerated dosages\n",
      "============================================================\n",
      "\n",
      "============================================================\n",
      "Source [2] \t File: [data/paper3.pdf] \t Page: [9]\n",
      "============================================================\n",
      "istant Hypertension Guideline\n",
      "============================================================\n",
      "\n",
      "============================================================\n",
      "Source [3] \t File: [data/paper3.pdf] \t Page: [9]\n",
      "============================================================\n",
      "Canadian initiative. Can J Cardiol 2006;22:559-64 .\n",
      "12.Noubiap JJ, Nansseu JR, Nyaga UF, et al. Global prevalence of resistant\n",
      "hypertension: a meta-analysis of data from 3.2 million patients.\n",
      "============================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "query = \"How to diagnose Resistant Hypertension?\"\n",
    "search(query, conversation_qa_chain, vectorstore, wiki_qa_chain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test 2: Ask a follow-up question\n",
    "\n",
    "The model can recognize `it` refers to `Resistant Hypertension` in the previous question. This is called [\"anaphora\"](https://youtu.be/FFRnDRcbQQU?si=7Uoc3204dw_mzQz4&t=1792) in linguistics. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "          Question\n",
      "==============================\n",
      "How to treat it?\n",
      "==============================\n",
      "\n",
      "==============================\n",
      "          Answer\n",
      "==============================\n",
      "The diagnostic algorithm for diagnosing Resistant Hypertension includes the following steps:\n",
      "\n",
      "1. Review and reiterate lifestyle measures such as sodium and potassium intake, stress management, exercise, and alcohol consumption.\n",
      "\n",
      "2. Improve adherence to medication regimen. This may include assessing and addressing any barriers to medication adherence.\n",
      "\n",
      "3. Eliminate drugs and substances that may be causing higher blood pressure. For example, certain medications or substances like calcineurin inhibitors or licorice can contribute to elevated blood pressure.\n",
      "\n",
      "4. Confirm the diagnosis of true resistant hypertension. This may involve ambulatory blood pressure monitoring (ABPM) to assess blood pressure over a 24-hour period.\n",
      "\n",
      "It's important to note that these steps are based on the guidelines provided by the American Heart Association and may vary depending on individual patient factors.\n",
      "==============================\n",
      "\n",
      "============================================================\n",
      "Source [1] \t File: [data/paper3.pdf] \t Page: [4]\n",
      "============================================================\n",
      "R\n",
      "C\n",
      "Conﬁrm diagnosis of true resistant hypertension\n",
      "Figure 1. Diagnostic algorithm for a patient with suspected resistant hypertension. ABPM, ambulatory blood pressure monitoring; BP, blood\n",
      "pressure; HT, hypertension. *Three or more drugs, at optimally tolerated dosages\n",
      "============================================================\n",
      "\n",
      "============================================================\n",
      "Source [2] \t File: [data/paper3.pdf] \t Page: [9]\n",
      "============================================================\n",
      "istant Hypertension Guideline\n",
      "============================================================\n",
      "\n",
      "============================================================\n",
      "Source [3] \t File: [data/paper3.pdf] \t Page: [9]\n",
      "============================================================\n",
      "Carey RM, Calhoun DA, Bakris GL, et al. Resistant hypertension:\n",
      "detection, evaluation, and management: a scienti ﬁc statement from the\n",
      "American Heart Association. Hypertension 2018;72:e53-90 .\n",
      "16.Bangalore\n",
      "============================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "query = \"How to treat it?\"\n",
    "search(query, conversation_qa_chain, vectorstore, wiki_qa_chain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test 3: Clear the chat history\n",
    "\n",
    "If the chat history is cleared, the model cannot know what \"it\" refers to. In others words, the model cannot know the antecedent of the anaphor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "          Question\n",
      "==============================\n",
      "How to treat it?\n",
      "==============================\n",
      "\n",
      "==============================\n",
      "          Answer\n",
      "==============================\n",
      "Based on the given context, it is not clear what \"it\" refers to. Could you please provide more specific information or clarify your question?\n",
      "==============================\n",
      "\n",
      "============================================================\n",
      "Source [1] \t File: [data/paper1.pdf] \t Page: [5]\n",
      "============================================================\n",
      "or combined heart-liver transplantation (HLivT) are treat-\n",
      "============================================================\n",
      "\n",
      "============================================================\n",
      "Source [2] \t File: [data/paper3.pdf] \t Page: [4]\n",
      "============================================================\n",
      " out white coat eﬀect, preferably with an \n",
      "ABPM†\n",
      "Consider/assess for nonadherenceOpƟmize drug therapy, taking into account the 4Ds \n",
      "(drug dose, duraƟon, and diureƟc choice)\n",
      "============================================================\n",
      "\n",
      "============================================================\n",
      "Source [3] \t File: [data/paper1.pdf] \t Page: [16]\n",
      "============================================================\n",
      " universal pro-phylaxis. A preemptive treatment approach should be\n",
      "============================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "memory.clear()\n",
    "query = \"How to treat it?\"\n",
    "search(query, conversation_qa_chain, vectorstore, wiki_qa_chain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}