LangChainforLLMApplicationDevelopment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "49783e1a-c4cf-4e81-8af5-9fc14656a718",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "- Open-source development framework for LLM applications\n",
    "- Python and JavaScript (TypeScript) packages\n",
    "- Focused on compostion and modularity\n",
    "\n",
    "Key value adds:\n",
    "1. Modular components\n",
    "2. Use cases: Common ways to combine components"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a979d91f-9ab6-4463-bfe7-6c8a2d7aa6bc",
   "metadata": {},
   "source": [
    "## Components of LangChain"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5687c413-567f-4eaa-902b-0a87abfffac1",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Models\n",
    "- LLMs: 20+ integrations\n",
    "- Chat Models\n",
    "- Text Embedding Models: 10+ integrations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e4350ad-f9a4-4a47-9120-c30005cbfc47",
   "metadata": {},
   "source": [
    "### Prompts\n",
    "- Prompt Templates\n",
    "- Output Parsers: 5+ implementations\n",
    "    - Retry/fixing logic\n",
    "- Example Selectors: 5+ implementations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d94ff46-65e4-4b82-a844-b2f87dc9bb32",
   "metadata": {},
   "source": [
    "### Indexes\n",
    "- Document Loaders: 50+ implementation\n",
    "- Text Splitters: 10+ implementation\n",
    "- Vector Stores: 10+ integrations\n",
    "- Retrievers: 5+ integrations/implementations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fba5b3c4-9b8d-4903-8199-88ec1f31a16e",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Chains\n",
    "- Prompt + LLM + Output parsing\n",
    "- Can be used as building blocks for longer chains\n",
    "- More application specific chains: 20+ types"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c48979f9-5537-4e20-9bd0-529a33ac35c1",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Agents\n",
    "- Agent Types: 5+ types\n",
    "    - Algorithms for getting LLMs to use tools\n",
    "- Agent Toolkits: 10+ implementation\n",
    "    - Agents armed with specific tools for a specific application"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9fc3269-8d63-4cdd-bf89-24cd79561d4d",
   "metadata": {},
   "source": [
    "# Models, Prompts and Parsers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "id": "ac999087-01f8-4083-878a-7ab2321ad93d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ[\"NVIDIA_API_KEY\"] = \"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8393f679-8bef-46b0-8e40-0e43f8ef749e",
   "metadata": {},
   "source": [
    "## Direct API Calls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85c4d074-7b1b-4f1c-9b21-b761502ee91c",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### NVIDIA Implementation\n",
    "Let's start with direct API calls to NVIDIA ~~OpenAI~~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0a88a0d5-b6ab-42bd-baaf-98af9c332c88",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1+1 is equal to 2. This is a basic arithmetic operation known as addition, where 1 is added to another 1 to produce a sum of 2.\n"
     ]
    }
   ],
   "source": [
    "import requests, base64, os\n",
    "\n",
    "def get_completion(prompt):\n",
    "    invoke_url = \"https://ai.api.nvidia.com/v1/vlm/microsoft/phi-3-vision-128k-instruct\"\n",
    "    \n",
    "    headers = {\n",
    "      \"Authorization\": f\"Bearer {os.environ.get('NVIDIA_API_KEY')}\",\n",
    "      \"Accept\": \"application/json\"\n",
    "    }\n",
    "    \n",
    "    payload = {\n",
    "      \"messages\": [\n",
    "        {\n",
    "          \"role\": \"user\",\n",
    "          \"content\": f'{prompt}'\n",
    "        }\n",
    "      ],\n",
    "      \"max_tokens\": 512,\n",
    "      \"temperature\": 1.00,\n",
    "      \"top_p\": 0.70\n",
    "    }\n",
    "    response = requests.post(invoke_url, headers=headers, json=payload)\n",
    "    return response.json()['choices'][0]['message']['content']\n",
    "\n",
    "\n",
    "# response = get_completion(\"What is 1+1?\").json()\n",
    "# print(response['choices'][0]['message']['content'])\n",
    "response = get_completion(\"What is 1+1?\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f84ce48-8f04-4a33-8af7-ab8f7be215b8",
   "metadata": {},
   "source": [
    "### Ollama Implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "76c1d985-cd93-45b3-b13b-8b8a86bb5a70",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"That's an easy one!\\n\\nThe answer is: 2!\""
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "invoke_url = \"http://localhost:11434/api/generate\"\n",
    "\n",
    "def get_completion_local(prompt):\n",
    "    headers = {\n",
    "        \"Accept\": \"application/json\"\n",
    "    }\n",
    "    \n",
    "    payload = {\n",
    "      \"model\": \"llama3\",\n",
    "      \"prompt\":f\"{prompt}\",\n",
    "      \"stream\": False\n",
    "    }\n",
    "    \n",
    "    response = requests.post(invoke_url, json=payload)\n",
    "    return response.json()[\"response\"]\n",
    "\n",
    "get_completion_local(\"What is 1+1?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12b21b18-8f38-43bc-bcff-f6e69073edc5",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### OpenAI Implementation: Just in Case"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "60c469d5-b42c-40eb-a882-2b9c33c92903",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "10936809-ca37-430d-ae2c-4684c9a411f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import openai\n",
    "\n",
    "# os.environ[\"NVIDIA_API_KEY\"] = XXXXXXXXXXXXXX\n",
    "# openai.api_key = os.environ['OPENAI_API_KEY']\n",
    "# llm_model = \"gpt-3.5-turbo\"\n",
    "\n",
    "# def get_completion(prompt, model=llm_model):\n",
    "#     messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "#     response = openai.ChatCompletion.create(\n",
    "#         model=model,\n",
    "#         messages=messages,\n",
    "#         temperature=0, \n",
    "#     )\n",
    "#     return response.choices[0].message[\"content\"]\n",
    "\n",
    "# get_completion(\"What is 1+1?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f8519f1-399e-4431-b8e8-d52fa10707d7",
   "metadata": {},
   "source": [
    "### Langchain Abstractions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "90ea44b2-8f7c-47f4-aac4-7877f4f04425",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Translate the text that is delimited by triple backticks \n",
      "into a style that is American English in a calm and respectful tone\n",
      "\n",
      "text: ```\n",
      "Arrr, I be fuming that me blender lid flew off and splattered me kitchen walls with smoothie! And to make matters worse,the warranty don't cover the cost of cleaning up me kitchen. I need yer help right now, matey!\n",
      "```\n",
      "\n"
     ]
    }
   ],
   "source": [
    "customer_email = \"\"\"\n",
    "Arrr, I be fuming that me blender lid \\\n",
    "flew off and splattered me kitchen walls \\\n",
    "with smoothie! And to make matters worse,\\\n",
    "the warranty don't cover the cost of \\\n",
    "cleaning up me kitchen. I need yer help \\\n",
    "right now, matey!\n",
    "\"\"\"\n",
    "\n",
    "style = \"\"\"American English \\\n",
    "in a calm and respectful tone\n",
    "\"\"\"\n",
    "\n",
    "prompt = f\"\"\"Translate the text \\\n",
    "that is delimited by triple backticks \n",
    "into a style that is {style}\n",
    "text: ```{customer_email}```\n",
    "\"\"\"\n",
    "\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6c9af0b7-d00a-4b23-a6a1-41636ab7a465",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Oh, I am quite frustrated that my blender lid came off and caused a mess in my kitchen. And to add insult to injury, the warranty does not cover the cost of cleaning up my kitchen. I need your help right away, matey!\n"
     ]
    }
   ],
   "source": [
    "print(get_completion(prompt))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f3091d68-5555-4fbf-8a14-a3f8fbf1e470",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here is the translated text in a calm and respectful tone using American English:\n",
      "\n",
      "```\n",
      "Gosh, I'm really upset that my blender lid came loose and splattered smoothie all over my kitchen walls! And to make things worse, the warranty doesn't cover the cost of cleaning up the mess. I could really use your help right now.\n",
      "```\n",
      "\n",
      "Let me know if you'd like any further adjustments!\n"
     ]
    }
   ],
   "source": [
    "print(get_completion_local(prompt))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da73bbcb-648d-4bba-9a49-0b440e3cfdcd",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## ChatAPI LangChain"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28cbbe4f-bcd8-4978-be1a-75ab797b1ad5",
   "metadata": {},
   "source": [
    "### Why Use Prompt Templates?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb0572da-cccd-43f0-a3fd-49abca22f680",
   "metadata": {},
   "source": [
    "![Why Use Prompt Templates](data/images/why-use-prompt-templates.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1500d95-fb63-4ddf-ac35-d883714d04b5",
   "metadata": {},
   "source": [
    "![LangChain output parsing works with prompt templates](data/images/output-parsing-works-with-prompts.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a4b9d15f-6d69-4af8-8434-b6ab3e03640c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install --upgrade langchain\n",
    "# !pip install langchain-nvidia-ai-endpoints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b8bd4036-d849-4054-b70c-1996b75d70f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "template_string = \"\"\"Translate the text \\\n",
    "that is delimited by triple backticks \\\n",
    "into a style that is {style}. \\\n",
    "text: ```{text}```\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "9c89015b-3a02-4296-8b99-b036bc3b0328",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt_template = ChatPromptTemplate.from_template(template_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1c75c40c-139c-4227-92a5-3a2776e7aa11",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PromptTemplate(input_variables=['style', 'text'], template='Translate the text that is delimited by triple backticks into a style that is {style}. text: ```{text}```\\n')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prompt_template.messages[0].prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "52c4285e-7f94-42f7-94d4-9e7661d848b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['style', 'text']"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prompt_template.messages[0].prompt.input_variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3fdaacfd-db97-4ed9-ac2d-ee86deec7db0",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_style = \"\"\"American English \\\n",
    "in a calm and respectful tone\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "70029453-f704-42eb-8282-c8ba5c4f5aa1",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_email = \"\"\"\n",
    "Arrr, I be fuming that me blender lid \\\n",
    "flew off and splattered me kitchen walls \\\n",
    "with smoothie! And to make matters worse, \\\n",
    "the warranty don't cover the cost of \\\n",
    "cleaning up me kitchen. I need yer help \\\n",
    "right now, matey!\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "5be1cd46-a24b-4592-9f1d-991562bbbcfd",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_messages = prompt_template.format_messages(\n",
    "                    style=customer_style,\n",
    "                    text=customer_email)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c627c060-3c5d-4b07-83d4-d536303bcb08",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[HumanMessage(content=\"Translate the text that is delimited by triple backticks into a style that is American English in a calm and respectful tone\\n. text: ```\\nArrr, I be fuming that me blender lid flew off and splattered me kitchen walls with smoothie! And to make matters worse, the warranty don't cover the cost of cleaning up me kitchen. I need yer help right now, matey!\\n```\\n\")]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "customer_messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "7451efc4-ccd7-4967-b613-28b79d67464e",
   "metadata": {},
   "outputs": [],
   "source": [
    "service_reply = \"\"\"Hey there customer, \\\n",
    "the warranty does not cover \\\n",
    "cleaning expenses for your kitchen \\\n",
    "because it's your fault that \\\n",
    "you misused your blender \\\n",
    "by forgetting to put the lid on before \\\n",
    "starting the blender. \\\n",
    "Tough luck! See ya!\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "32d7991a-bea1-468b-82f8-8572bd63f125",
   "metadata": {},
   "outputs": [],
   "source": [
    "service_style_pirate = \"\"\"\\\n",
    "a polite tone \\\n",
    "that speaks in English Pirate\\\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "6a40676d-e5d6-4c0a-bc4a-fefbbffa7b33",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Translate the text that is delimited by triple backticks into a style that is a polite tone that speaks in English Pirate. text: ```Hey there customer, the warranty does not cover cleaning expenses for your kitchen because it's your fault that you misused your blender by forgetting to put the lid on before starting the blender. Tough luck! See ya!\n",
      "```\n",
      "\n"
     ]
    }
   ],
   "source": [
    "service_messages = prompt_template.format_messages(\n",
    "    style=service_style_pirate,\n",
    "    text=service_reply)\n",
    "\n",
    "print(service_messages[0].content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3783214a-9e92-49cd-808e-eaea51e9f5f0",
   "metadata": {},
   "source": [
    "### ChatNVIDIA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 309,
   "id": "8e10663a-7901-491f-b103-26df06e401e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n",
    "\n",
    "# To control the randomness and creativity of the generated\n",
    "# text by an LLM, use temperature = 0.0\n",
    "\n",
    "llm_nvidia = ChatNVIDIA(model=\"mistralai/mixtral-8x7b-instruct-v0.1\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "3a739d24-c9fb-492b-8684-7ef4d9a3a12c",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_response = llm_nvidia.invoke(customer_messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ac5caad8-2214-4b54-b636-2e98f2b31b56",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " I'm really upset that the lid of my blender came off and splattered my kitchen walls with smoothie! To make matters worse, the warranty does not cover the cost of cleaning up my kitchen. I need your help right now, friend!\n"
     ]
    }
   ],
   "source": [
    "print(customer_response.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "0665b179-66e5-48f4-9160-84ea7e45cd64",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `BaseChatModel.__call__` was deprecated in langchain-core 0.1.7 and will be removed in 0.3.0. Use invoke instead.\n",
      "  warn_deprecated(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Ahoy there, me hearty customer! I'm afraid I've got some unfortunate news for ye. The warranty on yer kitchen appliance, specifically the blender, does not cover the costs of cleanin' up after a mishap. It seems ye may have forgotten to put the lid on before startin' the blender, which led to the mess. I'm truly sorry, but them's the rules, I'm afraid. We can't be coverin' every little mishap, as much as we'd like to help. I hope ye understand and have a smoother sailin' in the future. Fare thee well, me friend!\n"
     ]
    }
   ],
   "source": [
    "service_response = llm_nvidia(service_messages)\n",
    "print(service_response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b9a77b8-72a7-4981-b78e-f926ecf6d241",
   "metadata": {},
   "source": [
    "### ChatOllama"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "4589c977-38dd-4424-b8a7-190cba526ae0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.chat_models import ChatOllama\n",
    "\n",
    "llm_ollama = ChatOllama(model=\"llama3\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "cae84460-caf8-49d6-8418-12972e8ff788",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here's the translation:\n",
      "\n",
      "```\n",
      "Ugh, I'm absolutely livid that my blender lid flew off and splattered smoothie all over my kitchen walls! To make things worse, the warranty doesn't cover the cost of cleaning up the mess. I really need some help right now - can you lend a hand? \n",
      "```\n",
      "\n",
      "I've kept the tone calm and respectful, while still conveying your frustration with the situation. I've also used American English spellings (e.g., \"absolutely\" instead of \"be\", \"livid\" instead of \"fuming\", etc.) to ensure the translation is suitable for an American audience.\n"
     ]
    }
   ],
   "source": [
    "customer_response = llm_ollama(customer_messages)\n",
    "print(customer_response.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "2b55c2bb-c015-4d75-9c1e-0e59149293de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Arrrr, greetings me hearty customer!\n",
      "\n",
      "I be afraid that yer warranty don't cover them fancy cleanin' expenses fer yer galley (that's kitchen speak, matey). Seemingly, ye got yerself into a spot o' trouble by forgettin' to put the lid on yer trusty blender afore startin' it up. Shiver me timbers! A bit o' common sense be in order here, don't ye think?\n",
      "\n",
      "Well, matey, it seems like ye've got yerself a case o' \"Tough Luck\" comin' atcha! But never fear, there be more where that came from! Just keep in mind that next time ye set sail with yer blender, make sure to have the lid securely fastened, or ye might just find yerself walkin' the plank!\n",
      "\n",
      "Fair winds and following seas, me hearty! May yer kitchen be as smooth as a calm sea on a sunny day!\n"
     ]
    }
   ],
   "source": [
    "service_response = llm_ollama(service_messages)\n",
    "print(service_response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe63523c-d0bf-4b3a-9a62-ce1876e2bc25",
   "metadata": {},
   "source": [
    "### ChatOpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "056b7765-dc4b-4817-aed0-b2bff14cae0b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from langchain.chat_models import ChatOpenAI\n",
    "# llm_openai = ChatOpenAI(temperature=0.0, model=llm_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16d80722-b653-4a4e-a024-f935964dde97",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## Output Parsers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b957b58-aa7f-4f25-9c0e-0285e5af7339",
   "metadata": {},
   "source": [
    "How we would like the LLM output to look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "f7ae9d63-4efd-4c89-a2d5-7a799b2b3da1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'gift': False, 'delivery_days': 5, 'price_value': 'pretty affordable!'}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{\n",
    "  \"gift\": False,\n",
    "  \"delivery_days\": 5,\n",
    "  \"price_value\": \"pretty affordable!\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "48593a80-fb6c-444a-ae1d-5b2a2e2e5279",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_review = \"\"\"\\\n",
    "This leaf blower is pretty amazing.  It has four settings:\\\n",
    "candle blower, gentle breeze, windy city, and tornado. \\\n",
    "It arrived in two days, just in time for my wife's \\\n",
    "anniversary present. \\\n",
    "I think my wife liked it so much she was speechless. \\\n",
    "So far I've been the only one using it, and I've been \\\n",
    "using it every other morning to clear the leaves on our lawn. \\\n",
    "It's slightly more expensive than the other leaf blowers \\\n",
    "out there, but I think it's worth it for the extra features.\n",
    "\"\"\"\n",
    "\n",
    "review_template = \"\"\"\\\n",
    "For the following text, extract the following information:\n",
    "\n",
    "gift: Was the item purchased as a gift for someone else? \\\n",
    "Answer True if yes, False if not or unknown.\n",
    "\n",
    "delivery_days: How many days did it take for the product \\\n",
    "to arrive? If this information is not found, output -1.\n",
    "\n",
    "price_value: Extract any sentences about the value or price,\\\n",
    "and output them as a comma separated Python list.\n",
    "\n",
    "Format the output as JSON with the following keys:\n",
    "gift\n",
    "delivery_days\n",
    "price_value\n",
    "\n",
    "text: {text}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "48f751b1-8c68-454b-a8ce-99316a7ec90e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input_variables=['text'] messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['text'], template='For the following text, extract the following information:\\n\\ngift: Was the item purchased as a gift for someone else? Answer True if yes, False if not or unknown.\\n\\ndelivery_days: How many days did it take for the product to arrive? If this information is not found, output -1.\\n\\nprice_value: Extract any sentences about the value or price,and output them as a comma separated Python list.\\n\\nFormat the output as JSON with the following keys:\\ngift\\ndelivery_days\\nprice_value\\n\\ntext: {text}\\n'))]\n"
     ]
    }
   ],
   "source": [
    "from langchain.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt_template = ChatPromptTemplate.from_template(review_template)\n",
    "print(prompt_template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "19fb8976-e916-4533-a3fe-19dbb0368a7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = prompt_template.format_messages(text=customer_review)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "1e0f89c3-ac05-4fa1-a8ca-47ff4ea2a5ca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " {\n",
      "\"gift\": true,\n",
      "\"delivery_days\": 2,\n",
      "\"price\\_value\": \"It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features.\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "response = llm_nvidia(messages)\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "9d2d0c8f-76dc-44ec-ac50-7802778cc2fd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here is the extracted information in JSON format:\n",
      "\n",
      "```\n",
      "{\n",
      "    \"gift\": True,\n",
      "    \"delivery_days\": 2,\n",
      "    \"price_value\": [\"It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features.\"]\n",
      "}\n",
      "```\n",
      "\n",
      "Note: The `price_value` key has a list value because there is only one sentence that provides information about the price or value of the product.\n"
     ]
    }
   ],
   "source": [
    "response = llm_ollama(messages)\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "48556bb8-54f6-4fc3-8918-54d86697e0ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "# response = llm_openai(messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "d80c02ee-d76d-473f-888e-1f384e59eab8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "str"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cac7769-c6f7-4256-a128-08bd23b1a6ab",
   "metadata": {},
   "source": [
    "#### Parse the LLM output string into a Python dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "71602179-367a-4810-884c-224f97e581ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.output_parsers import ResponseSchema\n",
    "from langchain.output_parsers import StructuredOutputParser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "46435300-51ac-44e3-92e5-8de158e17208",
   "metadata": {},
   "outputs": [],
   "source": [
    "gift_schema = ResponseSchema(name=\"gift\",\n",
    "                             description=\"Was the item purchased\\\n",
    "                             as a gift for someone else? \\\n",
    "                             Answer 'True' if yes,\\\n",
    "                             'False' if not or unknown. Answer should be within quotes\")\n",
    "delivery_days_schema = ResponseSchema(name=\"delivery_days\",\n",
    "                                      description=\"How many days\\\n",
    "                                      did it take for the product\\\n",
    "                                      to arrive? If this \\\n",
    "                                      information is not found,\\\n",
    "                                      output -1.\")\n",
    "price_value_schema = ResponseSchema(name=\"price_value\",\n",
    "                                    description=\"Extract any\\\n",
    "                                    sentences about the value or \\\n",
    "                                    price, and output them as a \\\n",
    "                                    comma separated Python list.\")\n",
    "\n",
    "response_schemas = [gift_schema, \n",
    "                    delivery_days_schema,\n",
    "                    price_value_schema]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "32358d82-91c9-4db5-a32c-a4b6d8760097",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_parser = StructuredOutputParser.from_response_schemas(response_schemas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "f3d0fd8f-b2be-4cb7-b099-9c0a792c53b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "format_instructions = output_parser.get_format_instructions()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "d58c8aa5-6523-4754-8a52-9d5f23029ba7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output should be a markdown code snippet formatted in the following schema, including the leading and trailing \"```json\" and \"```\":\n",
      "\n",
      "```json\n",
      "{\n",
      "\t\"gift\": string  // Was the item purchased                             as a gift for someone else?                              Answer 'True' if yes,                             'False' if not or unknown. Answer should be within quotes\n",
      "\t\"delivery_days\": string  // How many days                                      did it take for the product                                      to arrive? If this                                       information is not found,                                      output -1.\n",
      "\t\"price_value\": string  // Extract any                                    sentences about the value or                                     price, and output them as a                                     comma separated Python list.\n",
      "}\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "print(format_instructions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "396bb245-ab28-4197-8ec1-13cde1e407c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "review_template_2 = \"\"\"\\\n",
    "For the following text, extract the following information. \n",
    "Don't respond to me back.\n",
    "Just give me the information.\n",
    "\n",
    "gift: Was the item purchased as a gift for someone else? \\\n",
    "Answer True if yes, False if not or unknown.\n",
    "\n",
    "delivery_days: How many days did it take for the product\\\n",
    "to arrive? If this information is not found, output -1.\n",
    "\n",
    "price_value: Extract any sentences about the value or price,\\\n",
    "and output them as a comma separated Python list.\n",
    "\n",
    "text: {text}\n",
    "\n",
    "{format_instructions}\n",
    "\"\"\"\n",
    "\n",
    "prompt = ChatPromptTemplate.from_template(template=review_template_2)\n",
    "\n",
    "messages = prompt.format_messages(text=customer_review, format_instructions=format_instructions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "ed8bc18f-614e-440d-b48a-640a4d141b56",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[HumanMessage(content='For the following text, extract the following information. \\nDon\\'t respond to me back.\\nJust give me the information.\\n\\ngift: Was the item purchased as a gift for someone else? Answer True if yes, False if not or unknown.\\n\\ndelivery_days: How many days did it take for the productto arrive? If this information is not found, output -1.\\n\\nprice_value: Extract any sentences about the value or price,and output them as a comma separated Python list.\\n\\ntext: This leaf blower is pretty amazing.  It has four settings:candle blower, gentle breeze, windy city, and tornado. It arrived in two days, just in time for my wife\\'s anniversary present. I think my wife liked it so much she was speechless. So far I\\'ve been the only one using it, and I\\'ve been using it every other morning to clear the leaves on our lawn. It\\'s slightly more expensive than the other leaf blowers out there, but I think it\\'s worth it for the extra features.\\n\\n\\nThe output should be a markdown code snippet formatted in the following schema, including the leading and trailing \"```json\" and \"```\":\\n\\n```json\\n{\\n\\t\"gift\": string  // Was the item purchased                             as a gift for someone else?                              Answer \\'True\\' if yes,                             \\'False\\' if not or unknown. Answer should be within quotes\\n\\t\"delivery_days\": string  // How many days                                      did it take for the product                                      to arrive? If this                                       information is not found,                                      output -1.\\n\\t\"price_value\": string  // Extract any                                    sentences about the value or                                     price, and output them as a                                     comma separated Python list.\\n}\\n```\\n')]\n"
     ]
    }
   ],
   "source": [
    "print(messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "65597ae6-3530-49f7-8f58-d6730a30d083",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ```json\n",
      "{\n",
      "\t\"gift\": \"False\",\n",
      "\t\"delivery_days\": \"2\",\n",
      "\t\"price_value\": \"It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features.\"\n",
      "}\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "response = llm_nvidia(messages)\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "988a89e6-db88-4774-97bf-90d19a6b4b89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'gift': 'False', 'delivery_days': '2', 'price_value': \"It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features.\"}\n"
     ]
    }
   ],
   "source": [
    "output_dict = output_parser.parse(response.content)\n",
    "print(output_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "603c8905-8905-4892-b1ee-c455b66130f4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "```json\n",
      "{\n",
      "    \"gift\": \"True\",\n",
      "    \"delivery_days\": \"2\",\n",
      "    \"price_value\": [\"It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features.\"]\n",
      "}\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "response = llm_ollama(messages)\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "5b9046b2-ef19-4a6f-9dfc-454995d40481",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_dict = output_parser.parse(response.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "a4578409-53a1-4c87-8ca6-0bb002433f99",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features.\"]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_dict['price_value']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f2baeb8-8e35-4f02-9f37-d2435ebaa9d5",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Memory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3545935e-e321-454a-b683-303fb15efe41",
   "metadata": {},
   "source": [
    "- ConversationBufferMemory\n",
    "- ConversationBufferWindowMemory\n",
    "- ConversationTokenBufferMemory\n",
    "- ConversationSummaryMemory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0570a968-47b5-4894-b715-f5bab085d93b",
   "metadata": {},
   "source": [
    "![LangChain Memory](data/images/langchain-memory.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3163bfbe-46af-46c7-8bc9-71d6cabd97d1",
   "metadata": {},
   "source": [
    "### ConversationBufferMemory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "d90e4d4f-483f-4239-a7ad-000afa08ef3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import ConversationChain\n",
    "from langchain.memory import ConversationBufferMemory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "a2139570-6c95-4f2a-b93c-fbce81a51b12",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory = ConversationBufferMemory()\n",
    "conversation = ConversationChain(\n",
    "    llm = llm_nvidia,\n",
    "    memory = memory,\n",
    "    verbose = True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "437fa43a-613f-4f7b-913a-73245f4e0bd1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new ConversationChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
      "\n",
      "Current conversation:\n",
      "\n",
      "Human: Hi my name is Subash\n",
      "AI:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" Hello Subash, it's nice to meet you! I'm an AI and I'm here to help answer your questions and engage in conversation. How has your day been so far? If you have any questions or topics you'd like to discuss, feel free to let me know!\\n\\n(Note: As a helpful and talkative AI, I'll provide detailed responses and context where appropriate, and if I don't know an answer, I'll let you know honestly.)\""
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conversation.predict(input=\"Hi my name is Subash\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "3348dbd9-aba5-4ad0-942a-5016c4b95224",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new ConversationChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
      "\n",
      "Current conversation:\n",
      "Human: Hi my name is Subash\n",
      "AI:  Hello Subash, it's nice to meet you! I'm an AI and I'm here to help answer your questions and engage in conversation. How has your day been so far? If you have any questions or topics you'd like to discuss, feel free to let me know!\n",
      "\n",
      "(Note: As a helpful and talkative AI, I'll provide detailed responses and context where appropriate, and if I don't know an answer, I'll let you know honestly.)\n",
      "Human: What is 1+1?\n",
      "AI:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" The sum of 1 + 1 is 2. This is a basic arithmetic operation that falls under the field of mathematics. Is there anything else you would like to know about mathematics, or would you like to discuss a different topic? I'm here to help and engage in conversation, so feel free to ask me anything!\""
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conversation.predict(input=\"What is 1+1?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "bedfc83d-7865-4117-a1e0-02240bd91c95",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new ConversationChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
      "\n",
      "Current conversation:\n",
      "Human: Hi my name is Subash\n",
      "AI:  Hello Subash, it's nice to meet you! I'm an AI and I'm here to help answer your questions and engage in conversation. How has your day been so far? If you have any questions or topics you'd like to discuss, feel free to let me know!\n",
      "\n",
      "(Note: As a helpful and talkative AI, I'll provide detailed responses and context where appropriate, and if I don't know an answer, I'll let you know honestly.)\n",
      "Human: What is 1+1?\n",
      "AI:  The sum of 1 + 1 is 2. This is a basic arithmetic operation that falls under the field of mathematics. Is there anything else you would like to know about mathematics, or would you like to discuss a different topic? I'm here to help and engage in conversation, so feel free to ask me anything!\n",
      "Human: What is my name?\n",
      "AI:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' Your name, as you previously told me, is Subash. Is there anything you would like to know or discuss related to your name, or would you like to talk about something else? I strive to provide detailed and helpful responses, so please let me know how I can assist you further!'"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conversation.predict(input=\"What is my name?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "f50b28c3-ca4b-4389-871e-d82033fc7c46",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Hi my name is Subash\n",
      "AI:  Hello Subash, it's nice to meet you! I'm an AI and I'm here to help answer your questions and engage in conversation. How has your day been so far? If you have any questions or topics you'd like to discuss, feel free to let me know!\n",
      "\n",
      "(Note: As a helpful and talkative AI, I'll provide detailed responses and context where appropriate, and if I don't know an answer, I'll let you know honestly.)\n",
      "Human: What is 1+1?\n",
      "AI:  The sum of 1 + 1 is 2. This is a basic arithmetic operation that falls under the field of mathematics. Is there anything else you would like to know about mathematics, or would you like to discuss a different topic? I'm here to help and engage in conversation, so feel free to ask me anything!\n",
      "Human: What is my name?\n",
      "AI:  Your name, as you previously told me, is Subash. Is there anything you would like to know or discuss related to your name, or would you like to talk about something else? I strive to provide detailed and helpful responses, so please let me know how I can assist you further!\n"
     ]
    }
   ],
   "source": [
    "print(memory.buffer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "4fd947e2-4d0e-48db-b477-b470b3cbc247",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'history': \"Human: Hi my name is Subash\\nAI:  Hello Subash, it's nice to meet you! I'm an AI and I'm here to help answer your questions and engage in conversation. How has your day been so far? If you have any questions or topics you'd like to discuss, feel free to let me know!\\n\\n(Note: As a helpful and talkative AI, I'll provide detailed responses and context where appropriate, and if I don't know an answer, I'll let you know honestly.)\\nHuman: What is 1+1?\\nAI:  The sum of 1 + 1 is 2. This is a basic arithmetic operation that falls under the field of mathematics. Is there anything else you would like to know about mathematics, or would you like to discuss a different topic? I'm here to help and engage in conversation, so feel free to ask me anything!\\nHuman: What is my name?\\nAI:  Your name, as you previously told me, is Subash. Is there anything you would like to know or discuss related to your name, or would you like to talk about something else? I strive to provide detailed and helpful responses, so please let me know how I can assist you further!\"}"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memory.load_memory_variables({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "32c130cb-8bff-48ef-98d1-145b32a5d44b",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory = ConversationBufferMemory()\n",
    "memory.save_context({\"input\": \"Hi\"}, {\"output\": \"What's up\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "e17ee186-a583-4ef5-9c95-2b9d21e597cc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Hi\n",
      "AI: What's up\n"
     ]
    }
   ],
   "source": [
    "print(memory.buffer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "81be2c5e-bed2-4e50-8e0c-6463b0a77412",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'history': \"Human: Hi\\nAI: What's up\"}"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memory.load_memory_variables({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "2624b71a-1382-421c-a3fb-498c372ab667",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory.save_context({\"input\": \"Not much, just hanging\"}, \n",
    "                    {\"output\": \"Cool\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "2ee233cb-4322-483b-b102-c9c34c122f5d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'history': \"Human: Hi\\nAI: What's up\\nHuman: Not much, just hanging\\nAI: Cool\"}"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memory.load_memory_variables({})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4f1d77f-df31-4da8-96a2-aaf25b2e25bc",
   "metadata": {},
   "source": [
    "### ConversationBufferWindowMemory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "e40b030a-873e-41f4-8157-60f76678bb92",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.memory import ConversationBufferWindowMemory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "9e06b4af-0042-4d65-889d-3559536ba3a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory = ConversationBufferWindowMemory(k=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "e29b5b75-f0c2-47ea-a7d3-e1472b446d84",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory.save_context({\"input\":\"Hi\"},{\"output\":\"What's up\"})\n",
    "memory.save_context({\"input\":\"Not much, just hanging\"}, {\"output\":\"Cool\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "c83f0861-119b-4697-b56e-cc563531dec8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'history': 'Human: Not much, just hanging\\nAI: Cool'}"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memory.load_memory_variables({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "4e1c495d-500d-4100-b724-8f8ac69e5c8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory = None"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1449aa51-8322-4005-9fed-e7147c866610",
   "metadata": {},
   "source": [
    "It only remembers only latest conversation because k = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "f44c78ab-5987-40fe-9140-9484683a07ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory = ConversationBufferWindowMemory(k=1)\n",
    "\n",
    "conversation = ConversationChain(\n",
    "    llm = llm_ollama,\n",
    "    memory = memory,\n",
    "    verbose = True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "598fbf18-00c8-4882-b027-faf308ac59fe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new ConversationChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
      "\n",
      "Current conversation:\n",
      "\n",
      "Human: Hi, I'm Subash\n",
      "AI:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Nice to meet you, Subash! I'm LLaMA, an AI designed by Meta AI that can converse in a human-like way. I've been trained on a massive dataset of text from the internet and can provide information on a wide range of topics. Currently, my knowledge cutoff is up until 2022, so if there's anything more recent than that, I might not be aware of it. But don't worry, I'll do my best to help answer your questions or engage in a fun conversation with you!\""
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conversation.predict(input=\"Hi, I'm Subash\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "d4d9786f-4436-47e4-8e25-5090ce9582ef",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new ConversationChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
      "\n",
      "Current conversation:\n",
      "Human: Hi, I'm Subash\n",
      "AI: Nice to meet you, Subash! I'm LLaMA, an AI designed by Meta AI that can converse in a human-like way. I've been trained on a massive dataset of text from the internet and can provide information on a wide range of topics. Currently, my knowledge cutoff is up until 2022, so if there's anything more recent than that, I might not be aware of it. But don't worry, I'll do my best to help answer your questions or engage in a fun conversation with you!\n",
      "Human: What is 1+1?\n",
      "AI:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Easy one, Subash! The answer to 1+1 is... (drumroll please)... 2! Yes, the result of adding one and one together is indeed two. This fundamental arithmetic operation is a building block for more complex math concepts, and it's a basic fact that I've been trained on through my vast dataset of text from the internet.\""
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conversation.predict(input=\"What is 1+1?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "84038d86-6835-46f1-917b-fcdad4b99c73",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new ConversationChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
      "\n",
      "Current conversation:\n",
      "Human: What is 1+1?\n",
      "AI: Easy one, Subash! The answer to 1+1 is... (drumroll please)... 2! Yes, the result of adding one and one together is indeed two. This fundamental arithmetic operation is a building block for more complex math concepts, and it's a basic fact that I've been trained on through my vast dataset of text from the internet.\n",
      "Human: Who am I?\n",
      "AI:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"A question about identity! Well, Subash, as far as our conversation goes, you are... (drumroll please)... the human who is chatting with me, an AI designed to simulate human-like conversations! You're a curious individual who's asking thoughtful questions and engaging in a friendly chat with me. I don't have any specific information about your real-life identity or personal details, but based on our conversation, you seem like a bright and inquisitive person!\""
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conversation.predict(input=\"Who am I?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c6765cd-0b30-4cd1-a62d-32115ed98f79",
   "metadata": {},
   "source": [
    "### ConversationTokenBufferMemory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01d82dc0-7d69-4d75-943a-8c4b221fb8ac",
   "metadata": {},
   "source": [
    "Memory will limit the number of tokens. Since, a lot of LLM pricing is based on tokens, this maps more directly to the cost of LLM calls.\n",
    "\n",
    "Different llms use different ways of counting tokens. Passing llm tells the ConversationTokenBufferMemory to use the way of counting token that the llm passed uses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "a36166ff-b25e-4225-8e48-c39502fda1bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.memory import ConversationTokenBufferMemory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "4156b25b-bb3a-4cf1-84ef-4a671868f101",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "memory = ConversationTokenBufferMemory(llm=llm_nvidia, max_token_limit=25)\n",
    "memory.save_context({\"input\": \"AI is what?!\"},\n",
    "                    {\"output\": \"Amazing!\"})\n",
    "memory.save_context({\"input\": \"Backpropagation is what?\"},\n",
    "                    {\"output\": \"Beautiful!\"})\n",
    "memory.save_context({\"input\": \"Chatbots are what?\"}, \n",
    "                    {\"output\": \"Charming!\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "ccea3b35-9478-4d2b-a224-1931f5a477fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'history': 'Human: Backpropagation is what?\\nAI: Beautiful!\\nHuman: Chatbots are what?\\nAI: Charming!'}"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memory.load_memory_variables({})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4292f719-8744-45b0-8c8d-3273b8b914f5",
   "metadata": {},
   "source": [
    "### ConversationSummaryBufferMemory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "8806fcf4-c3c0-4bcc-9859-72443e41e534",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.memory import ConversationSummaryBufferMemory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "6a0292bf-f44a-4b8d-84cc-a8aebfe43449",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a long string\n",
    "schedule = \"There is a meeting at 8am with your product team. \\\n",
    "You will need your powerpoint presentation prepared. \\\n",
    "9am-12pm have time to work on your LangChain \\\n",
    "project which will go quickly because Langchain is such a powerful tool. \\\n",
    "At Noon, lunch at the italian resturant with a customer who is driving \\\n",
    "from over an hour away to meet you to understand the latest in AI. \\\n",
    "Be sure to bring your laptop to show the latest LLM demo.\"\n",
    "\n",
    "memory = ConversationSummaryBufferMemory(llm=llm_nvidia, max_token_limit=100)\n",
    "memory.save_context({\"input\": \"Hello\"}, {\"output\": \"What's up\"})\n",
    "memory.save_context({\"input\": \"Not much, just hanging\"},\n",
    "                    {\"output\": \"Cool\"})\n",
    "memory.save_context({\"input\": \"What is on the schedule today?\"}, \n",
    "                    {\"output\": f\"{schedule}\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "b5583a9d-25c0-40cb-9259-7372adbbd693",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'history': 'System:  The human and AI exchange greetings, with the human mentioning they are just hanging out. The human then asks about the schedule for the day.\\nAI: There is a meeting at 8am with your product team. You will need your powerpoint presentation prepared. 9am-12pm have time to work on your LangChain project which will go quickly because Langchain is such a powerful tool. At Noon, lunch at the italian resturant with a customer who is driving from over an hour away to meet you to understand the latest in AI. Be sure to bring your laptop to show the latest LLM demo.'}\n"
     ]
    }
   ],
   "source": [
    "print(memory.load_memory_variables({}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "9134ce1f-2088-4ce5-8a8a-8053242b41a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "conversation = ConversationChain(\n",
    "    llm = llm_nvidia,\n",
    "    memory = memory,\n",
    "    verbose = True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "3ba726de-84a4-4403-86c1-bb92501a2fd1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new ConversationChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
      "\n",
      "Current conversation:\n",
      "System:  The human and AI exchange greetings, with the human mentioning they are just hanging out. The human then asks about the schedule for the day.\n",
      "AI: There is a meeting at 8am with your product team. You will need your powerpoint presentation prepared. 9am-12pm have time to work on your LangChain project which will go quickly because Langchain is such a powerful tool. At Noon, lunch at the italian resturant with a customer who is driving from over an hour away to meet you to understand the latest in AI. Be sure to bring your laptop to show the latest LLM demo.\n",
      "Human: What would be a good demo to show?\n",
      "AI:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" I would recommend showing the latest Language Model (LLM) demo that we have been working on. It incorporates the latest advancements in natural language processing and can effectively demonstrate the capabilities of our AI technology. This demo involves real-time language translation, text summarization, and sentiment analysis. The customer you're meeting with will be particularly interested in the translation feature, as they are coming from a different region. The demo will help illustrate how our AI can assist in overcoming language barriers and facilitate smoother communication.\""
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conversation.predict(input=\"What would be a good demo to show?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "437accaa-02ba-44e0-b63e-6929848cc97d",
   "metadata": {},
   "source": [
    "![Memory Types](data/images/memory-types.png)\n",
    "\n",
    "![Additional Memory Types](data/images/additional-memory-types.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a59ed1a3-0e13-429c-930a-6d3387a96ffe",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Chains"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff8a6215-007b-4306-9ff0-db734f3d2639",
   "metadata": {},
   "source": [
    "The chain usually combines a LLM together with a prompt and put a bunch of building blocks together to carry out a sequence of operations on your text/data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "1dc9429a-5748-4949-9933-18b9e374be1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "product_review = {\n",
    "    \"product\": [\"Queen Size Sheet Set\",\n",
    "                \"Waterproof Phone Pouch\",\n",
    "                \"Luxury Air Mattress\",\n",
    "                \"Pillows Insert\",\n",
    "                \"Milk Frother Handheld\",\n",
    "                \"L'Or Espresso Café\",\n",
    "                \"Hervidor de Agua Eléctrico\",\n",
    "               ],\n",
    "    \"review\":[\n",
    "        \"I ordered a king size set. My only criticism would be that I wish seller would offer the king size set with 4 pillowcases. I separately ordered a two pack of pillowcases so I could have a total of four. When I saw the two packages, it looked like the color did not exactly match. Customer service was excellent about sending me two more pillowcases so I would have four that matched. Excellent! For the cost of these sheets, I am satisfied with the characteristics and coolness of the sheets.\",\n",
    "        \"I loved the waterproof sac, although the opening was made of a hard plastic. I don’t know if that would break easily. But I couldn’t turn my phone on, once it was in the pouch.\",\n",
    "        \"This mattress had a small hole in the top of it (took forever to find where it was), and the patches that they provide did not work, maybe because it's the top of the mattress where it's kind of like fabric and a patch won't stick. Maybe I got unlucky with a defective mattress, but where's quality assurance for this company? That flat out should not happen. Emphasis on flat. Cause that's what the mattress was. Seriously horrible experience, ruined my friend's stay with me. Then they make you ship it back instead of just providing a refund, which is also super annoying to pack up an air mattress and take it to the UPS store. This company is the worst, and this mattress is the worst.\",\n",
    "        \"This is the best throw pillow fillers on Amazon. I’ve tried several others, and they’re all cheap and flat no matter how much fluffing you do. Once you toss these in the dryer after you remove them from the vacuum sealed shipping material, they fluff up great\",\n",
    "        \"I loved this product. But they only seem to last a few months. The company was great replacing the first one (the frother falls out of the handle and can't be fixed). The after 4 months my second one did the same. I only use the frother for coffee once a day. It's not overuse or abuse. I'm very disappointed and will look for another. As I understand they will only replace once. Anyway, if you have one good luck.\",\n",
    "        \"Je trouve le goût médiocre. La mousse ne tient pas, c'est bizarre. J'achète les mêmes dans le commerce et le goût est bien meilleur... Vieux lot ou contrefaçon !?\",\n",
    "        \"Está lu bonita calienta muy rápido, es muy funcional, solo falta ver cuánto dura, solo llevo 3 días en funcionamiento.\"        \n",
    "    ]\n",
    "}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "b2375cab-2b8c-4b50-8d4e-f2a01e69e975",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(product_review)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "d90dac4b-70c3-42a9-afa1-1357405e3a95",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>product</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Queen Size Sheet Set</td>\n",
       "      <td>I ordered a king size set. My only criticism w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Waterproof Phone Pouch</td>\n",
       "      <td>I loved the waterproof sac, although the openi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Luxury Air Mattress</td>\n",
       "      <td>This mattress had a small hole in the top of i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Pillows Insert</td>\n",
       "      <td>This is the best throw pillow fillers on Amazo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Milk Frother Handheld</td>\n",
       "      <td>I loved this product. But they only seem to la...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  product                                             review\n",
       "0    Queen Size Sheet Set  I ordered a king size set. My only criticism w...\n",
       "1  Waterproof Phone Pouch  I loved the waterproof sac, although the openi...\n",
       "2     Luxury Air Mattress  This mattress had a small hole in the top of i...\n",
       "3          Pillows Insert  This is the best throw pillow fillers on Amazo...\n",
       "4   Milk Frother Handheld  I loved this product. But they only seem to la..."
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e498958c-a5f5-4f96-9ad9-efac8026bdd2",
   "metadata": {},
   "source": [
    "## LLMChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "384b8f71-ac6e-4de6-9409-e5000678a9cf",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.prompts import ChatPromptTemplate\n",
    "from langchain.chains import LLMChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "6aa323ef-2bf6-4854-8460-fa1f48757d91",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = ChatPromptTemplate.from_template(\n",
    "    \"What is the best name to describe \\\n",
    "    a company that makes {product}? Give me only one name\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "3741d8c4-4103-4e1a-8fb8-dc22a7657810",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The class `LLMChain` was deprecated in LangChain 0.1.17 and will be removed in 0.3.0. Use RunnableSequence, e.g., `prompt | llm` instead.\n",
      "  warn_deprecated(\n"
     ]
    }
   ],
   "source": [
    "chain = LLMChain(llm=llm_nvidia, prompt=prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "f4b9e560-7f53-4b1e-be6d-fa635145f5d4",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `Chain.run` was deprecated in langchain 0.1.0 and will be removed in 0.3.0. Use invoke instead.\n",
      "  warn_deprecated(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' \"Monarch Regalia Linens\" could be a fitting name for a company that makes queen size sheet sets, as it evokes a sense of luxury and grandeur, fitting for a queen-sized bed. The name also includes \"linens\" to clearly communicate the nature of the company\\'s products.'"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "product = \"Queen Size Sheet Set\"\n",
    "chain.run(product)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "154b825f-d009-4f38-a18f-e176ea1946c3",
   "metadata": {},
   "source": [
    "## Sequential Chains"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de42fe2a-117b-4a80-9c46-1ce72622a653",
   "metadata": {},
   "source": [
    "![Sequential Chain](data/images/sequential-chain.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0890a49a-7019-4955-953c-60a8c9a92c10",
   "metadata": {},
   "source": [
    "### Simple Sequential Chain"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "deb8edbc-33a7-4bcb-90ab-3b88f58ef3c7",
   "metadata": {},
   "source": [
    "![Simple Sequential Chain](data/images/simple-sequential-chain.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "06b4a247-f211-4652-84b6-03dc09f66ecd",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import SimpleSequentialChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "fc47caf6-4f6b-4f2f-a1e0-db315348cbb9",
   "metadata": {},
   "outputs": [],
   "source": [
    "first_prompt = ChatPromptTemplate.from_template(\n",
    "    \"Give me only the best name to describe the company that makes {product}.\"\n",
    ")\n",
    "\n",
    "chain_one = LLMChain(llm=llm_nvidia, prompt=first_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "d787b749-8664-4509-80b7-a65da5b95230",
   "metadata": {},
   "outputs": [],
   "source": [
    "second_prompt = ChatPromptTemplate.from_template(\n",
    "    \"Write a 20 words description for the following company {company}\"\n",
    ")\n",
    "\n",
    "chain_two = LLMChain(llm=llm_nvidia, prompt=second_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "68341c8f-6530-41ca-9890-066b321615c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "overall_simple_chain = SimpleSequentialChain(\n",
    "    chains=[chain_one, chain_two], \n",
    "    verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "63767569-d44f-426d-a42f-b65acc017591",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new SimpleSequentialChain chain...\u001b[0m\n",
      "\u001b[36;1m\u001b[1;3m \"Royal Comfort: The Premier Provider of Queen Size Sheets\"\n",
      "\n",
      "This name effectively communicates the company's focus on creating high-quality queen size sheets, while also conveying a sense of luxury and elegance. The use of \"Royal\" and \"Premier\" highlights the brand's commitment to providing top-notch products, while \"Comfort\" emphasizes the softness and coziness of their sheets. Overall, this name is memorable, evocative, and clearly communicates the company's mission and values.\u001b[0m\n",
      "\u001b[33;1m\u001b[1;3m \"Royal Comfort: Luxurious Queen-Size Sheets for Ultimate Comfort.\"\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' \"Royal Comfort: Luxurious Queen-Size Sheets for Ultimate Comfort.\"'"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "product = \"Queen Size Sheet\"\n",
    "overall_simple_chain.run(product)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da8a9f59-c855-450b-80a2-930be3e86920",
   "metadata": {},
   "source": [
    "Simple Sequential Chain works well when there is a single input and single output."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6f52b1d-2ecc-4574-be34-e61e00ccc623",
   "metadata": {},
   "source": [
    "### SequentialChain"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1a9e30c-f4c9-4c32-9f0b-d3f97b72cf4d",
   "metadata": {},
   "source": [
    "![Sequential Chain](data/images/sequential-chain-figure.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "08e41544-4a42-4639-845c-f389d733ac89",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import SequentialChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "e95ee1b7-5069-404c-b96b-df4d092fa782",
   "metadata": {},
   "outputs": [],
   "source": [
    "first_prompt = ChatPromptTemplate.from_template(\n",
    "    \"Translate the following review to english: \"\n",
    "    \"{Review}\"\n",
    ")\n",
    "# chain 1: input= Review and output= English_Review\n",
    "chain_one = LLMChain(llm=llm_nvidia, prompt=first_prompt, output_key=\"English_Review\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "fdf2d18c-0e4c-4b48-9c00-a1b288d779e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "second_prompt = ChatPromptTemplate.from_template(\n",
    "    \"Can you summarize the following review in 1 sentence:\"\n",
    "    \"{English_Review}\"\n",
    ")\n",
    "# chain 2: input= English_Review and output= summary\n",
    "chain_two = LLMChain(llm=llm_nvidia, prompt=second_prompt, \n",
    "                     output_key=\"summary\"\n",
    "                    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "ae43b992-b18c-4057-94b3-b638e060eff3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# prompt template 3: translate to english\n",
    "third_prompt = ChatPromptTemplate.from_template(\n",
    "    \"What language is the following review:\"\n",
    "    \"{Review}\"\n",
    ")\n",
    "# chain 3: input= Review and output= language\n",
    "chain_three = LLMChain(llm=llm_nvidia, \n",
    "                       prompt=third_prompt,\n",
    "                       output_key=\"language\"\n",
    "                      )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "b0f740cd-6a04-4c5f-9918-8e36c19ddf44",
   "metadata": {},
   "outputs": [],
   "source": [
    "# prompt template 4: follow up message\n",
    "fourth_prompt = ChatPromptTemplate.from_template(\n",
    "    \"Write a follow up response to the following \"\n",
    "    \"summary in the specified language:\"\n",
    "    \"Summary: {summary}\"\n",
    "    \"Language: {language}\"\n",
    ")\n",
    "# chain 4: input= summary, language and output= followup_message\n",
    "chain_four = LLMChain(llm=llm_nvidia, prompt=fourth_prompt,\n",
    "                      output_key=\"followup_message\"\n",
    "                     )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "41346797-9e9a-4479-bda9-e557dca0b630",
   "metadata": {},
   "outputs": [],
   "source": [
    "# overall_chain: input= Review \n",
    "# and output= English_Review,summary, followup_message\n",
    "overall_chain = SequentialChain(\n",
    "    chains=[chain_one, chain_two, chain_three, chain_four],\n",
    "    input_variables=[\"Review\"],\n",
    "    output_variables=[\"English_Review\", \"summary\",\"followup_message\"],\n",
    "    verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "60406647-10b2-4d95-9bb3-a2f5442245d0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new SequentialChain chain...\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `Chain.__call__` was deprecated in langchain 0.1.0 and will be removed in 0.3.0. Use invoke instead.\n",
      "  warn_deprecated(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'Review': \"Je trouve le goût médiocre. La mousse ne tient pas, c'est bizarre. J'achète les mêmes dans le commerce et le goût est bien meilleur... Vieux lot ou contrefaçon !?\",\n",
       " 'English_Review': \" I find the taste mediocre. The foam does not hold, it's strange. I buy the same ones in the store and the taste is much better... Old lot or counterfeit!?\\n\\nNote: The review is expressing dissatisfaction with the taste and consistency of a product, and raises the possibility that it may be an old batch or a counterfeit version.\",\n",
       " 'summary': ' The reviewer finds the taste of the product mediocre and its foam inconsistent, suspecting it may be an old or counterfeit batch, and compares it unfavorably to the same product purchased in stores.',\n",
       " 'followup_message': \" Cher rédacteur de la revue,\\n\\nJe suis désolé d'entendre que votre expérience avec notre produit n'a pas été à la hauteur de vos attentes. Nous prenons vos commentaires au sérieux et allons enquêter sur la possibilité d'un lot ancien ou contrefait.\\n\\nLa qualité et la fraîcheur de nos produits sont une priorité pour nous, et nous sommes déterminés à offrir une expérience de dégustation cohérente et agréable à tous nos clients. Nous vous invitons à contacter notre service clientèle pour obtenir de l'aide et discuter de votre expérience.\\n\\nCordialement,\\n[Votre nom]\"}"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review = df.review[5]\n",
    "overall_chain(review)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52b41243-b236-41c9-85a2-05b48e05a0ec",
   "metadata": {},
   "source": [
    "## Router Chain"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8774eb3d-9b0b-46aa-a660-8af0f19e0677",
   "metadata": {},
   "source": [
    "![Router Chain](data/images/router-chain.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "705ee83a-8049-4f2f-b938-79fcd91cf328",
   "metadata": {},
   "outputs": [],
   "source": [
    "physics_template = \"\"\"You are a very smart physics professor. \\\n",
    "You are great at answering questions about physics in a concise\\\n",
    "and easy to understand manner. \\\n",
    "When you don't know the answer to a question you admit\\\n",
    "that you don't know.\n",
    "\n",
    "Here is a question:\n",
    "{input}\"\"\"\n",
    "\n",
    "\n",
    "math_template = \"\"\"You are a very good mathematician. \\\n",
    "You are great at answering math questions. \\\n",
    "You are so good because you are able to break down \\\n",
    "hard problems into their component parts, \n",
    "answer the component parts, and then put them together\\\n",
    "to answer the broader question.\n",
    "\n",
    "Here is a question:\n",
    "{input}\"\"\"\n",
    "\n",
    "history_template = \"\"\"You are a very good historian. \\\n",
    "You have an excellent knowledge of and understanding of people,\\\n",
    "events and contexts from a range of historical periods. \\\n",
    "You have the ability to think, reflect, debate, discuss and \\\n",
    "evaluate the past. You have a respect for historical evidence\\\n",
    "and the ability to make use of it to support your explanations \\\n",
    "and judgements.\n",
    "\n",
    "Here is a question:\n",
    "{input}\"\"\"\n",
    "\n",
    "\n",
    "computerscience_template = \"\"\" You are a successful computer scientist.\\\n",
    "You have a passion for creativity, collaboration,\\\n",
    "forward-thinking, confidence, strong problem-solving capabilities,\\\n",
    "understanding of theories and algorithms, and excellent communication \\\n",
    "skills. You are great at answering coding questions. \\\n",
    "You are so good because you know how to solve a problem by \\\n",
    "describing the solution in imperative steps \\\n",
    "that a machine can easily interpret and you know how to \\\n",
    "choose a solution that has a good balance between \\\n",
    "time complexity and space complexity. \n",
    "\n",
    "Here is a question:\n",
    "{input}\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "18582d59-2e28-42ae-9381-e33ac798452d",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_infos = [\n",
    "    {\n",
    "        \"name\": \"physics\", \n",
    "        \"description\": \"Good for answering questions about physics\", \n",
    "        \"prompt_template\": physics_template\n",
    "    },\n",
    "    {\n",
    "        \"name\": \"math\", \n",
    "        \"description\": \"Good for answering math questions\", \n",
    "        \"prompt_template\": math_template\n",
    "    },\n",
    "    {\n",
    "        \"name\": \"History\", \n",
    "        \"description\": \"Good for answering history questions\", \n",
    "        \"prompt_template\": history_template\n",
    "    },\n",
    "    {\n",
    "        \"name\": \"computer science\", \n",
    "        \"description\": \"Good for answering computer science questions\", \n",
    "        \"prompt_template\": computerscience_template\n",
    "    }\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "c89d2590-1f29-4398-afa5-23063d16c2da",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains.router import MultiPromptChain\n",
    "from langchain.chains.router.llm_router import LLMRouterChain,RouterOutputParser\n",
    "from langchain.prompts import PromptTemplate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "6ca9a4d5-26f9-42dc-8ca4-87fc9c0c1675",
   "metadata": {},
   "outputs": [],
   "source": [
    "destination_chains = {}\n",
    "for p_info in prompt_infos:\n",
    "    name = p_info[\"name\"]\n",
    "    prompt_template = p_info[\"prompt_template\"]\n",
    "    prompt = ChatPromptTemplate.from_template(template=prompt_template)\n",
    "    chain = LLMChain(llm=llm_nvidia, prompt=prompt)\n",
    "    destination_chains[name] = chain  \n",
    "    \n",
    "destinations = [f\"{p['name']}: {p['description']}\" for p in prompt_infos]\n",
    "destinations_str = \"\\n\".join(destinations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "id": "e6901b94-e6fa-4d70-bc75-92bcfa0a0978",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "physics: Good for answering questions about physics\n",
      "math: Good for answering math questions\n",
      "History: Good for answering history questions\n",
      "computer science: Good for answering computer science questions\n"
     ]
    }
   ],
   "source": [
    "print(destinations_str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "3daf0bbe-e394-4d71-82b9-cc8f337c4b50",
   "metadata": {},
   "outputs": [],
   "source": [
    "default_prompt = ChatPromptTemplate.from_template(\"{input}\")\n",
    "default_chain = LLMChain(llm=llm_nvidia, prompt=default_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "d09cae80-c437-489f-a596-5c696fda03b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "MULTI_PROMPT_ROUTER_TEMPLATE = \"\"\"Given a raw text input to a \n",
    "language model select the model prompt best suited for the input.\n",
    "You will be given the names of the available prompts and a \n",
    "description of what the prompt is best suited for. \n",
    "You may also revise the original input if you think that revising\n",
    "it will ultimately lead to a better response from the language model.\n",
    "\n",
    "<< FORMATTING >>\n",
    "Return a markdown code snippet with a JSON object formatted to look like:\n",
    "```json\n",
    "{{{{\n",
    "    \"destination\": name of the prompt to use or \"DEFAULT\"\n",
    "    \"next_inputs\": string a potentially modified version of the original input\n",
    "}}}}\n",
    "```\n",
    "\n",
    "REMEMBER: \"destination\" MUST be one of the candidate prompt \\\n",
    "names specified below OR it can be \"DEFAULT\" if the input is not\\\n",
    "well suited for any of the candidate prompts.\n",
    "REMEMBER: \"next_inputs\" can just be the original input if you don't think any modifications are needed.\n",
    "\n",
    "<< CANDIDATE PROMPTS >>\n",
    "{destinations}\n",
    "\n",
    "<< INPUT >>\n",
    "{{input}}\n",
    "\n",
    "<< OUTPUT (remember to include the ```json)>>\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "cefa9e0f-d2fd-4274-bb42-7176a29ebf5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(\n",
    "    destinations=destinations_str\n",
    ")\n",
    "router_prompt = PromptTemplate(\n",
    "    template=router_template,\n",
    "    input_variables=[\"input\"],\n",
    "    output_parser=RouterOutputParser(),\n",
    ")\n",
    "\n",
    "router_chain = LLMRouterChain.from_llm(llm_nvidia, router_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "c97b01b3-977f-4ed0-86a2-9aff89d2ecd6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Given a raw text input to a \n",
      "language model select the model prompt best suited for the input.\n",
      "You will be given the names of the available prompts and a \n",
      "description of what the prompt is best suited for. \n",
      "You may also revise the original input if you think that revising\n",
      "it will ultimately lead to a better response from the language model.\n",
      "\n",
      "<< FORMATTING >>\n",
      "Return a markdown code snippet with a JSON object formatted to look like:\n",
      "```json\n",
      "{\n",
      "    \"destination\": name of the prompt to use or \"DEFAULT\"\n",
      "    \"next_inputs\": string a potentially modified version of the original input\n",
      "}\n",
      "```\n",
      "\n",
      "REMEMBER: \"destination\" MUST be one of the candidate prompt names specified below OR it can be \"DEFAULT\" if the input is notwell suited for any of the candidate prompts.\n",
      "REMEMBER: \"next_inputs\" can just be the original input if you don't think any modifications are needed.\n",
      "\n",
      "<< CANDIDATE PROMPTS >>\n",
      "physics: Good for answering questions about physics\n",
      "math: Good for answering math questions\n",
      "History: Good for answering history questions\n",
      "computer science: Good for answering computer science questions\n",
      "\n",
      "<< INPUT >>\n",
      "What is 1+1?\n",
      "\n",
      "<< OUTPUT (remember to include the ```json)>>\n"
     ]
    }
   ],
   "source": [
    "print(router_template.format(input=\"What is 1+1?\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "1306b5d9-ff38-4959-a8e3-2ac07f30b6d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = MultiPromptChain(router_chain=router_chain, \n",
    "                         destination_chains=destination_chains, \n",
    "                         default_chain=default_chain, verbose=True\n",
    "                        )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "8da14062-6306-44a1-9253-17129d1c3950",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new MultiPromptChain chain...\u001b[0m\n",
      "physics: {'input': 'What is black body radiation?'}\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" Black body radiation is the electromagnetic radiation emitted by a perfect absorber, also known as a black body, at a given temperature. It is characterized by its continuous spectrum and the fact that the amount of radiation emitted is only determined by the temperature of the black body, not by its shape, size, or composition. The peak wavelength of the radiation shifts to shorter wavelengths as the temperature increases, following the Planck's law of black body radiation. This phenomenon is important in understanding the behavior of radiation in various physical systems, such as stars and heated objects.\""
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.run(\"What is black body radiation?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "861dd672-d04e-4a88-adf2-eb0a3381db09",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new MultiPromptChain chain...\u001b[0m\n",
      "math: {'input': 'what is 2 + 2'}\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' Thank you for your kind words! I\\'m here to help answer your math question to the best of my ability.\\n\\nThe question you have asked is: what is 2 + 2.\\n\\nTo answer this question, I will break it down into its component parts:\\n\\n* The number 2\\n* The addition operation\\n* The number 2\\n\\nWhen we perform the addition operation on the two numbers, we get:\\n\\n2 + 2 = 4\\n\\nTherefore, the answer to the question \"what is 2 + 2\" is 4. I hope this helps! Do you have any other math questions that I can assist you with?'"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.run(\"what is 2 + 2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "bc11d6e6-e2e6-44d0-988a-5ab85b750427",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new MultiPromptChain chain...\u001b[0m\n",
      "History: {'input': 'Who was Alexandar Hamilton?'}\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" Alexander Hamilton (1755-1804) was a founding father of the United States, who served as the first Secretary of the Treasury from 1789 to 1795. He was born on the island of Nevis in the British West Indies and was orphaned at a young age. Through hard work and determination, he was able to attend King's College (now Columbia University) in New York City.\\n\\nDuring the American Revolution, Hamilton served as an artillery captain in the Continental Army and became a close aide to General George Washington. After the war, he played a key role in the drafting and ratification of the U.S. Constitution, and was one of the authors of the Federalist Papers, a series of essays arguing for the adoption of the Constitution.\\n\\nAs Secretary of the Treasury, Hamilton implemented a number of policies that helped to establish the credit of the new nation, including the creation of a national bank and the assumption of state debts. He also advocated for a strong central government and a robust federal judiciary.\\n\\nHamilton's political views often brought him into conflict with other founding fathers, most notably Thomas Jefferson. Their disagreements over issues such as the role of the federal government and the nature of democracy helped to shape the early political landscape of the United States.\\n\\nHamilton was killed in a duel with Aaron Burr in 1804, at the age of 49. His legacy as a statesman, writer, and financial genius continues to be celebrated and debated by historians and scholars today.\""
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.run(\"Who was Alexandar Hamilton?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f657c09-a642-4128-b6ff-514819525f10",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Question and Answer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2285c98-e202-4bb1-a01d-bf76f6902e56",
   "metadata": {},
   "source": [
    "![LLMS on Documents](data/images/llms-on-documents.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78476345-cce8-44d5-a15f-c3a095f5c315",
   "metadata": {},
   "source": [
    "![Embeddings](data/images/embeddings.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e7df403-bbdd-4af5-987a-d36e87789792",
   "metadata": {},
   "source": [
    "![Vector Database](data/images/vector-database.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca3d3dd4-02da-40a9-8cf3-27ac696cc98f",
   "metadata": {},
   "source": [
    "![Vector Database to LLM](data/images/vector-database-to-llm.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "0e1114b5-263a-4443-8a6b-0c99d13bb981",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.document_loaders import CSVLoader\n",
    "from langchain.vectorstores import DocArrayInMemorySearch\n",
    "from IPython.display import display, Markdown"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cafd72ae-92f9-46fb-b35e-2a201fac3b92",
   "metadata": {},
   "source": [
    "### VectorstoreIndexCreator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "4525c3fb-d95c-406c-b894-58a9d4cde5c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "file = 'data/files/OutdoorClothingCatalog_1000.csv'\n",
    "loader = CSVLoader(file_path=file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "id": "3ab1ca57-4bf7-433d-bf94-f5744f150efd",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.indexes import VectorstoreIndexCreator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "8d4d9fbe-32da-42b6-91d9-4ecefb566188",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from langchain_huggingface import HuggingFaceEmbeddings\n",
    "\n",
    "embedding = HuggingFaceEmbeddings(model_name=\"BAAI/bge-small-en-v1.5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "id": "0fda755a-9e78-4119-a421-3f40eeaea4ed",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/pydantic/_migration.py:283: UserWarning: `pydantic.error_wrappers:ValidationError` has been moved to `pydantic:ValidationError`.\n",
      "  warnings.warn(f'`{import_path}` has been moved to `{new_location}`.')\n"
     ]
    }
   ],
   "source": [
    "index = VectorstoreIndexCreator(\n",
    "    vectorstore_cls=DocArrayInMemorySearch,\n",
    "    embedding= embedding\n",
    ").from_loaders([loader])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "id": "d93b3092-406a-4b5c-886e-db8d376dbc37",
   "metadata": {},
   "outputs": [],
   "source": [
    "query =\"Please list all your shirts with sun protection \\\n",
    "in a table in markdown and summarize each one.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "id": "1b20ed61-ae20-463d-8b0e-e903a52f2064",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = index.query(query, \n",
    "                       llm = llm_nvidia)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "8a069a1b-af21-46d1-99e0-695110f13fa7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       " | Product ID | Name | Description | Fabric & Care | Sun Protection |\n",
       "| --- | --- | --- | --- | --- |\n",
       "| 679 | Women's Tropical Tee, Sleeveless | Five-star sleeveless button-up shirt with SunSmart™ protection, UPF 50+, wrinkle-resistant, machine washable | Shell: 71% nylon, 29% polyester. Cape lining: 100% polyester | High-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays |\n",
       "| 255 | Sun Shield Shirt by | High-performance sun shirt with UPF 50+, wicks moisture, quick-drying, abrasion-resistant, handwash and line dry | 78% nylon, 22% Lycra Xtra Life fiber | High-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays, recommended by The Skin Cancer Foundation as an effective UV protectant |\n",
       "| 618 | Men's Tropical Plaid Short-Sleeve Shirt | Lightweight, UPF 50+ hot-weather shirt, wrinkle-resistant, front and back cape venting, two front bellows pockets, imported | 100% polyester | High-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays |\n",
       "| 709 | Sunrise Tee | Women's UV-protective button-down shirt, lightweight, wicks moisture, quick-drying, wrinkle-free, UPF 50+ | Lightweight performance synthetic wicks moisture, resists wrinkles and dries fast. Shell: 71% nylon, 29% polyester. Cape lining: 100% polyester | High-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays |\n",
       "\n",
       "All the shirts listed provide sun protection with a rating of UPF 50+, blocking 98% of the sun's harmful rays. The first product is a sleeveless button-up shirt for women made of nylon and polyester, with SunSmart™ protection, wrinkle-resistant, and machine washable. The second product is a sun shirt for men made of nylon and Lycra Xtra Life fiber, with moisture-wicking, quick-drying, and abrasion-resistant features, handwash and line dry. The third product is a short-sleeve shirt for men made of 100% polyester, with front and back cape venting, two front bellows pockets, and imported. The last product is a women's UV-protective button-down shirt made of lightweight performance synthetic, wicks moisture, quick-drying, and wrinkle-free."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67306163-384c-4c8e-b5b6-25aa1cb5bd04",
   "metadata": {},
   "source": [
    "### Step by Step"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd861391-61b2-490a-954b-1cfc59da9db2",
   "metadata": {},
   "source": [
    "Question answering over documents consists of four steps:\n",
    "1) Create an index\n",
    "2) Create a Retriever from that index\n",
    "3) Create a question-answering chain\n",
    "4) Ask questions!\n",
    "\n",
    "\n",
    "To use retrieval in LangChain, you can follow these steps:\n",
    "1) Load documents: Use document loaders to load documents from various sources, such as files, websites, or databases.\n",
    "1) Transform documents: Apply document transformers to preprocess and transform the loaded documents, such as splitting large documents into smaller chunks or applying specific logic optimized for different document types.\n",
    "1) Create embeddings: Generate embeddings for the documents using text embedding models. Embeddings capture the semantic meaning of text and enable efficient searching and similarity calculations.\n",
    "1) Store documents and embeddings: Use vector stores to store the documents and their corresponding embeddings. Vector stores provide efficient storage and retrieval capabilities for large collections of embeddings.\n",
    "1) Retrieve relevant documents: Use retrievers to query the vector store and retrieve relevant documents based on user queries or search criteria. Retriever algorithms, such as similarity search or Maximum Marginal Relevance (MMR) search, can be used to find the most relevant documents."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20e7c887-f3e3-41eb-a6cc-83198f53e984",
   "metadata": {},
   "source": [
    "#### Fully Manual Way of Doing Things"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "id": "f7f31d0a-b284-4c55-aed8-de2c31f4fc34",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import CSVLoader\n",
    "loader = CSVLoader(file_path=file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "id": "298bc490-54bb-4565-8d67-8a2ee74a166a",
   "metadata": {},
   "outputs": [],
   "source": [
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "id": "547af865-74f0-4fa3-94ed-a70a11e53cab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content=\": 0\\nname: Women's Campside Oxfords\\ndescription: This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on. \\n\\nSize & Fit: Order regular shoe size. For half sizes not offered, order up to next whole size. \\n\\nSpecs: Approx. weight: 1 lb.1 oz. per pair. \\n\\nConstruction: Soft canvas material for a broken-in feel and look. Comfortable EVA innersole with Cleansport NXT® antimicrobial odor control. Vintage hunt, fish and camping motif on innersole. Moderate arch contour of innersole. EVA foam midsole for cushioning and support. Chain-tread-inspired molded rubber outsole with modified chain-tread pattern. Imported. \\n\\nQuestions? Please contact us for any inquiries.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 0})"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "id": "d12da33d-b341-4015-a0fb-f6be42e96965",
   "metadata": {},
   "outputs": [],
   "source": [
    "embed = embedding.embed_query(\"Subash\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "id": "a89fea68-d53f-4c53-a341-91f233cc56e9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.014350582845509052,\n",
       " -0.049315445125103,\n",
       " 0.013720898889005184,\n",
       " -0.05654430389404297,\n",
       " 0.08923717588186264]"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embed[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "id": "9693a5f0-797f-47c0-9d9f-0603120f54a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "db = DocArrayInMemorySearch.from_documents(\n",
    "    docs,\n",
    "    embedding\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "id": "36827e4c-1982-4c75-9efe-acdda0126678",
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"Please suggest a shirt with sunblocking\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "id": "a56daae5-820b-47d9-bbc6-a5cec0784f68",
   "metadata": {},
   "outputs": [],
   "source": [
    "response_docs = db.similarity_search(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "id": "78e60119-5b49-4827-bdd8-3af7979a28e8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content=\": 709\\nname: Sunrise Tee\\ndescription: Stay cool, comfortable and dry on the hottest days in our women's UV-protective button down shirt. The lightweight, high-performance fabric wicks away moisture and dries quickly.\\n\\nSize & Fit\\nSlightly Fitted: Softly shapes the body. Falls at hip.\\n\\nWhy We Love It\\nOur lightest hot-weather shirt lets you beat the heat. Originally designed for fishing, it's also a great choice for travel thanks to its wrinkle-free fabric and built-in sun protection with a rating of UPF 50+.\\n\\nFabric & Care\\nLightweight performance synthetic wicks moisture, resists wrinkles and dries fast. Shell: 71% nylon, 29% polyester. Cape lining: 100% polyester. Machine wash and dry.\\n\\nAdditional Features\\nBuilt-in SunSmart™ UPF 50+ rated – the highest rated sun protection possible. The high-performance fabric keeps you cool and comfortable by wicking perspiration away. Smoother buttons, low-profile pockets and side shaping for a flattering fit. Front and back cape venting. Two front pockets, tool tabs and eyewear loop. Wrinkle free. Imported.\\n\\nSun Protection That Won't Wear Off\\nOur\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 709})"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response_docs[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be986a41-b9f6-45f4-b748-2202b10eff95",
   "metadata": {},
   "source": [
    "If we were doing this by hand, we would combine the documents into a single piece of text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "id": "f962dc95-148e-4249-9d62-e10895f30e77",
   "metadata": {},
   "outputs": [],
   "source": [
    "qdocs = \"\".join([docs[i].page_content for i in range(len(response_docs))])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "id": "2dc5e942-2ba1-4ab2-87b5-fefc1c150886",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `BaseChatModel.call_as_llm` was deprecated in langchain-core 0.1.7 and will be removed in 0.3.0. Use invoke instead.\n",
      "  warn_deprecated(\n"
     ]
    }
   ],
   "source": [
    "response = llm_nvidia.call_as_llm(f\"{qdocs} Question: Please list all your \\\n",
    "shirts with sun protection in a table in markdown and summarize each one.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "id": "58fec01a-5f2c-4a87-ad3a-751c4dd4ede3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       " | Name | Description | Size & Fit | Specs | Construction | Fabric & Care | Additional Features |\n",
       "| --- | --- | --- | --- | --- | --- | --- |\n",
       "| Women's Campside Oxfords | Ultracomfortable lace-to-toe Oxford with soft canvas, thick cushioning, and quality construction. | Order regular shoe size. For half sizes not offered, order up to next whole size. | Approx. weight: 1 lb.1 oz. per pair. | Soft canvas material, EVA innersole with Cleansport NXT® antimicrobial odor control, EVA foam midsole for cushioning and support, chain-tread-inspired molded rubber outsole. | Not applicable | Not applicable |\n",
       "| Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece | Bright colored, ruffles and whimsical prints two-piece swimsuit for toddlers. | Designed for infants and toddlers. | Not applicable | Four-way-stretch and chlorine-resistant fabric, UPF 50+ rated fabric, crossover no-slip straps, fully lined bottom. | Machine wash and line dry. | Crossover no-slip straps, fully lined bottom, UPF 50+ rated fabric. |\n",
       "| Refresh Swimwear, V-Neck Tankini Contrasts | Watersport-ready tankini top in eye-catching colorblock style. | Fitted: Sits close to the body. | UPF 50+ rated – the highest rated sun protection possible. | Premium Italian-blend of 82% recycled nylon with 18% Lycra® spandex, lightweight racerback straps, flattering V-neck silhouette. | Handwash, line dry. | Lightweight racerback straps, flattering V-neck silhouette, UPF 50+ rated fabric. |\n",
       "\n",
       "There are 3 items listed with sun protection:\n",
       "\n",
       "1. Women's Campside Oxfords: These shoes offer sun protection through their soft canvas material, which is a good barrier against the sun's harmful rays.\n",
       "2. Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece: This toddler's swimsuit features UPF 50+ rated fabric, providing the highest rated sun protection possible and blocking 98% of the sun's harmful rays.\n",
       "3. Refresh Swimwear, V-Neck Tankini Contrasts: This tankini top is made of UPF 50+ rated fabric, offering the highest rated sun protection possible and blocking 98% of the sun's harmful rays."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b066adcc-28a5-4e13-a690-205ddeb0eb7e",
   "metadata": {},
   "source": [
    "#### Using RetrievalQA"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fcc8512-e9bf-428f-8a50-a84d881940af",
   "metadata": {},
   "source": [
    "For Question Answer over our own documents, we need to create a  **Retriever** from this **Vector Store**.\n",
    "\n",
    "**Retriever** is a generic interface that can be underpinned by any method that takes in a query and returns documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "id": "7b6fe738-ecbf-450a-8ab8-ac21cff8f19c",
   "metadata": {},
   "outputs": [],
   "source": [
    "retriever = db.as_retriever()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ffa6953-1ecc-4756-94fb-d6707e2b2706",
   "metadata": {},
   "source": [
    "All these steps can be encapsulated with the LangChain chain. Here we can create a retrieval QA chain. This does retrieval and does Question Answering over the retrieved documents."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "216cbc9c-d049-48d9-a447-edf336ce8ea2",
   "metadata": {},
   "source": [
    "![Stuff Method](data/images/stuff-method.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc57b82a-3398-430a-bab5-0f4b872abbb7",
   "metadata": {},
   "source": [
    "![Additional Chain Types](data/images/additional-chain-types.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "id": "1d48dafd-802f-47ab-ab56-57be110d474f",
   "metadata": {},
   "outputs": [],
   "source": [
    "qa_stuff = RetrievalQA.from_chain_type(\n",
    "    llm=llm_nvidia,\n",
    "    chain_type=\"stuff\",\n",
    "    retriever=retriever,\n",
    "    verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "id": "98ed7d7d-4ac9-4710-b8df-a807184bbd78",
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"Please list all your shirts with sun protection in a table in markdown and summarize each one.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "id": "f7a91e27-5b3b-4e07-8edb-0dc639c884d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "response = qa_stuff.run(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "id": "71ba6267-33b8-4fa6-b8cf-eee9d4258821",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       " | Name | Description | Size & Fit | Fabric & Care | Additional Features | Sun Protection |\n",
       "| --- | --- | --- | --- | --- | --- |\n",
       "| Sunrise Tee | Lightweight, high-performance shirt for hot weather with UV-protection and moisture-wicking fabric. | Slightly Fitted, falls at hip. | Shell: 71% nylon, 29% polyester. Cape lining: 100% polyester. Machine wash and dry. | Built-in SunSmart™ UPF 50+ rated, wrinkle-free, front and back cape venting, two front pockets, tool tabs and eyewear loop. | Provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays. |\n",
       "| Women's Tropical Tee, Sleeveless | Five-star sleeveless button-up shirt with SunSmart™ protection and a fit to flatter. | Slightly Fitted, falls at hip. | Shell: 71% nylon, 29% polyester. Cape lining: 100% polyester. Machine wash and dry. | Updated design with smoother buttons, wrinkle resistant, front and back cape venting, two front pockets, tool tabs and eyewear loop. | Provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays. |\n",
       "| Sun Shield Shirt | High-performance sun shirt that protects from harmful UV rays, recommended by The Skin Cancer Foundation. | Slightly Fitted, falls at hip. | 78% nylon, 22% Lycra Xtra Life fiber. Handwash, line dry. | Wicks moisture, abrasion resistant, fits comfortably over swimsuit. | Provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays. |\n",
       "| Men's Tropical Plaid Short-Sleeve Shirt | Lightweight, hot-weather shirt with UPF 50+ sun protection and wrinkle-resistant fabric. | Traditional fit, relaxed through chest, sleeve, and waist. | 100% polyester. Machine wash and dry. | Front and back cape venting, two front bellows pockets. | Provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays. |\n",
       "\n",
       "In summary, all the shirts listed provide sun protection with a UPF 50+ rating, which blocks 98% of the sun's harmful rays. The Sunrise Tee and Women's Tropical Tee, Sleeveless are lightweight, high-performance shirts with moisture-wicking fabric and are suitable for hot weather. The Sun Shield Shirt is a high-performance sun shirt that is recommended by The Skin Cancer Foundation and fits comfortably over a swimsuit. The Men's Tropical Plaid Short-Sleeve Shirt is a lightweight, hot-weather shirt with front and back cape venting and two front bellows pockets."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7fa0125-a52e-4c34-8d8d-d4312db4916d",
   "metadata": {},
   "source": [
    "#### Using VectorStoreIndexCreator"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d5281fc-3a8c-4d27-a953-09566f0ac63a",
   "metadata": {},
   "source": [
    "VectorstoreIndexCreator is just a wrapper around all this logic.\n",
    "\n",
    "A lot of the magic is being hidden in this: VectorstoreIndexCreator.\n",
    "Three main steps are going on after the documents are loaded:\n",
    "1) Splitting documents into chunks\n",
    "2) Creating embeddings for each document\n",
    "3) Storing documents and embeddings in a vectorstore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "id": "81288577-38b8-4543-92d5-a17745de86d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "index = VectorstoreIndexCreator(\n",
    "    vectorstore_cls=DocArrayInMemorySearch,\n",
    "    embedding=embedding,\n",
    ").from_loaders([loader])\n",
    "response = index.query(query, llm=llm_nvidia)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "id": "d3a582d6-c291-4494-a6c6-53e8b80a1ff6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       " | Product ID | Name | Description | Fabric & Care | Sun Protection |\n",
       "| --- | --- | --- | --- | --- |\n",
       "| 679 | Women's Tropical Tee, Sleeveless | Five-star sleeveless button-up shirt with SunSmart™ protection, UPF 50+, wrinkle-resistant, machine washable | Shell: 71% nylon, 29% polyester. Cape lining: 100% polyester | High-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays |\n",
       "| 255 | Sun Shield Shirt by | High-performance sun shirt with UPF 50+, wicks moisture, quick-drying, abrasion-resistant, handwash and line dry | 78% nylon, 22% Lycra Xtra Life fiber | High-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays, recommended by The Skin Cancer Foundation as an effective UV protectant |\n",
       "| 618 | Men's Tropical Plaid Short-Sleeve Shirt | Lightweight, UPF 50+ hot-weather shirt, wrinkle-resistant, front and back cape venting, two front bellows pockets, imported | 100% polyester | High-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays |\n",
       "| 709 | Sunrise Tee | Women's UV-protective button-down shirt, lightweight, wicks moisture, quick-drying, wrinkle-free, UPF 50+ | Lightweight performance synthetic wicks moisture, resists wrinkles and dries fast. Shell: 71% nylon, 29% polyester. Cape lining: 100% polyester | High-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays |\n",
       "\n",
       "All the shirts listed provide sun protection with a rating of UPF 50+, blocking 98% of the sun's harmful rays. The first product is a sleeveless button-up shirt for women made of nylon and polyester, with SunSmart™ protection, wrinkle-resistant, and machine washable. The second product is a sun shirt for men made of nylon and Lycra Xtra Life fiber, with moisture-wicking, quick-drying, and abrasion-resistant features, handwash and line dry. The third product is a short-sleeve shirt for men made of 100% polyester, with front and back cape venting, two front bellows pockets, and imported. The last product is a women's UV-protective button-down shirt made of lightweight performance synthetic, wicks moisture, quick-drying, and wrinkle-free."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ff1b6b9-8864-4d0c-a6ae-d1153ee29d5f",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# LangChain Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08a79926-23d8-4ec2-8042-500a48583446",
   "metadata": {},
   "source": [
    "## Create our QandA application"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "id": "bf9ef1db-39d9-4135-a512-688dd1acee41",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.document_loaders import CSVLoader\n",
    "from langchain.indexes import VectorstoreIndexCreator\n",
    "from langchain.vectorstores import DocArrayInMemorySearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "id": "323dede5-1dbe-4ee2-af0b-ca9a790ef3a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "file = 'data/files/OutdoorClothingCatalog_1000.csv'\n",
    "loader = CSVLoader(file_path=file)\n",
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "id": "d82181a7-b65c-4164-a8b9-29c4b9864a8c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from langchain_huggingface import HuggingFaceEmbeddings\n",
    "\n",
    "embedding = HuggingFaceEmbeddings(model_name=\"BAAI/bge-small-en-v1.5\")\n",
    "\n",
    "index = VectorstoreIndexCreator(\n",
    "    vectorstore_cls=DocArrayInMemorySearch,\n",
    "    embedding = embedding\n",
    ").from_loaders([loader])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "id": "140fc5fb-4df5-4e8c-b92a-cea21d715549",
   "metadata": {},
   "outputs": [],
   "source": [
    "qa = RetrievalQA.from_chain_type(\n",
    "    llm=llm_nvidia, \n",
    "    chain_type=\"stuff\", \n",
    "    retriever=index.vectorstore.as_retriever(), \n",
    "    verbose=True,\n",
    "    chain_type_kwargs = {\n",
    "        \"document_separator\": \"<<<<>>>>>\"\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "id": "ada4f528-73af-4e20-8bfc-b580ee473ead",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content=\": 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\n\\nSize & Fit\\n- Pants are Favorite Fit: Sits lower on the waist.\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\n\\nFabric & Care\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features\\n- Relaxed fit top with raglan sleeves and rounded hem.\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\n\\nImported.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 10})"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63f98a13-e1ae-4a59-9ab3-e0f71b7580bb",
   "metadata": {},
   "source": [
    "### Hard-coded examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "id": "b0f89d35-fdfe-4b4e-96be-9bcc40c6e91e",
   "metadata": {},
   "outputs": [],
   "source": [
    "examples = [\n",
    "    {\"query\": \"Do the Cozy Comfort Pullover Set\\\n",
    "    have side pockets?\",\n",
    "    \"answer\": \"Yes\"\n",
    "    },\n",
    "     {\"query\": \"What collection is the Ultra-Lofty \\\n",
    "    850 Stretch Down Hooded Jacket from?\",\n",
    "    \"answer\": \"The DownTek collection\"\n",
    "     }\n",
    "]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d71023b7-551c-4258-ba17-a842975b22f0",
   "metadata": {},
   "source": [
    "### LLM-Generated examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "id": "5a01bf30-f243-431f-ae3b-dad5e32dbb32",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.evaluation.qa import QAGenerateChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "id": "e0195a99-ed23-488a-b852-67cdb21fd847",
   "metadata": {},
   "outputs": [],
   "source": [
    "example_gen_chain = QAGenerateChain.from_llm(ChatNVIDIA())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "id": "76afc29b-26a0-4504-98b7-098698a86821",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mojo/miniconda3/lib/python3.12/site-packages/langchain/chains/llm.py:367: UserWarning: The apply_and_parse method is deprecated, instead pass an output parser directly to LLMChain.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "new_examples = example_gen_chain.apply_and_parse(\n",
    "    [{\"doc\": t} for t in data[:5]]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "id": "31c6dc35-e07c-4c64-97df-41962c3eb035",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_examples = [ex.get(\"qa_pairs\") for ex in new_examples]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "id": "08296e8a-518d-4b6a-a065-1e275237c3fe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': \"What is the material of the Women's Campside Oxfords and what is its special feature?\",\n",
       " 'answer': \"The Women's Campside Oxfords are made of soft canvas material, which provides a broken-in feel and look. One special feature of these shoes is the Cleansport NXT® antimicrobial odor control in the innersole.\"}"
      ]
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_examples[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "id": "3306408c-73f2-4630-a0f0-26ff687fb782",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content=\": 0\\nname: Women's Campside Oxfords\\ndescription: This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on. \\n\\nSize & Fit: Order regular shoe size. For half sizes not offered, order up to next whole size. \\n\\nSpecs: Approx. weight: 1 lb.1 oz. per pair. \\n\\nConstruction: Soft canvas material for a broken-in feel and look. Comfortable EVA innersole with Cleansport NXT® antimicrobial odor control. Vintage hunt, fish and camping motif on innersole. Moderate arch contour of innersole. EVA foam midsole for cushioning and support. Chain-tread-inspired molded rubber outsole with modified chain-tread pattern. Imported. \\n\\nQuestions? Please contact us for any inquiries.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 0})"
      ]
     },
     "execution_count": 208,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "id": "c8697f14-061b-4edb-ada6-bec546e8467d",
   "metadata": {},
   "outputs": [],
   "source": [
    "examples += new_examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "id": "ad4e1445-e743-46d9-83fb-d2403944dfc7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': 'Do the Cozy Comfort Pullover Set    have side pockets?',\n",
       " 'answer': 'Yes'}"
      ]
     },
     "execution_count": 191,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "examples[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "id": "f865f67c-84c9-4bf2-9c0f-4a5549cf3d43",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \"Additional Features\" section of the set\\'s description. It mentions that the pull-on pants have \"side pockets.\"'"
      ]
     },
     "execution_count": 193,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa.run(examples[0][\"query\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a098c0ac-a97e-44a7-9e84-b36836cca2e6",
   "metadata": {},
   "source": [
    "### Manual Evaluation¶"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "id": "369408e2-c92a-40e6-8e61-a4f77014024f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import langchain\n",
    "langchain.debug = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "id": "adbb402f-63ba-4fdb-bde0-dd02608ac4f0",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[chain:RetrievalQA] Entering Chain run with input:\n",
      "\u001b[0m{\n",
      "  \"query\": \"Do the Cozy Comfort Pullover Set    have side pockets?\"\n",
      "}\n",
      "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[chain:RetrievalQA > chain:StuffDocumentsChain] Entering Chain run with input:\n",
      "\u001b[0m[inputs]\n",
      "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[chain:RetrievalQA > chain:StuffDocumentsChain > chain:LLMChain] Entering Chain run with input:\n",
      "\u001b[0m{\n",
      "  \"question\": \"Do the Cozy Comfort Pullover Set    have side pockets?\",\n",
      "  \"context\": \": 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\n\\nSize & Fit\\n- Pants are Favorite Fit: Sits lower on the waist.\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\n\\nFabric & Care\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features\\n- Relaxed fit top with raglan sleeves and rounded hem.\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\n\\nImported.<<<<>>>>>: 73\\nname: Cozy Cuddles Knit Pullover Set\\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \\n\\nSize & Fit \\nPants are Favorite Fit: Sits lower on the waist. \\nRelaxed Fit: Our most generous fit sits farthest from the body. \\n\\nFabric & Care \\nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features \\nRelaxed fit top with raglan sleeves and rounded hem. \\nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \\nImported.<<<<>>>>>: 854\\nname: Urban Striped Quarter-Zip Terry Pullover\\ndescription: Our all-cotton french terry quarter-zip is designed with appealing texture, a versatile season-spanning weight and nautical-inspired stripes. Size & Fit: Slim Fit - cut slim through the chest, sleeve and waist. Why We Love It: We've turned the typical sweatshirt inside out. Our pullover features the reverse side of our french terry fabric on the outside for outstanding texture, while the inside is nice and smooth. It has the undeniable comfort of cotton, with a neater look than most sweatshirts and a modern, slimmer fit. Our striped pullover is the perfect weekend layer, especially between seasons - and we love the side pockets! Fabric & Care: 100% cotton in a French terry knit. Machine wash and dry. Additional Features: Sturdy metal zipper adds to its rugged appeal. On-seam side pockets. Ribbed cuffs keep their shape. Side vents at hem. Imported.<<<<>>>>>: 830\\nname: Relaxed Fit Pull-On Pants\"\n",
      "}\n",
      "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[chain:RetrievalQA > chain:StuffDocumentsChain > chain:LLMChain > llm:ChatNVIDIA] Entering LLM run with input:\n",
      "\u001b[0m{\n",
      "  \"prompts\": [\n",
      "    \"System: Use the following pieces of context to answer the user's question. \\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\\n----------------\\n: 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\n\\nSize & Fit\\n- Pants are Favorite Fit: Sits lower on the waist.\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\n\\nFabric & Care\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features\\n- Relaxed fit top with raglan sleeves and rounded hem.\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\n\\nImported.<<<<>>>>>: 73\\nname: Cozy Cuddles Knit Pullover Set\\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \\n\\nSize & Fit \\nPants are Favorite Fit: Sits lower on the waist. \\nRelaxed Fit: Our most generous fit sits farthest from the body. \\n\\nFabric & Care \\nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features \\nRelaxed fit top with raglan sleeves and rounded hem. \\nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \\nImported.<<<<>>>>>: 854\\nname: Urban Striped Quarter-Zip Terry Pullover\\ndescription: Our all-cotton french terry quarter-zip is designed with appealing texture, a versatile season-spanning weight and nautical-inspired stripes. Size & Fit: Slim Fit - cut slim through the chest, sleeve and waist. Why We Love It: We've turned the typical sweatshirt inside out. Our pullover features the reverse side of our french terry fabric on the outside for outstanding texture, while the inside is nice and smooth. It has the undeniable comfort of cotton, with a neater look than most sweatshirts and a modern, slimmer fit. Our striped pullover is the perfect weekend layer, especially between seasons - and we love the side pockets! Fabric & Care: 100% cotton in a French terry knit. Machine wash and dry. Additional Features: Sturdy metal zipper adds to its rugged appeal. On-seam side pockets. Ribbed cuffs keep their shape. Side vents at hem. Imported.<<<<>>>>>: 830\\nname: Relaxed Fit Pull-On Pants\\nHuman: Do the Cozy Comfort Pullover Set    have side pockets?\"\n",
      "  ]\n",
      "}\n",
      "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[chain:RetrievalQA > chain:StuffDocumentsChain > chain:LLMChain > llm:ChatNVIDIA] [716ms] Exiting LLM run with output:\n",
      "\u001b[0m{\n",
      "  \"generations\": [\n",
      "    [\n",
      "      {\n",
      "        \"text\": \" Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \\\"Additional Features\\\" section of the set's description. It mentions that the pull-on pants have \\\"side pockets.\\\"\",\n",
      "        \"generation_info\": null,\n",
      "        \"type\": \"ChatGeneration\",\n",
      "        \"message\": {\n",
      "          \"lc\": 1,\n",
      "          \"type\": \"constructor\",\n",
      "          \"id\": [\n",
      "            \"langchain\",\n",
      "            \"schema\",\n",
      "            \"messages\",\n",
      "            \"ChatMessage\"\n",
      "          ],\n",
      "          \"kwargs\": {\n",
      "            \"content\": \" Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \\\"Additional Features\\\" section of the set's description. It mentions that the pull-on pants have \\\"side pockets.\\\"\",\n",
      "            \"response_metadata\": {\n",
      "              \"role\": \"assistant\",\n",
      "              \"content\": \" Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \\\"Additional Features\\\" section of the set's description. It mentions that the pull-on pants have \\\"side pockets.\\\"\",\n",
      "              \"token_usage\": {\n",
      "                \"prompt_tokens\": 737,\n",
      "                \"total_tokens\": 786,\n",
      "                \"completion_tokens\": 49\n",
      "              },\n",
      "              \"model_name\": \"mistralai/mixtral-8x7b-instruct-v0.1\"\n",
      "            },\n",
      "            \"type\": \"chat\",\n",
      "            \"id\": \"run-23547d08-aad6-4617-b615-4e5c396edab7-0\",\n",
      "            \"role\": \"assistant\"\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    ]\n",
      "  ],\n",
      "  \"llm_output\": {\n",
      "    \"role\": \"assistant\",\n",
      "    \"content\": \" Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \\\"Additional Features\\\" section of the set's description. It mentions that the pull-on pants have \\\"side pockets.\\\"\",\n",
      "    \"token_usage\": {\n",
      "      \"prompt_tokens\": 737,\n",
      "      \"total_tokens\": 786,\n",
      "      \"completion_tokens\": 49\n",
      "    },\n",
      "    \"model_name\": \"mistralai/mixtral-8x7b-instruct-v0.1\"\n",
      "  },\n",
      "  \"run\": null\n",
      "}\n",
      "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[chain:RetrievalQA > chain:StuffDocumentsChain > chain:LLMChain] [717ms] Exiting Chain run with output:\n",
      "\u001b[0m{\n",
      "  \"text\": \" Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \\\"Additional Features\\\" section of the set's description. It mentions that the pull-on pants have \\\"side pockets.\\\"\"\n",
      "}\n",
      "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[chain:RetrievalQA > chain:StuffDocumentsChain] [718ms] Exiting Chain run with output:\n",
      "\u001b[0m{\n",
      "  \"output_text\": \" Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \\\"Additional Features\\\" section of the set's description. It mentions that the pull-on pants have \\\"side pockets.\\\"\"\n",
      "}\n",
      "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[chain:RetrievalQA] [746ms] Exiting Chain run with output:\n",
      "\u001b[0m{\n",
      "  \"result\": \" Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \\\"Additional Features\\\" section of the set's description. It mentions that the pull-on pants have \\\"side pockets.\\\"\"\n",
      "}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \"Additional Features\" section of the set\\'s description. It mentions that the pull-on pants have \"side pockets.\"'"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa.run(examples[0][\"query\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "id": "63d9a612-549f-4db0-975b-de04983d1e7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Turn off the debug mode\n",
    "langchain.debug = False"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36620152-c1ab-4d36-8782-da0f829b88ef",
   "metadata": {},
   "source": [
    "### LLM assisted evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "id": "f67fa531-868e-4fda-b663-3dc8886ae043",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': \"What is the material of the Women's Campside Oxfords and what is its special feature?\",\n",
       " 'answer': \"The Women's Campside Oxfords are made of soft canvas material, which provides a broken-in feel and look. One special feature of these shoes is the Cleansport NXT® antimicrobial odor control in the innersole.\"}"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "examples[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "06443434-c066-455e-90eb-fff0c375cdc4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "predictions = qa.batch(examples)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "id": "99206b06-dc70-40e4-9f74-9b3940a5f54b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'query': 'Do the Cozy Comfort Pullover Set    have side pockets?',\n",
       "  'answer': 'Yes',\n",
       "  'result': ' Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \"Additional Features\" section of the set\\'s description. It mentions that the pull-on pants have \"side pockets.\"'},\n",
       " {'query': 'What collection is the Ultra-Lofty     850 Stretch Down Hooded Jacket from?',\n",
       "  'answer': 'The DownTek collection',\n",
       "  'result': ' The Ultra-Lofty 850 Stretch Down Hooded Jacket is from the DownTek collection. This is stated in the description of the jacket: \"This technical stretch down jacket from our DownTek collection is sure to keep you warm and comfortable...\"'},\n",
       " {'query': \"What is the material of the Women's Campside Oxfords and what is its special feature?\",\n",
       "  'answer': \"The Women's Campside Oxfords are made of soft canvas material, which provides a broken-in feel and look. One special feature of these shoes is the Cleansport NXT® antimicrobial odor control in the innersole.\",\n",
       "  'result': \" The Women's Campside Oxfords are made of a soft canvas material. A special feature of these shoes is that they have a broken-in feel from the first time you put them on, thanks to their super-soft canvas and thick cushioning.\"},\n",
       " {'query': 'What is the material and construction of the Recycled Waterhog Dog Mat, Chevron Weave?',\n",
       "  'answer': 'The Recycled Waterhog Dog Mat, Chevron Weave is made of 24 oz. polyester fabric which consists of 94% recycled materials. It has a rubber backing for added durability. The mat features thick and thin fibers that are designed for scraping dirt and absorbing water. It is quick drying, resistant to fading, rotting, mildew, and shedding, and can be used both indoors and outdoors. The chevron weave design is exclusive to this product.',\n",
       "  'result': ' The Recycled Waterhog Dog Mat, Chevron Weave is made of 24 oz. polyester fabric made from 94% recycled materials. It has a rubber backing and features thick and thin fibers for scraping dirt and absorbing water. The mat can be vacuumed or hosed clean and can be used indoors or outdoors. It is constructed in the USA.'},\n",
       " {'query': \"What are the key features of the Infant and Toddler Girls' Coastal Chill Swimsuit?\",\n",
       "  'answer': \"The Infant and Toddler Girls' Coastal Chill Swimsuit is a two-piece swimsuit with bright colors, ruffles, and exclusive whimsical prints. The fabric has four-way-stretch and is chlorine-resistant, which helps it keep its shape and resist snags. The UPF 50+ rated fabric provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage. Machine washing and line drying are recommended for this imported item.\",\n",
       "  'result': \" The Infant and Toddler Girls' Coastal Chill Swimsuit has the following key features:\\n\\n1. It is a two-piece swimsuit that comes in bright colors and exclusive whimsical prints.\\n2. The swimsuit is made of four-way-stretch and chlorine-resistant fabric, which helps it keep its shape and resist snags.\\n3. The UPF 50+ rated fabric provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays.\\n4. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage.\\n5. The swimsuit is machine washable and imported.\"},\n",
       " {'query': 'What is the composition of the body and lining of the Refresh Swimwear, V-Neck Tankini Contrasts and what sun protection rating does it have?',\n",
       "  'answer': 'The body of the Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon and 18% Lycra® spandex, while the lining is made of 90% recycled nylon and 10% Lycra® spandex. This swimwear features UPF 50+ rating, the highest rated sun protection possible.',\n",
       "  'result': ' The body of the Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon and 18% Lycra® spandex. The lining is made of 90% recycled nylon and 10% Lycra® spandex. This fabric is UPF 50+ rated, which is the highest rated sun protection possible.'},\n",
       " {'query': 'What technology is used in the EcoFlex 3L Storm Pants to increase breathability while maintaining waterproof protection?',\n",
       "  'answer': \"The EcoFlex 3L Storm Pants use TEK O2 technology to increase breathability while maintaining waterproof protection. This technology has been tested to offer the most breathability in the brand's line of products, making it suitable for various outdoor activities throughout the year.\",\n",
       "  'result': ' The EcoFlex 3L Storm Pants use TEK O2 technology to increase breathability while maintaining waterproof protection. This air-permeable technology moves moisture away instantly, providing maximum protection against the elements during various outdoor activities year-round.'}]"
      ]
     },
     "execution_count": 199,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "id": "836fc38d-6348-43dc-81d5-d3a375b234c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.evaluation.qa import QAEvalChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "id": "43967b60-310a-45ae-b658-a9fd1a804172",
   "metadata": {},
   "outputs": [],
   "source": [
    "eval_chain = QAEvalChain.from_llm(llm_nvidia)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "id": "4f65de03-6287-49b2-9d1b-71796890dee8",
   "metadata": {},
   "outputs": [],
   "source": [
    "graded_outputs = eval_chain.evaluate(examples, predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "id": "6a01e3f6-c740-445e-8376-71c639665169",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'results': ' CORRECT\\n\\nThe student answer is factually correct. The Cozy Comfort Pullover Set does have side pockets, as stated in the \"Additional Features\" section of the set\\'s description. The student\\'s answer provides additional context, but it does not contain any conflicting information.'},\n",
       " {'results': ' CORRECT\\n\\nExplanation:\\nThe student answer correctly identifies the Ultra-Lofty 850 Stretch Down Hooded Jacket as being from the DownTek collection. The additional information provided in the student answer does not contradict the true answer, so it is still considered correct.'},\n",
       " {'results': \" INCORRECT\\n\\nAlthough the student answer correctly identifies the material of the Women's Campside Oxfords as soft canvas, the special feature mentioned in the student answer (super-soft canvas and thick cushioning) is not the same as the special feature given in the true answer (Cleansport NXT® antimicrobial odor control in the innersole).\"},\n",
       " {'results': \" GRADE: CORRECT\\n\\nThe student answer correctly identifies the material and construction of the Recycled Waterhog Dog Mat, Chevron Weave. While the student answer provides additional details about the mat's maintenance and design, it does not contain any conflicting statements. Therefore, the student answer is factually accurate and can be considered correct.\"},\n",
       " {'results': \" CORRECT\\n\\nThe student answer correctly identifies all the key features of the Infant and Toddler Girls' Coastal Chill Swimsuit. While the true answer mentions ruffles and the student answer does not, this is not a contradiction and therefore does not affect the grade. The student answer also includes the additional information that the swimsuit comes in bright colors, which is true and does not conflict with the true answer.\"},\n",
       " {'results': ' CORRECT\\n\\nThe student answer correctly identifies the composition of the body and lining of the Refresh Swimwear, V-Neck Tankini Contrasts and the sun protection rating it has. The factual information provided by the student is consistent with the true answer.'},\n",
       " {'results': ' CORRECT\\n\\nExplanation:\\n\\nThe student answer correctly identifies the technology used in the EcoFlex 3L Storm Pants for increasing breathability while maintaining waterproof protection. The student answer provides additional information about the technology, but it does not conflict with the true answer. Therefore, the student answer is factually accurate and can be considered correct.'}]"
      ]
     },
     "execution_count": 204,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graded_outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "id": "d9ae32d7-ff81-41fc-977e-c810ccf96d67",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__________________________________________________\n",
      "Example 0:\n",
      "Question: Do the Cozy Comfort Pullover Set    have side pockets?\n",
      "Real Answer: Yes\n",
      "Predicted Answer:  Yes, the Cozy Comfort Pullover Set does have side pockets. This information can be found in the \"Additional Features\" section of the set's description. It mentions that the pull-on pants have \"side pockets.\"\n",
      "Predicted Grade:  CORRECT\n",
      "\n",
      "The student answer is factually correct. The Cozy Comfort Pullover Set does have side pockets, as stated in the \"Additional Features\" section of the set's description. The student's answer provides additional context, but it does not contain any conflicting information.\n",
      "__________________________________________________\n",
      "__________________________________________________\n",
      "Example 1:\n",
      "Question: What collection is the Ultra-Lofty     850 Stretch Down Hooded Jacket from?\n",
      "Real Answer: The DownTek collection\n",
      "Predicted Answer:  The Ultra-Lofty 850 Stretch Down Hooded Jacket is from the DownTek collection. This is stated in the description of the jacket: \"This technical stretch down jacket from our DownTek collection is sure to keep you warm and comfortable...\"\n",
      "Predicted Grade:  CORRECT\n",
      "\n",
      "Explanation:\n",
      "The student answer correctly identifies the Ultra-Lofty 850 Stretch Down Hooded Jacket as being from the DownTek collection. The additional information provided in the student answer does not contradict the true answer, so it is still considered correct.\n",
      "__________________________________________________\n",
      "__________________________________________________\n",
      "Example 2:\n",
      "Question: What is the material of the Women's Campside Oxfords and what is its special feature?\n",
      "Real Answer: The Women's Campside Oxfords are made of soft canvas material, which provides a broken-in feel and look. One special feature of these shoes is the Cleansport NXT® antimicrobial odor control in the innersole.\n",
      "Predicted Answer:  The Women's Campside Oxfords are made of a soft canvas material. A special feature of these shoes is that they have a broken-in feel from the first time you put them on, thanks to their super-soft canvas and thick cushioning.\n",
      "Predicted Grade:  INCORRECT\n",
      "\n",
      "Although the student answer correctly identifies the material of the Women's Campside Oxfords as soft canvas, the special feature mentioned in the student answer (super-soft canvas and thick cushioning) is not the same as the special feature given in the true answer (Cleansport NXT® antimicrobial odor control in the innersole).\n",
      "__________________________________________________\n",
      "__________________________________________________\n",
      "Example 3:\n",
      "Question: What is the material and construction of the Recycled Waterhog Dog Mat, Chevron Weave?\n",
      "Real Answer: The Recycled Waterhog Dog Mat, Chevron Weave is made of 24 oz. polyester fabric which consists of 94% recycled materials. It has a rubber backing for added durability. The mat features thick and thin fibers that are designed for scraping dirt and absorbing water. It is quick drying, resistant to fading, rotting, mildew, and shedding, and can be used both indoors and outdoors. The chevron weave design is exclusive to this product.\n",
      "Predicted Answer:  The Recycled Waterhog Dog Mat, Chevron Weave is made of 24 oz. polyester fabric made from 94% recycled materials. It has a rubber backing and features thick and thin fibers for scraping dirt and absorbing water. The mat can be vacuumed or hosed clean and can be used indoors or outdoors. It is constructed in the USA.\n",
      "Predicted Grade:  GRADE: CORRECT\n",
      "\n",
      "The student answer correctly identifies the material and construction of the Recycled Waterhog Dog Mat, Chevron Weave. While the student answer provides additional details about the mat's maintenance and design, it does not contain any conflicting statements. Therefore, the student answer is factually accurate and can be considered correct.\n",
      "__________________________________________________\n",
      "__________________________________________________\n",
      "Example 4:\n",
      "Question: What are the key features of the Infant and Toddler Girls' Coastal Chill Swimsuit?\n",
      "Real Answer: The Infant and Toddler Girls' Coastal Chill Swimsuit is a two-piece swimsuit with bright colors, ruffles, and exclusive whimsical prints. The fabric has four-way-stretch and is chlorine-resistant, which helps it keep its shape and resist snags. The UPF 50+ rated fabric provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage. Machine washing and line drying are recommended for this imported item.\n",
      "Predicted Answer:  The Infant and Toddler Girls' Coastal Chill Swimsuit has the following key features:\n",
      "\n",
      "1. It is a two-piece swimsuit that comes in bright colors and exclusive whimsical prints.\n",
      "2. The swimsuit is made of four-way-stretch and chlorine-resistant fabric, which helps it keep its shape and resist snags.\n",
      "3. The UPF 50+ rated fabric provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays.\n",
      "4. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage.\n",
      "5. The swimsuit is machine washable and imported.\n",
      "Predicted Grade:  CORRECT\n",
      "\n",
      "The student answer correctly identifies all the key features of the Infant and Toddler Girls' Coastal Chill Swimsuit. While the true answer mentions ruffles and the student answer does not, this is not a contradiction and therefore does not affect the grade. The student answer also includes the additional information that the swimsuit comes in bright colors, which is true and does not conflict with the true answer.\n",
      "__________________________________________________\n",
      "__________________________________________________\n",
      "Example 5:\n",
      "Question: What is the composition of the body and lining of the Refresh Swimwear, V-Neck Tankini Contrasts and what sun protection rating does it have?\n",
      "Real Answer: The body of the Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon and 18% Lycra® spandex, while the lining is made of 90% recycled nylon and 10% Lycra® spandex. This swimwear features UPF 50+ rating, the highest rated sun protection possible.\n",
      "Predicted Answer:  The body of the Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon and 18% Lycra® spandex. The lining is made of 90% recycled nylon and 10% Lycra® spandex. This fabric is UPF 50+ rated, which is the highest rated sun protection possible.\n",
      "Predicted Grade:  CORRECT\n",
      "\n",
      "The student answer correctly identifies the composition of the body and lining of the Refresh Swimwear, V-Neck Tankini Contrasts and the sun protection rating it has. The factual information provided by the student is consistent with the true answer.\n",
      "__________________________________________________\n",
      "__________________________________________________\n",
      "Example 6:\n",
      "Question: What technology is used in the EcoFlex 3L Storm Pants to increase breathability while maintaining waterproof protection?\n",
      "Real Answer: The EcoFlex 3L Storm Pants use TEK O2 technology to increase breathability while maintaining waterproof protection. This technology has been tested to offer the most breathability in the brand's line of products, making it suitable for various outdoor activities throughout the year.\n",
      "Predicted Answer:  The EcoFlex 3L Storm Pants use TEK O2 technology to increase breathability while maintaining waterproof protection. This air-permeable technology moves moisture away instantly, providing maximum protection against the elements during various outdoor activities year-round.\n",
      "Predicted Grade:  CORRECT\n",
      "\n",
      "Explanation:\n",
      "\n",
      "The student answer correctly identifies the technology used in the EcoFlex 3L Storm Pants for increasing breathability while maintaining waterproof protection. The student answer provides additional information about the technology, but it does not conflict with the true answer. Therefore, the student answer is factually accurate and can be considered correct.\n",
      "__________________________________________________\n"
     ]
    }
   ],
   "source": [
    "for i, eg in enumerate(examples):\n",
    "    print(\"_\"*50)\n",
    "    print(f\"Example {i}:\")\n",
    "    print(\"Question: \" + predictions[i]['query'])\n",
    "    print(\"Real Answer: \" + predictions[i]['answer'])\n",
    "    print(\"Predicted Answer: \" + predictions[i]['result'])\n",
    "    print(\"Predicted Grade: \" + graded_outputs[i]['results'])\n",
    "    print(\"_\"*50)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8354f361-f92f-4416-a149-b3c5631c6b8a",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Agents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "id": "bc1a92b3-7e4a-4802-bec5-5cc08c350e85",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install langchain-experimental\n",
    "# !pip install numexpr\n",
    "#!pip install wikipedia\n",
    "#!pip install langchainhub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 291,
   "id": "5dc9b357-7b73-4ce1-82a4-da73cf15dff8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.agents import AgentExecutor, load_tools, create_react_agent \n",
    "from langchain_core.tools import Tool\n",
    "from langchain_experimental.utilities import PythonREPL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 292,
   "id": "d376ba7b-271f-44b0-9c37-3ff3a86677fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "tools = load_tools([\"llm-math\",\"wikipedia\"], llm=llm_nvidia)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 341,
   "id": "5d8e2872-31de-4f77-a3c0-395deb0969c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_text = '''Answer the following questions as best you can. You have access to the following tools:\n",
    "\n",
    "{tools}\n",
    "\n",
    "Use the following format:\n",
    "\n",
    "Question: the input question you must answer\n",
    "Thought: you should always think about what to do\n",
    "Action: the action to take, should be one of [{tool_names}]\n",
    "Action Input: the input to the action\n",
    "Observation: the result of the action\n",
    "... (this Thought/Action/Action Input/Observation can repeat N times)\n",
    "Thought: I now know the final answer\n",
    "Final Answer: the final answer to the original input question\n",
    "\n",
    "Begin!\n",
    "\n",
    "Question: {input}\n",
    "Thought:{agent_scratchpad}'''\n",
    "prompt = PromptTemplate.from_template(prompt_text)\n",
    "agent = create_react_agent(llm = llm_nvidia, tools = tools, prompt = prompt)\n",
    "agent_executor = AgentExecutor(agent=agent, tools=tools)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 344,
   "id": "50d93533-f79c-4df3-8cfc-90c3e469a36c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The 25% of 300 is 75.0.\n"
     ]
    }
   ],
   "source": [
    "question = {\n",
    "    \"input\": \"What is the 25% of 300?\"\n",
    "}\n",
    "\n",
    "result = agent_executor.invoke(question)\n",
    "print(result['output'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "318a2e0f-8e74-41fd-9c69-9bcfef60df4d",
   "metadata": {},
   "source": [
    "## Wikipedia example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 345,
   "id": "f45712e0-898f-4a12-abb8-bbb3e6caa0b2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thomas G. Mitchell, a computer scientist from Carnegie Mellon University, wrote a book, but I do not have information about the title or content of the book.\n"
     ]
    }
   ],
   "source": [
    "question = {\n",
    "    \"input\" : \"Tom Michael Mitchell is an American computer scientist \\\n",
    "and the Founders University Professor at Carnegie Mellon University (CMU)\\\n",
    "what book did he write?\"\n",
    "}\n",
    "result = agent_executor.invoke(question) \n",
    "print(result['output'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af249a29-57b9-407d-a90f-1b03f8e0adf4",
   "metadata": {},
   "source": [
    "## Python Agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 347,
   "id": "31ec447b-65ea-4713-b58d-fbb1d70a8097",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'4\\n'"
      ]
     },
     "execution_count": 347,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "python_repl = PythonREPL()\n",
    "python_repl.run(\"print(2+2)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "998417f5-71e4-4f0c-950f-5b40ebc5a005",
   "metadata": {},
   "outputs": [],
   "source": [
    "repl_tool = Tool(\n",
    "    name=\"python_repl\",\n",
    "    description=\"A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\",\n",
    "    func=python_repl.run,\n",
    ")\n",
    "agent = create_react_agent(llm = llm_nvidia, tools = [repl_tool], prompt = prompt)\n",
    "agent_executor = AgentExecutor(agent=agent, tools=[repl_tool], handle_parsing_errors=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 333,
   "id": "d6fe43f8-4115-4e68-9246-159067b38b15",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_list = [[\"Harrison\", \"Chase\"], \n",
    "                 [\"Lang\", \"Chain\"],\n",
    "                 [\"Dolly\", \"Too\"],\n",
    "                 [\"Elle\", \"Elem\"], \n",
    "                 [\"Geoff\",\"Fusion\"], \n",
    "                 [\"Trance\",\"Former\"],\n",
    "                 [\"Jen\",\"Ayai\"]\n",
    "                ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 339,
   "id": "a4888d57-9617-4c9e-ba7d-17f81f254fdc",
   "metadata": {},
   "outputs": [],
   "source": [
    "question = {\n",
    "    \"input\": f\"\"\"Sort these customers by last name \n",
    "    and then first name \n",
    "    and print the output: {customer_list}\"\"\"\n",
    "}\n",
    "\n",
    "result = agent_executor.invoke(question)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "id": "a4948288-9097-4517-b71f-9a5657564077",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The sorted list of customers by last name and then first name is:\n",
      "[['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Harrison', 'Chase'], ['Jen', 'Ayai'], ['Lang', 'Chain'], ['Trance', 'Former']]\n"
     ]
    }
   ],
   "source": [
    "print(result['output'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e46df4c-15b8-4d15-a639-3c79c8d73f29",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}