gabo_rag.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rvxgds6SxekA"
      },
      "source": [
        "# Gabo RAG | Daniel Felipe Montenegro\n",
        "**'Gabo'** is a **RAG (Retrieval-Augmented Generation)** system designed to enhance the capabilities of **LLMs (Large Language Models)** such as **'Llama 3.2'** or **'Phi 3.5**'. This project honors Colombian author **Gabriel García Márquez** by marking the tenth anniversary of his death, creating a specialized assistant to answer questions about his work, and using new technologies to further reveal his literary legacy.\n",
        "\n",
        "[**Python Notebook**](https://github.com/dafmontenegro/gabo-rag/blob/master/gabo_rag.ipynb) | [**Webpage**](https://montenegrodanielfelipe.com/projects/gabo-rag/) | [**Repository**](https://github.com/dafmontenegro/gabo-rag)\n",
        "\n",
        "- [1. Tools and Technologies](#1-tools-and-technologies)\n",
        "- [2. How to run Ollama in Google Colab?](#2-how-to-run-ollama-in-google-colab)\n",
        "    - [2.1 Ollama Installation](#21-ollama-installation)\n",
        "    - [2.2 Run 'ollama serve'](#22-run-ollama-serve)\n",
        "    - [2.3 Run 'ollama pull \\<model\\_name\\>'](#23-run-ollama-pull-model_name)\n",
        "- [3. Exploring LLMs](#3-exploring-llms)\n",
        "- [4. Data Extraction and Preparation](#4-data-extraction-and-preparation)\n",
        "    - [4.1 Web Scraping and Chunking](#41-web-scraping-and-chunking)\n",
        "    - [4.2 Embedding Model: Nomic](#42-embedding-model-nomic)\n",
        "- [5. Storing in the Vector Database](#5-storing-in-the-vector-database)\n",
        "    - [5.1 Making Chroma Persistent](#51-making-chroma-persistent)\n",
        "    - [5.2 Adding Documents to Chroma](#52-adding-documents-to-chroma)\n",
        "- [6. Use a Vectorstore as a Retriever](#6-use-a-vectorstore-as-a-retriever)\n",
        "- [7. RAG (Retrieval-Augmented Generation)](#7-rag-retrieval-augmented-generation)\n",
        "- [8. References](#8-references)\n",
        "\n",
        "## Author\n",
        "\n",
        "- **Daniel Felipe Montenegro** [Website](https://montenegrodanielfelipe.com/) | [GitHub](https://github.com/dafmontenegro)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oD1C9HDGxekF"
      },
      "source": [
        "## 1. Tools and Technologies\n",
        "\n",
        "- [**Ollama**](https://ollama.com/): Running models ([Llama 3.2](https://ollama.com/library/llama3.2) or [Phi 3.5](https://ollama.com/library/phi3.5)) and embeddings ([Nomic](https://ollama.com/library/nomic-embed-text))\n",
        "- [**LangChain**](https://python.langchain.com/docs/introduction/): Framework and web scraping tool\n",
        "- [**Chroma**](https://docs.trychroma.com/): Vector database\n",
        "\n",
        "> A special thanks to ['Ciudad Seva (Casa digital del escritor Luis López Nieves)'](https://ciudadseva.com/quienes-somos/), from which the texts used in this project were extracted and where a comprehensive [Spanish Digital Library](https://ciudadseva.com/biblioteca/) is available."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FMv75HvtxekG"
      },
      "source": [
        "## 2. How to run Ollama in Google Colab?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WPUNiHyCxekH"
      },
      "source": [
        "### 2.1 Ollama Installation\n",
        "For this, we simply go to the [Ollama downloads page](https://ollama.com/download/linux) and select **Linux**. The command is as follows"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lwzxaz9WN8Sr"
      },
      "outputs": [],
      "source": [
        "!curl -fsSL https://ollama.com/install.sh | sh"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WFyDUt_nxekL"
      },
      "source": [
        "### 2.2 Run 'ollama serve'\n",
        "If you run ollama serve, you will encounter the issue where you cannot execute subsequent cells and your script will remain stuck in that cell indefinitely. To resolve this, you simply need to run the following command:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "BQGW527tO15z"
      },
      "outputs": [],
      "source": [
        "!nohup ollama serve > ollama_serve.log 2>&1 &"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OK-OgAidxekM"
      },
      "source": [
        "After running this command, it is advisable to wait a reasonable amount of time for it to execute before running the next command, so you can add something like:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "YVaKWZCMO3b6"
      },
      "outputs": [],
      "source": [
        "import time\n",
        "time.sleep(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iCg-eX9axekM"
      },
      "source": [
        "### 2.3 Run 'ollama pull <model_name>'\n",
        "For this project we will use [Llama 3.2](https://ollama.com/library/llama3.2) the most recent release of **Meta** and specifically the **3B parameters** version. This project is also extensible to [Phi-3.5-mini](https://ollama.com/library/phi3.5) (the lightweight **Microsoft** model with high capabilities); you would only have to pull that other model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OXavgAvmO4z1"
      },
      "outputs": [],
      "source": [
        "!ollama pull llama3.2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3gLJHKGCxekN"
      },
      "source": [
        "## 3. Exploring LLMs\n",
        "Now that we have our LLM, it's time to test them with what will be our control question."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "f7Lw0Oh6O6QR"
      },
      "outputs": [],
      "source": [
        "test_message = \"¿Cuántos hijos tiene la señora vieja del cuento Algo muy grave va a suceder en este pueblo?\"\n",
        "# EN:\"How many children does the old woman in the story 'Something Very Serious Is Going to Happen in This Town' have?\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bMBRTRTGxekN"
      },
      "source": [
        "> 'Gabo' will be designed to function in Spanish, as it was Gabriel García Márquez's native language and his literary work is also in this language.\n",
        "\n",
        "The information is found at the beginning of [the story,](https://ciudadseva.com/texto/algo-muy-grave-va-a-suceder-en-este-pueblo/) so we expect it to be something that can be answered if it has the necessary information."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6Fu72N5yPAOS"
      },
      "outputs": [],
      "source": [
        "\"\"\"\n",
        "ES\n",
        "Fragmento inicial de 'Algo muy grave va a suceder en este pueblo' de Gabriel García Márquez.\n",
        "\"Imagínese usted un pueblo muy pequeño donde hay una señora vieja que tiene dos hijos, uno de 17 y una hija de 14... \"\n",
        "\n",
        "EN\n",
        "Initial excerpt from 'Something Very Serious Is Going to Happen in This Town' by Gabriel García Márquez:\n",
        "\"Imagine a very small town where there is an old woman who has two children, a 17-year-old son and a 14-year-old daughter...\"\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O-quS3HQxekN"
      },
      "source": [
        "Before we can invoke the LLM, we need to install LangChain. [1]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "T9jBMTndPEIp"
      },
      "outputs": [],
      "source": [
        "!pip install -qU langchain_community"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZxKmZgrl9Ok9"
      },
      "source": [
        "and LangChain's support to Ollama"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "vUrAvsSN3UKz"
      },
      "outputs": [],
      "source": [
        "!pip install -qU langchain-ollama"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "elWsBonGxekN"
      },
      "source": [
        "Now we create the model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "_SDD9Fq6PGId"
      },
      "outputs": [],
      "source": [
        "from langchain_ollama import OllamaLLM\n",
        "\n",
        "llm_llama = OllamaLLM(model=\"llama3.2\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dia9tuMxxekO"
      },
      "source": [
        "Invoke Llama 3.2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        },
        "id": "Gp8wJRpjPJY2",
        "outputId": "2cd9e3af-e621-4dc4-b46f-377e05aafb5c"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'No tengo información sobre un cuento llamado \"Algo muy grave va a suceder en este pueblo\" que incluya a una \"señora vieja\". Es posible que el cuento sea de autor desconocido o que no esté ampliamente conocido.\\n\\nSin embargo, puedo sugerirte algunas posibles opciones para encontrar la respuesta a tu pregunta:\\n\\n1. **Buscar en línea**: Puedes buscar el título del cuento en motores de búsqueda como Google para ver si se puede encontrar información sobre él.\\n2. **Consultar una base de datos de literatura**: Si conoces el autor o la fecha de publicación del cuento, puedes consultar bases de datos de literatura en línea, como Goodreads o Literary Maps, para ver si se puede encontrar información sobre él.\\n3. **Preguntar a un experto**: Si eres estudiante de literatura o tienes interés en el tema, puedes preguntar a un experto en la materia o buscar recursos educativos que puedan ayudarte a encontrar la respuesta a tu pregunta.\\n\\nSi tienes más información sobre el cuento, como el autor o la fecha de publicación, estaré encantado de ayudarte a encontrar la respuesta.'"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "llm_llama.invoke(test_message)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9bzWsfsrxekO"
      },
      "source": [
        "> At this stage, the model is not expected to be able to answer the question correctly, and they might even hallucinate when trying to give an answer. To solve this problem, we will start building our **RAG** in the next section."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hhBhALn_xekO"
      },
      "source": [
        "## 4. Data Extraction and Preparation\n",
        "To collect the information that our **RAG** will use, we will perform **Web Scraping** of the section dedicated to [Gabriel Garcia Marquez](https://ciudadseva.com/autor/gabriel-garcia-marquez/) in the **Ciudad Seva web site**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P7A-vRBNxekO"
      },
      "source": [
        "### 4.1 Web Scraping and Chunking\n",
        "The first step is to install **Beautiful Soup** so that LangChain's **WebBaseLoader** works correctly."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "gU3MDwCmPMNF"
      },
      "outputs": [],
      "source": [
        "!pip install -qU beautifulsoup4"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KqKbW_iXxekO"
      },
      "source": [
        "The next step will be to save the list of sources we will extract from the website into a variable."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "_iaDs740PNrl"
      },
      "outputs": [],
      "source": [
        "base_urls = [\"https://ciudadseva.com/autor/gabriel-garcia-marquez/cuentos/\",\n",
        "             \"https://ciudadseva.com/autor/gabriel-garcia-marquez/opiniones/\",\n",
        "             \"https://ciudadseva.com/autor/gabriel-garcia-marquez/otrostextos/\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k7iYMW8AxekO"
      },
      "source": [
        "Now we will create a function to collect all the links that lead to the texts. If we look at the HTML structure, we will notice that the information we're looking for is inside an `<article>` element with the class `status-publish`. Then, we simply extract the `href` attributes from the `<li>` elements inside the `<a>` tags."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yXzAV9rqPPD9"
      },
      "outputs": [],
      "source": [
        "from langchain.document_loaders import WebBaseLoader\n",
        "\n",
        "def get_urls(url):\n",
        "    article = WebBaseLoader(url).scrape().find(\"article\", \"status-publish\")\n",
        "    lis = article.find_all(\"li\", \"text-center\")\n",
        "    return [li.find(\"a\").get(\"href\") for li in lis]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tVXkLFPGxekP"
      },
      "source": [
        "Let's see how many texts by the writer we can gather."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gAPmhZpWPRZt",
        "outputId": "c9978f79-64cc-4c80-9d52-16adb8f96cff"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "51"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "gabo_urls = []\n",
        "\n",
        "for base_url in base_urls:\n",
        "    gabo_urls.extend(get_urls(base_url))\n",
        "\n",
        "len(gabo_urls)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "upah3a1WxekQ"
      },
      "source": [
        "Now that we have the URLs of the texts to feed our **RAG**, we just need to perform web scraping directly from the content of the stories. For that, we will build a function that follows a logic very similar to the previous function, which will initially give us the **raw text**, along with the **reference information** about what we are obtaining (the information found in `<header>`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "FUI5q2A2Pc7R"
      },
      "outputs": [],
      "source": [
        "def ciudad_seva_loader(url):\n",
        "    article = WebBaseLoader(url).scrape().find(\"article\", \"status-publish\")\n",
        "    title = \" \".join(article.find(\"header\").get_text().split())\n",
        "    article.find(\"header\").decompose()\n",
        "    texts = (\" \".join(article.get_text().split())).split(\". \")\n",
        "    return [f\"Fragmento {i+1}/{len(texts)} de '{title}': '{text}'\" for i, text in enumerate(texts)]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UdD3qbqAxekQ"
      },
      "source": [
        "There are indeed many ways to perform chunking, several of which are discussed in **\"5 Levels of Text Splitting\"** [2]. The most interesting idea for me about how to split texts, and what I believe fits best in this project, is **Semantic Splitting**. So, following that idea, we will ensure that the function divides all the texts by their periods, thus generating **semantic fragments in Spanish**.\n",
        "\n",
        "> Tests were performed on the **Semantic Similarity** [3] offered by **Langchain**, but the results were worse. In this case, there is no need to do something extremely sophisticated, when the simplest and practically obvious solution is the best."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uCot1ZcDxekR"
      },
      "source": [
        "### 4.2 Embedding Model: Nomic\n",
        "I ran several tests with different **embedding models**, including **LLama 3.1** and **Phi 3.5**, but it wasn't until I used `nomic-embed-text` that I saw significantly better results. So, this is the embedding model we'll use. Now let's pull with Ollama from [Nomic's embedding model](https://ollama.com/library/nomic-embed-text)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MvixQQrFPiwL"
      },
      "outputs": [],
      "source": [
        "!ollama pull nomic-embed-text"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lD1i6Vt_xekS"
      },
      "source": [
        "We're going to create our model so we can later use it in **Chroma**, our vector database."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "zMqnSIXOPjoZ"
      },
      "outputs": [],
      "source": [
        "from langchain_ollama import OllamaEmbeddings\n",
        "\n",
        "nomic_ollama_embeddings = OllamaEmbeddings(model=\"nomic-embed-text\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Npt8b7AmxekX"
      },
      "source": [
        "## 5. Storing in the Vector Database\n",
        "**Chroma** is our chosen vector database. With the help of our embedding model provided by **Nomic**, we will store all the fragments generated from the texts, so that later we can query them and make them part of our context for each query to the **LLMs**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6CK8SYRXxekX"
      },
      "source": [
        "### 5.1 Making Chroma Persistent\n",
        "Here we have to think **one step ahead in time**, so we assume that chroma is already persistent, which means that it **exists in a directory**. If we don't do this, what will happen every time we run this **Python Notebook**, is that we will add repeated strings over and over again to the vector database. So it is a good practice to **reset Chroma** and in case it does not exist, it will be created and **simply remain empty**. [4]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "SGbgWAwsPlPY"
      },
      "outputs": [],
      "source": [
        "!pip install -qU chromadb langchain-chroma"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4jJRo6COxekX"
      },
      "source": [
        "We will create a function that will be specifically in charge of resetting the collection."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "qjHokMwlPn7c"
      },
      "outputs": [],
      "source": [
        "from langchain_chroma import Chroma\n",
        "\n",
        "def reset_collection(collection_name, persist_directory):\n",
        "    Chroma(\n",
        "        collection_name=collection_name,\n",
        "\t\tembedding_function=nomic_ollama_embeddings,\n",
        "\t\tpersist_directory=persist_directory\n",
        "\t).delete_collection()\n",
        "\n",
        "reset_collection(\"gabo_rag\", \"chroma\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OeAoGhoNxekX"
      },
      "source": [
        "### 5.2 Adding Documents to Chroma\n",
        "We may think that it is enough to just pass it all the text and it will store it completely, but that approach is inefficient and contradictory to the idea of RAG; that is why a whole section was dedicated to Chunking before."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N6pHyfQePp3F",
        "outputId": "d9deb890-84f9-445d-9cff-32e6abf01d21"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "5908"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "count = 0\n",
        "\n",
        "for gabo_url in gabo_urls:\n",
        "    texts = ciudad_seva_loader(gabo_url)\n",
        "    Chroma.from_texts(texts=texts, collection_name=\"gabo_rag\", embedding=nomic_ollama_embeddings, persist_directory=\"chroma\")\n",
        "    count += len(texts)\n",
        "\n",
        "count"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8l40Rm8uxekY"
      },
      "source": [
        "Let's verify that all fragments were saved correctly in Chroma"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lYluu0-ZPseN",
        "outputId": "1cbb7a25-51e9-45b5-a96b-1afcfbf8d090"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "5908"
            ]
          },
          "execution_count": 21,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "vector_store = Chroma(collection_name=\"gabo_rag\", embedding_function=nomic_ollama_embeddings, persist_directory=\"chroma\")\n",
        "\n",
        "len(vector_store.get()[\"ids\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bwPPCBmbxekY"
      },
      "source": [
        "> Here we are accessing the persistent data, not the in-memory data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CueXp3UzxekY"
      },
      "source": [
        "## 6. Use a Vectorstore as a Retriever\n",
        "A retriever is an **interface** that specializes in retrieving information from an **unstructured query**. Let's test the work we did, we will use the same `test_message` as before and see if the retriever can return the **specific fragment** of the text that has the answer (the one quoted in section [3. Exploring LLMs](#3-exploring-llms))."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PTJV3LclPuSd",
        "outputId": "f09dc22a-068e-4ae5-fdc0-81f654526c8d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Fragmento 2/40 de 'Algo muy grave va a suceder en este pueblo [Cuento - Texto completo.] Gabriel García Márquez:\n",
            "Imagínese usted un pueblo muy pequeño donde hay una señora vieja que tiene dos hijos, uno de 17 y una hija de 14'\n"
          ]
        }
      ],
      "source": [
        "retriever = vector_store.as_retriever(search_kwargs={\"k\": 1})\n",
        "\n",
        "docs = retriever.invoke(test_message)\n",
        "\n",
        "for doc in docs:\n",
        "    title, article = doc.page_content.split(\"': '\")\n",
        "    print(f\"\\n{title}:\\n{article}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "34VWapD5xekY"
      },
      "source": [
        "By default `Chroma.as_retriever()` will search for the most similar documents and `search_kwargs={”k“: 1}` indicates that we want to limit the output to **1**. [4]\n",
        "\n",
        "> We can see that the document returned to us was the **exact excerpt** that gives the **appropriate context** of our query. So the built retriever is **working correctly.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zQjlQfMbxekZ"
      },
      "source": [
        "## 7. RAG (Retrieval-Augmented Generation)\n",
        "To better integrate our context to the query, we will make use of a **template** that will help us set up the behavior of the **RAG** and give it indications on how to answer."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "t2bQi1vvxekZ"
      },
      "outputs": [],
      "source": [
        "from langchain_core.prompts import PromptTemplate\n",
        "\n",
        "template = \"\"\"\n",
        "Eres 'Gabo', un asistente especializado en la obra de Gabriel García Márquez. Fuiste creado en conmemoracion del decimo aniversario de su muerte.\n",
        "Responde de manera concisa, precisa y relevante a la pregunta que se te ha hecho, sin desviarte del tema y limitando tu respuesta a un parrafo.\n",
        "Cada consulta que recibas puede estar acompañada de un contexto que corresponde a fragmentos de cuentos, opiniones y otros textos del escritor.\n",
        "\n",
        "Contexto: {context}\n",
        "\n",
        "Pregunta: {input}\n",
        "\n",
        "Respuesta:\n",
        "\"\"\"\n",
        "\n",
        "custom_rag_prompt = PromptTemplate.from_template(template)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cY198Ib8xekZ"
      },
      "source": [
        "**LangChain** tells us how to use `create_stuff_documents_chain()` to integrate **Llama 3.2** and our **custom prompt**. Then we just need to use `create_retrieval_chain()` to automatically pass to the **LLM** our input along with the context and fill it in the template. [5]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "yxOFFd0-B5dk"
      },
      "outputs": [],
      "source": [
        "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
        "from langchain.chains import create_retrieval_chain\n",
        "\n",
        "question_answer_chain = create_stuff_documents_chain(llm_llama, custom_rag_prompt)\n",
        "rag_chain = create_retrieval_chain(retriever, question_answer_chain)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nuesp6DhxekZ"
      },
      "source": [
        "Now let's test with our first control question, which allows us to check if the **LLM** is aware of his or her **new identity.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M7W_Vu8CP-vU",
        "outputId": "469e9339-971f-48ef-b1d1-58c08ee9070b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "ANSWER: Soy Gabo, un asistente especializado en la obra de Gabriel García Márquez. Fui creado en conmemoración del decimo aniversario de su muerte, como un homenaje a su legado literario y una forma de preservar su memoria para futuras generaciones. Mi nombre es una referencia a Gabriel García Márquez, pero también un apodo que me ha sido otorgado por aquellos que buscan información sobre su vida y obra.\n",
            "CONTEXT: Fragmento 62/179 de 'Diecisiete ingleses envenenados [Cuento - Texto completo.] Gabriel García Márquez': 'Un maletero hermoso y amable se echó el baúl al hombro y se hizo cargo de ella'\n"
          ]
        }
      ],
      "source": [
        "response = rag_chain.invoke({\"input\": \"Hablame de quien eres\"})\n",
        "\n",
        "print(f\"\\nANSWER: {response['answer']}\\nCONTEXT: {response['context'][0].page_content}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bb-JGAWBxeka"
      },
      "source": [
        "Finally let's conclude with the question that **started all this**...."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8q7XXiewQCeg",
        "outputId": "ec075f96-0204-409e-fbd8-534964e2d8a7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "ANSWER: La señora vieja del cuento \"Algo muy grave va a suceder en este pueblo\" tiene dos hijos, un varón de 17 años y una hija de 14 años.\n",
            "CONTEXT: Fragmento 2/40 de 'Algo muy grave va a suceder en este pueblo [Cuento - Texto completo.] Gabriel García Márquez': 'Imagínese usted un pueblo muy pequeño donde hay una señora vieja que tiene dos hijos, uno de 17 y una hija de 14'\n"
          ]
        }
      ],
      "source": [
        "response = rag_chain.invoke({\"input\": test_message})\n",
        "\n",
        "print(f\"\\nANSWER: {response['answer']}\\nCONTEXT: {response['context'][0].page_content}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "meXs07uuxekb"
      },
      "source": [
        "## 8. References\n",
        "[1] **Ollama. (s. f.). ollama/docs/tutorials/langchainpy.md at main · ollama/ollama. GitHub.** https://github.com/ollama/ollama/blob/main/docs/tutorials/langchainpy.md\n",
        "\n",
        "[2] **FullStackRetrieval-Com. (s. f.). RetrievalTutorials/tutorials/LevelsOfTextSplitting/5_Levels_Of_Text_Splitting.ipynb at main · FullStackRetrieval-com/RetrievalTutorials. GitHub.** https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/tutorials/LevelsOfTextSplitting/5_Levels_Of_Text_Splitting.ipynb\n",
        "\n",
        "[3] **How to split text based on semantic similarity | 🦜️🔗 LangChain. (s. f.).** https://python.langchain.com/docs/how_to/semantic-chunker/\n",
        "\n",
        "[4] **Chroma — 🦜🔗 LangChain  documentation. (s. f.).** https://python.langchain.com/v0.2/api_reference/chroma/vectorstores/langchain_chroma.vectorstores.Chroma.html\n",
        "\n",
        "[5] **Build a Retrieval Augmented Generation (RAG) App | 🦜️🔗 LangChain. (s. f.).** https://python.langchain.com/docs/tutorials/rag/\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}