demo_advanced_astradb_rag_using_llama_parser.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jaideep11061982/GenAINotebooks/blob/main/demo_advanced_astradb_rag_using_llama_parser.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x0QzIY094ciq"
      },
      "source": [
        "# Advanced RAG with LlamaParse + AstraDB\n",
        "\n",
        "This notebook shows you how to use LlamaParse with our advanced markdown ingestion and recursive retrieval algorithms to model tables/text within a document hierarchically. This lets you ask questions over both tables and text.\n",
        "\n",
        "This notebook is similar to our `demo_advanced.ipynb` notebook, with the main difference being AstraDB."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "K_NObl7D4cit"
      },
      "outputs": [],
      "source": [
        "%pip install llama-index\n",
        "%pip install llama-index-core==0.10.6.post1\n",
        "%pip install llama-index-embeddings-openai\n",
        "%pip install llama-index-postprocessor-flag-embedding-reranker\n",
        "%pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
        "%pip install llama-parse\n",
        "%pip install llama-index-vector-stores-astra-db"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rNJtyelX4civ"
      },
      "outputs": [],
      "source": [
        "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10q/uber_10q_march_2022.pdf' -O './uber_10q_march_2022.pdf'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MdjRlYRP4civ"
      },
      "source": [
        "Some OpenAI and LlamaParse details"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TdLLZ5JP4ciw"
      },
      "outputs": [],
      "source": [
        "# llama-parse is async-first, running the async code in a notebook requires the use of nest_asyncio\n",
        "import nest_asyncio\n",
        "\n",
        "nest_asyncio.apply()\n",
        "\n",
        "import os\n",
        "\n",
        "# API access to llama-cloud\n",
        "os.environ[\"LLAMA_CLOUD_API_KEY\"] = \"llx-\"\n",
        "\n",
        "# Using OpenAI API for embeddings/llms\n",
        "os.environ[\"OPENAI_API_KEY\"] = \"sk-\"\n",
        "\n",
        "ASTRA_API_ENDPOINT = \"<enter AstraDB endpoint>\"\n",
        "ASTRA_TOKEN = \"<enter your Astra DB Token>\"\n",
        "ASTRA_NAMESPACE = None  # or: \"my_keyspace\". Must exist on Astra already."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MM87huYu4ciw"
      },
      "outputs": [],
      "source": [
        "from llama_index.llms.openai import OpenAI\n",
        "from llama_index.embeddings.openai import OpenAIEmbedding\n",
        "from llama_index.core import VectorStoreIndex\n",
        "from llama_index.core import Settings\n",
        "\n",
        "embed_model = OpenAIEmbedding(model=\"text-embedding-3-small\")\n",
        "llm = OpenAI(model=\"gpt-3.5-turbo-0125\")\n",
        "\n",
        "Settings.llm = llm\n",
        "Settings.embed_model = embed_model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LOeBHnki4cix"
      },
      "source": [
        "## Using brand new `LlamaParse` PDF reader for PDF Parsing\n",
        "\n",
        "We feed the Uber March 2022 10QA into LlamaParse to get back the Markdown representation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jPMxflJv4cix",
        "outputId": "c920ef34-89e4-4def-9fdb-de5e755e4b92"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Started parsing the file under job_id c27837c3-ca0f-4c19-9c5b-205e44f7cc73\n"
          ]
        }
      ],
      "source": [
        "from llama_parse import LlamaParse\n",
        "\n",
        "documents = LlamaParse(result_type=\"markdown\").load_data(\"./uber_10q_march_2022.pdf\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PDj_OmxM4cix",
        "outputId": "f92827a6-4dc0-4086-8465-3e84e2488da4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "## UNITED STATES SECURITIES AND EXCHANGE COMMISSION\n",
            "\n",
            "Washington, D.C. 20549\n",
            "\n",
            "## FORM 10-Q\n",
            "\n",
            "(Mark One)\n",
            "\n",
            "☒ QUARTERLY REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934\n",
            "\n",
            "For the quarterly period ended March 31, 2022\n",
            "\n",
            "OR\n",
            "\n",
            "☐ TRANSITION REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934\n",
            "\n",
            "For the transition period from_____ to _____\n",
            "\n",
            "Commission File Number: 001-38902\n",
            "\n",
            "UBER TECHNOLOGIES, INC.\n",
            "\n",
            "(Exact name of registrant as specified in its charter)\n",
            "\n",
            "Not Applicable\n",
            "\n",
            "(Former name, former address and former fiscal year, if changed since last report)\n",
            "\n",
            "Delaware 45-2647441\n",
            "\n",
            "(State or other jurisdiction of incorporation or organization) (I.R.S. Employer Identification No.)\n",
            "\n",
            "1515 3rd Street\n",
            "\n",
            "San Francisco, California 94158\n",
            "\n",
            "(Address of principal executive offices, including zip code)\n",
            "\n",
            "(415) 612-8582\n",
            "\n",
            "(Registrant’s telephone number, including area code)\n",
            "\n",
            "Securities registered pursuant to Section 12(b) of the Act:\n",
            "\n",
            "|Title of each class|Trading Symbol(...\n"
          ]
        }
      ],
      "source": [
        "print(documents[0].text[:1000] + \"...\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nzAwfJ0d4ciy"
      },
      "source": [
        "## Setup AstraDB"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rOTO1AMo4ciy"
      },
      "outputs": [],
      "source": [
        "from llama_index.vector_stores.astra_db import AstraDBVectorStore\n",
        "\n",
        "# define two storage classes representing two collections (to compare advanced approach vs. baseline)\n",
        "\n",
        "astra_db_store_advanced = AstraDBVectorStore(\n",
        "    token=ASTRA_TOKEN,\n",
        "    api_endpoint=ASTRA_API_ENDPOINT,\n",
        "    namespace=ASTRA_NAMESPACE,\n",
        "    collection_name=\"astra_v_table_llamaparse_advanced\",\n",
        "    embedding_dimension=1536,\n",
        ")\n",
        "astra_db_store_base = AstraDBVectorStore(\n",
        "    token=ASTRA_TOKEN,\n",
        "    api_endpoint=ASTRA_API_ENDPOINT,\n",
        "    namespace=ASTRA_NAMESPACE,\n",
        "    collection_name=\"astra_v_table_llamaparse_base\",\n",
        "    embedding_dimension=1536,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cilTaa-r4ciz"
      },
      "source": [
        "## Index Data, Get Query Engine\n",
        "\n",
        "We compare two different retrieval/query engine strategies:\n",
        "1. Using raw Markdown text as nodes for building index and apply simple query engine for generating the results;\n",
        "2. Using `MarkdownElementNodeParser` for parsing the `LlamaParse` output Markdown results and building recursive retriever query engine for generation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "htFzFlvp4ciz"
      },
      "outputs": [],
      "source": [
        "from llama_index.core.node_parser import MarkdownElementNodeParser\n",
        "\n",
        "node_parser = MarkdownElementNodeParser(\n",
        "    llm=OpenAI(model=\"gpt-3.5-turbo-0125\"), num_workers=8\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fuH0zkHo4ciz"
      },
      "outputs": [],
      "source": [
        "nodes = node_parser.get_nodes_from_documents(documents)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jBXrsVRJ4ci0"
      },
      "outputs": [],
      "source": [
        "base_nodes, objects = node_parser.get_nodes_and_objects(nodes)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GoMAU0mX4ci0"
      },
      "outputs": [],
      "source": [
        "from llama_index.core import StorageContext\n",
        "\n",
        "storage_context_advanced = StorageContext.from_defaults(\n",
        "    vector_store=astra_db_store_advanced\n",
        ")\n",
        "storage_context_base = StorageContext.from_defaults(vector_store=astra_db_store_base)\n",
        "\n",
        "recursive_index = VectorStoreIndex(\n",
        "    nodes=base_nodes + objects, storage_context=storage_context_advanced\n",
        ")\n",
        "raw_index = VectorStoreIndex.from_documents(\n",
        "    documents, storage_context=storage_context_base\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "av4StlXJ4ci0"
      },
      "outputs": [],
      "source": [
        "from llama_index.postprocessor.flag_embedding_reranker import (\n",
        "    FlagEmbeddingReranker,\n",
        ")\n",
        "\n",
        "reranker = FlagEmbeddingReranker(\n",
        "    top_n=5,\n",
        "    model=\"BAAI/bge-reranker-large\",\n",
        ")\n",
        "\n",
        "recursive_query_engine = recursive_index.as_query_engine(\n",
        "    similarity_top_k=15, node_postprocessors=[reranker], verbose=True\n",
        ")\n",
        "\n",
        "raw_query_engine = raw_index.as_query_engine(\n",
        "    similarity_top_k=15, node_postprocessors=[reranker]\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tgCyW9EW4ci0"
      },
      "source": [
        "## Using `new LlamaParse` as pdf data parsing methods and retrieve tables with two different methods\n",
        "we compare base query engine vs recursive query engine with tables"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "haRGUkHL4ci0"
      },
      "source": [
        "### Table Query Task: Queries for Table Question Answering"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8x7viRLV4ci1",
        "outputId": "4e29d9bd-050c-4fef-9e4f-e14554692a27"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "***********Basic Query Engine***********\n",
            "$41\n",
            "\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_40_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_322_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_38_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_318_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_36_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_34_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_116_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_260_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\n",
            "\u001b[0m\n",
            "***********New LlamaParse+ Recursive Retriever Query Engine***********\n",
            "$22 for the period ending March 31, 2021 and $41 for the period ending March 31, 2022.\n"
          ]
        }
      ],
      "source": [
        "query = \"how is the Cash paid for Income taxes, net of refunds from Supplemental disclosures of cash flow information?\"\n",
        "\n",
        "response_1 = raw_query_engine.query(query)\n",
        "print(\"\\n***********New LlamaParse+ Basic Query Engine***********\")\n",
        "print(response_1)\n",
        "\n",
        "response_2 = recursive_query_engine.query(query)\n",
        "print(\"\\n***********New LlamaParse+ Recursive Retriever Query Engine***********\")\n",
        "print(response_2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TNHLWK-B4ci1"
      },
      
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bzxKSKML4ci1",
        "outputId": "33ca0d7c-f3f8-4e89-90fc-455b2fcd1d4f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "***********Basic Query Engine***********\n",
            "The change in free cash flow from the financial and operational highlights is a decrease from $(682) million in 2021 to $(47) million in 2022. The rate of change in free cash flow is an improvement of $635 million from 2021 to 2022.\n",
            "\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_318_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_322_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_34_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_36_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_40_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_302_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_236_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_274_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_232_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_38_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_234_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the change of free cash flow and what is the rate from the financial and operational highlights?\n",
            "\u001b[0m\n",
            "***********New LlamaParse+ Recursive Retriever Query Engine***********\n",
            "The change in free cash flow from the financial and operational highlights is an improvement from $(682) million in 2021 to $(47) million in 2022. This represents a significant positive change in free cash flow.\n"
          ]
        }
      ],
      "source": [
        "query = \"what is the change of free cash flow and what is the rate from the financial and operational highlights?\"\n",
        "\n",
        "response_1 = raw_query_engine.query(query)\n",
        "print(\"\\n***********New LlamaParse+ Basic Query Engine***********\")\n",
        "print(response_1)\n",
        "\n",
        "response_2 = recursive_query_engine.query(query)\n",
        "print(\"\\n***********New LlamaParse+ Recursive Retriever Query Engine***********\")\n",
        "print(response_2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dVseu1Fi4ci1"
      },
     
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mKtFKqjH4ci1",
        "outputId": "342f9927-b629-4b73-afb0-d9a657d9862d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "***********Basic Query Engine***********\n",
            "The net loss value attributable to Uber in the current year is $5.9 billion, which is an increase compared to the net loss of $108 million in the same period last year.\n",
            "\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_22_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the net loss value attributable to Uber compared to last year?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_228_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the net loss value attributable to Uber compared to last year?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_314_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the net loss value attributable to Uber compared to last year?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_224_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the net loss value attributable to Uber compared to last year?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_20_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the net loss value attributable to Uber compared to last year?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_16_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the net loss value attributable to Uber compared to last year?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_186_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query what is the net loss value attributable to Uber compared to last year?\n",
            "\u001b[0m\n",
            "***********New LlamaParse+ Recursive Retriever Query Engine***********\n",
            "The net loss attributable to Uber Technologies, Inc. for the first quarter of 2022 was $5,930 million, which is significantly higher compared to the net loss of $108 million for the same period in 2021.\n"
          ]
        }
      ],
      "source": [
        "query = \"what is the net loss value attributable to Uber compared to last year?\"\n",
        "\n",
        "response_1 = raw_query_engine.query(query)\n",
        "print(\"\\n***********New LlamaParse+ Basic Query Engine***********\")\n",
        "print(response_1)\n",
        "\n",
        "response_2 = recursive_query_engine.query(query)\n",
        "print(\"\\n***********New LlamaParse+ Recursive Retriever Query Engine***********\")\n",
        "print(response_2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WwhLzmCX4ci2"
      },
      
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w3ipuNd34ci2",
        "outputId": "c5224e3d-9099-428a-d09f-174b93ff2e72"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "***********Basic Query Engine***********\n",
            "Cash flows from investing activities were negative in both 2021 and 2022. In 2021, the net cash used in investing activities was $250 million, while in 2022, it was $135 million. These negative cash flows were primarily driven by purchases of property and equipment, acquisitions of businesses, and purchases of marketable and non-marketable securities.\n",
            "\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_322_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_318_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_40_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_34_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_36_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_264_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_248_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_120_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\u001b[1;3;38;2;11;159;203mRetrieval entering id_52c21485-37e1-4c04-b5af-e42659709ecc_38_table: TextNode\n",
            "\u001b[0m\u001b[1;3;38;2;237;90;200mRetrieving from object TextNode with query What were cash flows like from investing activities?\n",
            "\u001b[0m\n",
            "***********New LlamaParse+ Recursive Retriever Query Engine***********\n",
            "Cash flows from investing activities were as follows:\n",
            "- For the three months ended March 31, 2021, net cash used in investing activities was $(250) million.\n",
            "- For the three months ended March 31, 2022, net cash used in investing activities was $(135) million.\n"
          ]
        }
      ],
      "source": [
        "query = \"What were cash flows like from investing activities?\"\n",
        "\n",
        "response_1 = raw_query_engine.query(query)\n",
        "print(\"\\n***********New LlamaParse+ Basic Query Engine***********\")\n",
        "print(response_1)\n",
        "\n",
        "response_2 = recursive_query_engine.query(query)\n",
        "print(\"\\n***********New LlamaParse+ Recursive Retriever Query Engine***********\")\n",
        "print(response_2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5NhxpXNe4ci2"
      },
      
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "llama_parse",
      "language": "python",
      "name": "llama_parse"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}