doc_qa/openai_embeddings.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "81babb43",
   "metadata": {},
   "source": [
    "This notebook investigates the behavior of OpenAI text embeddings. It attempts to explore some of the strengths and weaknesses of embeddings, and answer questions like \"What makes a good embedding query for information retrieval?\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "516a775d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from doc_qa.embed import generate_pdf_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "aba98bb5",
   "metadata": {},
   "outputs": [],
   "source": [
    "PDF_FILE = \"test_docs/2304.02643.pdf\" # The Segment Anything Model (SAM) paper."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "be33fc19",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found existing document embeddings at 'output/embeddings/2304.02643'. Not re-generating.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using embedded DuckDB with persistence: data will be stored in: output/embeddings/2304.02643\n"
     ]
    }
   ],
   "source": [
    "vector_store = generate_pdf_embeddings(PDF_FILE)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b65db113",
   "metadata": {},
   "source": [
    "# False Positive Example (Authors)\n",
    "\n",
    "In this example, attempting to query for the authors of a paper returns authors of the referenced papers instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "40bc75da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='Robert Kuo for help with data annotation platform. We\\nthank Allen Goodman and Bram Wasti for help in optimiz-\\ning web-version of our model. Finally, we thank Morteza\\nBehrooz, Ashley Gabriel, Ahuva Goldstand, Sumanth Gur-\\nram, Somya Jain, Devansh Kukreja, Joshua Lane, Lilian\\nLuong, Mallika Malhotra, William Ngan, Omkar Parkhi,\\nNikhil Raina, Dirk Rowe, Neil Sejoor, Vanessa Stark, Bala\\nVaradarajan, and Zachary Winstrom for their help in mak-\\ning the demo, dataset viewer, and other assets and tooling.\\n12', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 11}),\n",
       " Document(page_content='garajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona\\nRyan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhong-\\ncong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Car-\\ntillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli,\\nChristoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Chris-\\ntian Fuegen, Abrham Gebreselasie, Cristina Gonzalez, James Hillis,\\nXuhua Huang, Yifei Huang, Wenqi Jia, Weslie Khoo, Jachym Ko-\\nlar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li,\\nYanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Mod-\\nhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will\\nPrice, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran\\nSomasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao,\\nMinh V o, Yuchen Wang, Xindi Wu, Takuma Yagi, Yunyi Zhu,\\nPablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria\\nFarinella, Bernard Ghanem, Vamsi Krishna Ithapu, C. V . Jawahar,\\nHanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 13}),\n",
       " Document(page_content='Straehle, Bernhard X. Kausler, Carsten Haubold, Martin Schiegg,\\nJanez Ales, Thorsten Beier, Markus Rudy, Kemal Eren, Jaime I.\\nCervantes, Buote Xu, Fynn Beuttenmueller, Adrian Wolny, Chong\\nZhang, Ullrich Koethe, Fred A. Hamprecht, and Anna Kreshuk.\\nilastik: interactive machine learning for (bio)image analysis. Na-\\nture Methods , 2019. 12\\n[8] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman,\\nSimran Arora, Sydney von Arx, Michael S Bernstein, Jeannette\\nBohg, Antoine Bosselut, Emma Brunskill, et al. On the opportu-\\nnities and risks of foundation models. arXiv:2108.07258 , 2021. 1,\\n12\\n[9] Gustav Bredell, Christine Tanner, and Ender Konukoglu. Iterative\\ninteraction training for segmentation editing networks. MICCAI ,\\n2018. 17\\n[10] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah,\\nJared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav\\nShyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel\\nHerbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child,', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 12}),\n",
       " Document(page_content='indoor scene understanding. ICCV , 2021. 9, 19, 20\\n[87] Candice Schumann, Susanna Ricco, Utsav Prabhu, Vittorio Ferrari,\\nand Caroline Pantofaru. A step toward more inclusive people anno-\\ntations for fairness. Proceedings of the 2021 AAAI/ACM Conference\\non AI, Ethics, and Society , 2021. 8, 19\\n[88] Seﬁk Ilkin Serengil and Alper Ozpinar. LightFace: A hybrid deep\\nface recognition framework. ASYU , 2020. 26\\n[89] Seﬁk Ilkin Serengil and Alper Ozpinar. HyperExtended LightFace:\\nA facial attribute analysis framework. ICEET , 2021. 26\\n[90] Jamie Shotton, John Winn, Carsten Rother, and Antonio Crimin-\\nisi. TextonBoost: Joint appearance, shape and context modeling for\\nmulit-class object recognition and segmentation. ECCV , 2006. 4\\n[91] Corey Snyder and Minh Do. STREETS: A novel camera network\\ndataset for trafﬁc ﬂow. NeurIPS , 2019. 9, 20\\n[92] Konstantin Soﬁiuk, Ilya A Petrov, and Anton Konushin. Reviving\\niterative training with mask guidance for interactive segmentation.', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 14})]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vector_store.similarity_search(\"authors\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "252285be",
   "metadata": {},
   "source": [
    "Posing the query as a question did not help:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e8966c6a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='garajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona\\nRyan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhong-\\ncong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Car-\\ntillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli,\\nChristoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Chris-\\ntian Fuegen, Abrham Gebreselasie, Cristina Gonzalez, James Hillis,\\nXuhua Huang, Yifei Huang, Wenqi Jia, Weslie Khoo, Jachym Ko-\\nlar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li,\\nYanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Mod-\\nhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will\\nPrice, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran\\nSomasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao,\\nMinh V o, Yuchen Wang, Xindi Wu, Takuma Yagi, Yunyi Zhu,\\nPablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria\\nFarinella, Bernard Ghanem, Vamsi Krishna Ithapu, C. V . Jawahar,\\nHanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 13}),\n",
       " Document(page_content='Robert Kuo for help with data annotation platform. We\\nthank Allen Goodman and Bram Wasti for help in optimiz-\\ning web-version of our model. Finally, we thank Morteza\\nBehrooz, Ashley Gabriel, Ahuva Goldstand, Sumanth Gur-\\nram, Somya Jain, Devansh Kukreja, Joshua Lane, Lilian\\nLuong, Mallika Malhotra, William Ngan, Omkar Parkhi,\\nNikhil Raina, Dirk Rowe, Neil Sejoor, Vanessa Stark, Bala\\nVaradarajan, and Zachary Winstrom for their help in mak-\\ning the demo, dataset viewer, and other assets and tooling.\\n12', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 11}),\n",
       " Document(page_content='Straehle, Bernhard X. Kausler, Carsten Haubold, Martin Schiegg,\\nJanez Ales, Thorsten Beier, Markus Rudy, Kemal Eren, Jaime I.\\nCervantes, Buote Xu, Fynn Beuttenmueller, Adrian Wolny, Chong\\nZhang, Ullrich Koethe, Fred A. Hamprecht, and Anna Kreshuk.\\nilastik: interactive machine learning for (bio)image analysis. Na-\\nture Methods , 2019. 12\\n[8] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman,\\nSimran Arora, Sydney von Arx, Michael S Bernstein, Jeannette\\nBohg, Antoine Bosselut, Emma Brunskill, et al. On the opportu-\\nnities and risks of foundation models. arXiv:2108.07258 , 2021. 1,\\n12\\n[9] Gustav Bredell, Christine Tanner, and Ender Konukoglu. Iterative\\ninteraction training for segmentation editing networks. MICCAI ,\\n2018. 17\\n[10] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah,\\nJared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav\\nShyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel\\nHerbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child,', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 12}),\n",
       " Document(page_content='[76] Dim P Papadopoulos, Jasper RR Uijlings, Frank Keller, and Vittorio\\nFerrari. Extreme clicking for efﬁcient object annotation. ICCV ,\\n2017. 6\\n[77] David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-\\nMiquel Munguia, Daniel Rothchild, David So, Maud Texier, and\\nJeff Dean. Carbon emissions and large neural network training.\\narXiv:2104.10350 , 2021. 28\\n[78] Matthew E Peters, Waleed Ammar, Chandra Bhagavatula, and Rus-\\nsell Power. Semi-supervised sequence tagging with bidirectional\\nlanguage models. Proceedings of the 55th Annual Meeting of the\\nAssociation for Computational Linguistics , 2017. 18\\n[79] Mengyang Pu, Yaping Huang, Yuming Liu, Qingji Guan, and\\nHaibin Ling. EDTER: Edge detection with transformer. CVPR ,\\n2022. 10\\n[80] Mattia Pugliatti and Francesco Topputo. DOORS: Dataset fOr\\nbOuldeRs Segmentation. Zenodo , 2022. 9, 20\\n[81] Jiyang Qi, Yan Gao, Yao Hu, Xinggang Wang, Xiaoyu Liu, Xiang\\nBai, Serge Belongie, Alan Yuille, Philip Torr, and Song Bai. Oc-', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 14})]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vector_store.similarity_search(\"Who are the authors of this paper?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "040fafa1",
   "metadata": {},
   "source": [
    "# Comparing Query Styles\n",
    "\n",
    "In this section, we test different query styles for information retrieval via embedding search.\n",
    "\n",
    "The queries are designed to retrieve the size of the SA-1B dataset. (The correct answer is 11M images.)\n",
    "\n",
    "All of the query styles succesfully retrieved documents that contained the correct answer. A much more thorough analysis would be required to draw any meaningful conclusions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af2446a2",
   "metadata": {},
   "source": [
    "## Search Term\n",
    "\n",
    "Use a short search term to find related text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6ffeb9d7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='in our dataset, producing a total of 1.1B high-quality masks.\\nWe describe and analyze the resulting dataset, SA-1B, next.\\nFigure 5: Image-size normalized mask center distributions.\\n5. Segment Anything Dataset\\nOur dataset, SA-1B, consists of 11M diverse, high-\\nresolution, licensed, and privacy protecting images and\\n1.1B high-quality segmentation masks collected with our\\ndata engine. We compare SA-1B with existing datasets\\nand analyze mask quality and properties. We are releasing\\nSA-1B to aid future development of foundation models for\\ncomputer vision. We note that SA-1B will be released un-\\nder a favorable license agreement for certain research uses\\nand with protections for researchers.\\nImages . We licensed a new set of 11M images from a\\nprovider that works directly with photographers. These im-\\nages are high resolution (3300 \\x024950 pixels on average),\\nand the resulting data size can present accessibility and stor-\\nage challenges. Therefore, we are releasing downsampled', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 5}),\n",
       " Document(page_content='of the 7 selected datasets.\\n23', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 22}),\n",
       " Document(page_content='For single point experiments, 1000 masks per dataset\\nwere selected randomly from the same subsets used for\\nbenchmarking zero-shot interactive segmentation (see §D.1\\nfor details on these subsets). The model input was the cen-\\ntermost point, calculated as the largest value of the distance\\ntransform from the edge of the mask. For instance seg-\\nmentation experiments, 1000 masks were selected from the\\nLVIS v1 validation set, and the model input was the LVIS\\nground truth box. In all experiments, masks with a size\\nsmaller than 242pixels were excluded from sampling, to\\nprevent showing raters a mask that was too small to judge\\naccurately. For both memory and display reasons, large im-\\nages were rescaled to have a max side-length of 2000 before\\npredicting a mask. In all experiments, the same inputs were\\nfed to each model to produce a predicted mask.\\nFor comparison, the ground truth masks from each\\ndataset were also submitted for rating. For single-point', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 22}),\n",
       " Document(page_content='and edges)? Please provide a description. All of the instances in the dataset\\nare photos. The photos vary in subject matter; common themes of the photo\\ninclude: locations, objects, scenes. All of the photos are distinct, however\\nthere are some sets of photos that were taken of the same subject matter.\\n2.How many instances are there in total (of each type, if appropriate)? There\\nare 11 million images.\\n3.Does the dataset contain all possible instances or is it a sample (not nec-\\nessarily random) of instances from a larger set? If the dataset is a sample,\\nthen what is the larger set? Is the sample representative of the larger set\\n(e.g., geographic coverage)? If so, please describe how this representa-\\ntiveness was validated/veriﬁed. If it is not representative of the larger set,\\nplease describe why not ( e.g., to cover a more diverse range of instances,\\nbecause instances were withheld or unavailable). The dataset is composed', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 24})]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vector_store.similarity_search(\"Dataset size\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d03d82d",
   "metadata": {},
   "source": [
    "# Full Question\n",
    "\n",
    "Use the question that we are trying to answer as the vector embedding query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "811a7a99",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='of the 7 selected datasets.\\n23', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 22}),\n",
       " Document(page_content='and edges)? Please provide a description. All of the instances in the dataset\\nare photos. The photos vary in subject matter; common themes of the photo\\ninclude: locations, objects, scenes. All of the photos are distinct, however\\nthere are some sets of photos that were taken of the same subject matter.\\n2.How many instances are there in total (of each type, if appropriate)? There\\nare 11 million images.\\n3.Does the dataset contain all possible instances or is it a sample (not nec-\\nessarily random) of instances from a larger set? If the dataset is a sample,\\nthen what is the larger set? Is the sample representative of the larger set\\n(e.g., geographic coverage)? If so, please describe how this representa-\\ntiveness was validated/veriﬁed. If it is not representative of the larger set,\\nplease describe why not ( e.g., to cover a more diverse range of instances,\\nbecause instances were withheld or unavailable). The dataset is composed', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 24}),\n",
       " Document(page_content='in our dataset, producing a total of 1.1B high-quality masks.\\nWe describe and analyze the resulting dataset, SA-1B, next.\\nFigure 5: Image-size normalized mask center distributions.\\n5. Segment Anything Dataset\\nOur dataset, SA-1B, consists of 11M diverse, high-\\nresolution, licensed, and privacy protecting images and\\n1.1B high-quality segmentation masks collected with our\\ndata engine. We compare SA-1B with existing datasets\\nand analyze mask quality and properties. We are releasing\\nSA-1B to aid future development of foundation models for\\ncomputer vision. We note that SA-1B will be released un-\\nder a favorable license agreement for certain research uses\\nand with protections for researchers.\\nImages . We licensed a new set of 11M images from a\\nprovider that works directly with photographers. These im-\\nages are high resolution (3300 \\x024950 pixels on average),\\nand the resulting data size can present accessibility and stor-\\nage challenges. Therefore, we are releasing downsampled', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 5}),\n",
       " Document(page_content='Redundancies: While no two images are the same, there are instances of\\nimages of the same subject taken close together in time.\\n9.Is the dataset self-contained, or does it link to or otherwise rely on external\\nresources ( e.g., websites, tweets, other datasets)? If it links to or relies on\\nexternal resources, a) are there guarantees that they will exist, and remain\\nconstant, over time; b) are there ofﬁcial archival versions of the complete\\ndataset ( i.e., including the external resources as they existed at the time\\nthe dataset was created); c) are there any restrictions ( e.g., licenses, fees)\\nassociated with any of the external resources that might apply to a dataset\\nconsumer? Please provide descriptions of all external resources and any\\nrestrictions associated with them, as well as links or other access points, as\\nappropriate. The dataset is self-contained.\\n10. Does the dataset contain data that might be considered conﬁdential ( e.g.,', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 24})]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vector_store.similarity_search(\"What is the dataset size?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c8053f3",
   "metadata": {},
   "source": [
    "## Imaginary Statement\n",
    "\n",
    "Write a statement that resembles the statement that you are *hoping* to find in the doc store. Use the imaginary statement for the embedding search. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "32f7e099",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='images with their shortest side set to 1500 pixels. Even af-\\nter downsampling, our images are signiﬁcantly higher reso-\\nlution than many existing vision datasets ( e.g., COCO [66]\\nimages are \\x18480\\x02640 pixels). Note that most models today\\noperate on much lower resolution inputs. Faces and vehicle\\nlicense plates have been blurred in the released images.\\nMasks . Our data engine produced 1.1B masks, 99.1% of\\nwhich were generated fully automatically. Therefore, the\\nquality of the automatic masks is centrally important. We\\ncompare them directly to professional annotations and look\\nat how various mask properties compare to prominent seg-\\nmentation datasets. Our main conclusion, as borne out in\\nthe analysis below and the experiments in §7, is that our\\nautomatic masks are high quality and effective for training\\nmodels. Motivated by these ﬁndings, SA-1B only includes\\nautomatically generated masks.\\nMask quality. To estimate mask quality, we randomly sam-', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 5}),\n",
       " Document(page_content='in our dataset, producing a total of 1.1B high-quality masks.\\nWe describe and analyze the resulting dataset, SA-1B, next.\\nFigure 5: Image-size normalized mask center distributions.\\n5. Segment Anything Dataset\\nOur dataset, SA-1B, consists of 11M diverse, high-\\nresolution, licensed, and privacy protecting images and\\n1.1B high-quality segmentation masks collected with our\\ndata engine. We compare SA-1B with existing datasets\\nand analyze mask quality and properties. We are releasing\\nSA-1B to aid future development of foundation models for\\ncomputer vision. We note that SA-1B will be released un-\\nder a favorable license agreement for certain research uses\\nand with protections for researchers.\\nImages . We licensed a new set of 11M images from a\\nprovider that works directly with photographers. These im-\\nages are high resolution (3300 \\x024950 pixels on average),\\nand the resulting data size can present accessibility and stor-\\nage challenges. Therefore, we are releasing downsampled', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 5}),\n",
       " Document(page_content='and edges)? Please provide a description. All of the instances in the dataset\\nare photos. The photos vary in subject matter; common themes of the photo\\ninclude: locations, objects, scenes. All of the photos are distinct, however\\nthere are some sets of photos that were taken of the same subject matter.\\n2.How many instances are there in total (of each type, if appropriate)? There\\nare 11 million images.\\n3.Does the dataset contain all possible instances or is it a sample (not nec-\\nessarily random) of instances from a larger set? If the dataset is a sample,\\nthen what is the larger set? Is the sample representative of the larger set\\n(e.g., geographic coverage)? If so, please describe how this representa-\\ntiveness was validated/veriﬁed. If it is not representative of the larger set,\\nplease describe why not ( e.g., to cover a more diverse range of instances,\\nbecause instances were withheld or unavailable). The dataset is composed', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 24}),\n",
       " Document(page_content='<50 masks\\n 50-100 masks\\n 100-200 masks\\n 200-300 masks\\n 300-400 masks\\n 400-500 masks\\n >500 masks\\nFigure 2: Example images with overlaid masks from our newly introduced dataset, SA-1B . SA-1B contains 11M diverse,\\nhigh-resolution, licensed, and privacy protecting images and 1.1B high-quality segmentation masks. These masks were\\nannotated fully automatically by SAM, and as we verify by human ratings and numerous experiments, are of high quality and\\ndiversity. We group images by number of masks per image for visualization (there are \\x18100 masks per image on average).\\n3', metadata={'source': 'test_docs/2304.02643.pdf', 'page': 2})]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vector_store.similarity_search(\"The size of the dataset that we introduced in this paper is __ images.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "896acfdb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}