src/generative_ai/retrieval_augmented_generation_poc.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "czxW4C_gf5Eo"
   },
   "source": [
    "# DATASCI 290 - GenAI - Assignment 5\n",
    "\n",
    "> Submitted by [Eshwaran Venkat](eshwaran@ischool.berkeley.edu)\n",
    "\n",
    "In Assignment 5 you will create and test a RAG system yourself, and write a corresponding business proposal.\n",
    "\n",
    "The overall scenarios is as follows:\n",
    "\n",
    "You work at a tech company that is looking for new ways to organize their question answering and search capabilities to accelerate both engineering activity and the marketing team. The company also wants to roll out new GenAI-based products, so a lot of the questions will center around Generative AI concepts. The company has about 300 engineers and a marketing staff of 40. Product releases are done quarterly.\n",
    "\n",
    "Your role is to implement and conduct a (mini-)POC helping the company to evaluate RAG capabilities for the improvement of their document search (and corresponding question answering), supporting particularly the engineering and marketing organizations. You will have a gold dataset with 'good' responses to questions from marketing and engineering teams. You need to develop metric(s) that help you to evaluate how well your RAG system performs relative to the gold data. You should work with the tunables of the setup (LLM, chunking, embeddings, ...) for your iterations.\n",
    "\n",
    "You will also need to write up your findings as a short proposal.\n",
    "\n",
    "(See instructions throughout this notebook.)\n",
    "\n",
    "So overall, the goals of this assignment is for you to:\n",
    "\n",
    "*  To implement a RAG system using LangChain\n",
    "*  Be able to formulate metric(s) that you may want to choose as your evaluation to what degree your system replicates gold answers (labeled data) that we will provide.\n",
    "* Try out various hyper-parameters and settings to see which configuration works the best (given your chosen metric)  \n",
    "* Write a comprehensive evaluation, which also includes risks and limitations (and a lot more)\n",
    "\n",
    "The notebook is organized as follows:\n",
    "\n",
    "1. Set-Up\n",
    "\n",
    "2. Base RAG components\n",
    "\n",
    "    We will provide a base LangChain-based framework for you to use for your RAG system. The components we’ll need include:  \n",
    "\n",
    "  2.1 Text Embeddings    \n",
    "  2.2 Text Chunking   \n",
    "  2.3 The Vector DB & Semantic Search  \n",
    "  2.4 The Language Model   \n",
    "  2.5 Testing the LLM in a LangChain Chain   \n",
    "  2.6. Setting up a simple RAG Chain     \n",
    "\n",
    "\n",
    "3. Using RAG  \n",
    "  3.1 Loading of Data  \n",
    "  3.2 Test Queries\n",
    "\n",
    "\n",
    "4.  Evaluations\n",
    "\n",
    "  Here, you will conduct your evaluations\n",
    "\n",
    "\n",
    "5. Final Results\n",
    "\n",
    "  In this section you provide the RAG answers to the test questions\n",
    "\n",
    "RULES:  \n",
    "\n",
    "* You can only use the language models specified here  \n",
    "* You can only use the embedding methods we discuss  \n",
    "* You can only use the focuments we provide. And they all must be in your store   \n",
    "* Apart from the provided specifications, some of the things you can freely experiment with include chunk sizes, prompts, etc.\n",
    "\n",
    "\n",
    "**To run this notebook** you should copy it to your personal Colab Pro Google account by uploading it into your Google Drive. From there you can open it as a Colab notebook and run it.  Note it needs a T4 GPU to run.  You may be able to run it in a free Colab notebook.\n",
    "\n",
    "NOTES:\n",
    "* The Open Source Model is not trained for safety. So unsafe answers could be returned.\n",
    "\n",
    "Reference Blogs:\n",
    "\n",
    "* [Understanding BLEU and ROUGE](https://medium.com/@sthanikamsanthosh1994/understanding-bleu-and-rouge-score-for-nlp-evaluation-1ab334ecadcb)\n",
    "* [Document Retrieval - ChatGPT RAG](https://medium.com/@filipkny/chatgpt-for-document-retrieval-langchain-rag-with-code-0c68ebb19c9c)\n",
    "* [Langchain RAG - Bible](https://medium.com/@mosesdaudu001/indexing-and-querying-the-holy-bible-with-rag-langchain-gcp-postgresql-hnsw-and-vertex-ai-for-92a8f49ff613)\n",
    "\n",
    "\n",
    "Let's begin!\n",
    "\n",
    "---\n",
    "\n",
    "##1. Setup\n",
    "\n",
    "We will first install a number of libraries and import what we will need.\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Colab Setup\n",
    "# For google colab environment variables\n",
    "from google.colab import userdata\n",
    "\n",
    "# For temporary checkpoints of data and resuming\n",
    "from google.colab import drive\n",
    "\n",
    "drive.mount(\"/content/gdrive\")"
   ],
   "metadata": {
    "id": "JtKgvJTqre-b",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "20bf26b8-0a12-4c91-c785-50d6a28cfadb"
   },
   "execution_count": 1,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Mounted at /content/gdrive\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "WIlhfQj-KUlZ"
   },
   "outputs": [],
   "source": [
    "# @title Package Installation\n",
    "%%capture\n",
    "!pip -q install git+https://github.com/huggingface/transformers\n",
    "!pip install -q datasets loralib sentencepiece\n",
    "!pip -q install bitsandbytes accelerate\n",
    "!pip -q install langchain\n",
    "!pip install einops\n",
    "!pip install faiss-gpu\n",
    "!pip install --upgrade --quiet  langchain-community chromadb bs4 qdrant-client\n",
    "!pip install langchainhub\n",
    "!pip install joblib\n",
    "!pip install evaluate\n",
    "!pip install rouge_score bert_score\n",
    "\n",
    "!pip install --upgrade --quiet  wikipedia\n",
    "!pip install --upgrade --quiet  arxiv\n",
    "!pip install --upgrade --quiet  pymupdf\n",
    "!pip install -U langchain-cohere\n",
    "\n",
    "!pip install xmltodict\n",
    "\n",
    "!pip install cohere"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "3NjcvYABKieZ"
   },
   "outputs": [],
   "source": [
    "# @title Imports\n",
    "# Ignore Warnings\n",
    "import warnings\n",
    "\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "# Native Libraries\n",
    "import json\n",
    "import time\n",
    "import os\n",
    "\n",
    "# Other External Libraries\n",
    "import torch\n",
    "import numpy as np\n",
    "from pprint import pprint\n",
    "import locale\n",
    "import bs4  # For parsing web text\n",
    "from tqdm.autonotebook import tqdm\n",
    "import pandas as pd\n",
    "import joblib\n",
    "\n",
    "# Huggingface Transformers\n",
    "from transformers import (\n",
    "    AutoTokenizer,\n",
    "    AutoModelForCausalLM,\n",
    "    pipeline,\n",
    "    BitsAndBytesConfig,\n",
    ")  # To declare and download Mistral LLM\n",
    "\n",
    "# Main Langchain Imports\n",
    "from langchain.llms import (\n",
    "    HuggingFacePipeline,\n",
    ")  # Can use an existing transformer pipeline and convert to LC\n",
    "from langchain import PromptTemplate, LLMChain, hub\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_community.embeddings import (\n",
    "    HuggingFaceEmbeddings,\n",
    ")  # To generate embeddings to store in vectorstore\n",
    "\n",
    "# Vector Stores\n",
    "from langchain_community.vectorstores import FAISS, Chroma, Qdrant  # Main vectorstore\n",
    "from langchain_community.utils.math import cosine_similarity\n",
    "\n",
    "# Text\n",
    "from langchain_text_splitters import (\n",
    "    CharacterTextSplitter,\n",
    "    RecursiveCharacterTextSplitter,\n",
    ")  # for vector store generation\n",
    "from langchain_core.output_parsers import (\n",
    "    StrOutputParser,\n",
    ")  # for parsing [INST] in Mistral\n",
    "from langchain_core.runnables import (\n",
    "    RunnablePassthrough,\n",
    ")  # for passing str {} formatters to llm prompt template\n",
    "import evaluate\n",
    "\n",
    "\n",
    "# Cohere Community Specific Model\n",
    "from langchain_community.chat_models import ChatCohere\n",
    "from langchain_cohere import ChatCohere\n",
    "\n",
    "# Document Loaders for RAG\n",
    "from langchain_community.document_loaders import (\n",
    "    WebBaseLoader,\n",
    "    ArxivLoader,\n",
    "    TextLoader,\n",
    "    WikipediaLoader,\n",
    "    OnlinePDFLoader,\n",
    "    PyMuPDFLoader,\n",
    "    PubMedLoader,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "faZ5fLk_xxAO"
   },
   "outputs": [],
   "source": [
    "# @title Setting Configs\n",
    "locale.getpreferredencoding = lambda: \"UTF-8\"\n",
    "os.makedirs(\"mids_290_a5\", exist_ok=True)\n",
    "tqdm.pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "Stlb_ciPxxWA"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "!pip install sentence_transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xJWr9TkCa7gG"
   },
   "source": [
    "Add your keys from the secret store (do **NOT** print them out or leave them exposed as plaintext in your notebook!):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "ll9IqkVMa7qP"
   },
   "outputs": [],
   "source": [
    "COHERE_API_KEY: str = userdata.get(\"COHERE_API_KEY\")"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "---\n",
    "\n",
    "\n",
    "\n",
    "#### Fine Tuning Vs Indexing\n",
    "\n",
    "To get large language models to answer questions based on domain specific knowledge, we can either fine-tune a large language model or let the large language model use an external index to first find similar embeddings related to the query and subsequently answer the query based on similar embeddings.\n",
    "\n",
    "With `Indexing`, new documents are available in real time compared to tuning where we would could require a couple of hours depending on the size of the data as well as the model. When `Fine tuning`, we could also run into the context size limitation which is posed by most LLMs. Most LLMs allow around 4000 tokens per request and this makes it impossible to provide a large lump of data. With the indexing approach, we could literally provide an unlimited amount of data because of the retrieval of similar documents which we only send to the LLM\n",
    "\n",
    "Fine-tuning might not be necessary when:\n",
    "\n",
    "* The pre-trained model already performs well on the type of data or domain in question.\n",
    "* Computational resources or time are limited.\n",
    "* The task involves only retrieving relevant documents without needing to adapt the model's generation capabilities to specific nuances of the new data.\n",
    "\n",
    "In such cases, you could directly use the retriever to fetch relevant documents sorted by their relevance to the input question, leveraging existing embeddings and a similarity measure without adjusting the model's parameters."
   ],
   "metadata": {
    "id": "DHzyLsFHgj0A"
   }
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xlSHHPW-f3ZL"
   },
   "source": [
    "##2. Building the Components of our RAG System\n",
    "\n",
    "Let us introduce and test the base components of our RAG system. We will largely use the `Hugging Face` and `LangChain` libraries.\n",
    "\n",
    "\n",
    "#### RAG Components\n",
    "\n",
    "A pre-trained `Retrieval-Augmented Generation (RAG)` model is not just any large language model (LLM) like Mistral7B; it specifically combines the powers of a retriever and a generator to enhance question-answering capabilities. The RAG setup uses:\n",
    "\n",
    "* `Retriever`: This component fetches relevant documents or passages (e.g., from a database of Wikipedia or arXiv texts) based on the input query. Typically, this uses dense passage retrieval techniques like `DPR (Dense Passage Retrieval)`.\n",
    "\n",
    "* `Generator`: After retrieval, this component, usually a transformer-based model, generates an answer based on the retrieved texts combined with the query. This is often a fine-tuned version of a large language model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6N3fqR5vKV9b"
   },
   "source": [
    "###2.1 The Embedding Model\n",
    "\n",
    "We will need to represent text (pieces) as vectors. For this, we will use the sentence_transformer architecture.\n",
    "\n",
    "**NOTE:** The models you can use are: `all-MiniLM-L6-v2`, `multi-qa-mpnet-base-dot-v1` and `avsolatorio/GIST-Embedding-v0`\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "m_AqjidjKWif"
   },
   "outputs": [],
   "source": [
    "# @title Embeddings Setup\n",
    "%%capture\n",
    "\n",
    "minilm_embd: str = \"all-MiniLM-L6-v2\"\n",
    "multi_qa_embd: str = \"multi-qa-mpnet-base-dot-v1\"\n",
    "avso_embd: str = \"avsolatorio/GIST-Embedding-v0\"\n",
    "\n",
    "base_embeddings = HuggingFaceEmbeddings(model_name=multi_qa_embd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "Cgzrqje8PN8S",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "aefee7de-f2a3-4472-8ce6-03e47452307e"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Embedding dimension: 768\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "metadata": {},
     "execution_count": 8
    }
   ],
   "source": [
    "text = \"This is a test document.\"\n",
    "query_result = base_embeddings.embed_query(text)\n",
    "print(f\"Embedding dimension: {len(query_result)}\")\n",
    "\n",
    "doc_result = base_embeddings.embed_documents([text, \"This is not a test document.\"])\n",
    "len(doc_result)"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "query_result[0:5]"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "prjt06E2UC4I",
    "outputId": "bdd4d23b-e93e-4bad-8028-f2d357e6e6e3"
   },
   "execution_count": 9,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "[-0.33193400502204895,\n",
       " -0.31659939885139465,\n",
       " -0.43882352113723755,\n",
       " -0.12748152017593384,\n",
       " -0.15996131300926208]"
      ]
     },
     "metadata": {},
     "execution_count": 9
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fAp4fF5dPhbM"
   },
   "source": [
    "That looks reasonable. This is how you should define your embedding models.\n",
    "\n",
    "Next, we turn to text chunks.\n",
    "\n",
    "###2.2. Loading and Chunking Texts\n",
    "\n",
    "We first need to load the documents. Here is an example: [`https://lilianweng.github.io/posts/2023-06-23-agent/`](https://lilianweng.github.io/posts/2023-06-23-agent/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "4ak5XlaUP1cW"
   },
   "outputs": [],
   "source": [
    "loader = WebBaseLoader(\n",
    "    web_paths=(\"https://lilianweng.github.io/posts/2023-06-23-agent/\",),\n",
    "    bs_kwargs=dict(\n",
    "        parse_only=bs4.SoupStrainer(\n",
    "            class_=(\"post-content\", \"post-title\", \"post-header\")\n",
    "        )\n",
    "    ),\n",
    ")\n",
    "\n",
    "documents = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "doc_content = documents[0].to_json()[\"kwargs\"][\"page_content\"]\n",
    "print(len(doc_content.strip()))"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8AhyH23AUKc7",
    "outputId": "0f0ed4cd-b7f9-4631-b408-5ce02bf93ed7"
   },
   "execution_count": 11,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "43122\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cZXtDqKcQHlT"
   },
   "source": [
    "Once the documents are loaded, they may need to be split or chunked into smaller parts. This is particularly useful for large documents. LangChain also offers different algorithms for splitting documents, optimized for specific document types like code or markdown. Some of the splitters are:\n",
    "\n",
    "* `RecursiveCharacterTextSplitter`: Splits by characters; adjustable fragment size\n",
    "* `CharacterTextSplitter`: Like the above, but with custom separators like `“\\n\\n”` and `“ “`.\n",
    "* `RecursiveTextSplitter`: Splits by words/tokens, ideal for content analysis.\n",
    "* `TokenTextSplitter`: Uses OpenAI for token-based, context-aware segmentation. Ideal for advanced NLP.\n",
    "\n",
    "---\n",
    "We will need to split the  text in chunks that are 'suitable' as retrieval units. Let's for starters define a chunk size of 128 and have no overlap between the chunks:  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "qhcWoTajQnw6",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "74bda08c-a073-4db5-805d-8c01fc91bb64"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Number of splits/chunks:  444\n"
     ]
    }
   ],
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=128, chunk_overlap=0)\n",
    "splits = text_splitter.split_documents(documents)\n",
    "print(\"Number of splits/chunks: \", str(len(splits)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OUrNISkVRB_U"
   },
   "source": [
    "Ok, so it looks like we have now many splits (chunks) from one document. Here is how you can get the content:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "du9avBkhP1ll",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 36
    },
    "outputId": "3a51ccb4-fafc-4096-b016-f60add4e8b7f"
   },
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "'correcting previous mistakes. It plays a crucial role in real-world tasks where trial and error are inevitable.'"
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      }
     },
     "metadata": {},
     "execution_count": 13
    }
   ],
   "source": [
    "splits[39].page_content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Z-RuP_9xRVtG"
   },
   "source": [
    "Perfect. Now we have the splits and embeddings. Next, the embeddings need to be stores in a vector db.\n",
    "\n",
    "###2.3 Storing the Embeddings of Chunks in Vectorstores\n",
    "\n",
    "> `Collections` in `Qdrant` are similar to tables in traditional databases. They store vectors along with their payload\n",
    "\n",
    "After loading and chunking the data, we need to save the vector representations of the chunks in a vectorstore. We will use Qdrant here for simplicity. We load the splits (structured chunks) and the embeddings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "9T3FVDoJRglN"
   },
   "outputs": [],
   "source": [
    "vectorstore = Qdrant.from_documents(\n",
    "    splits,\n",
    "    base_embeddings,\n",
    "    location=\":memory:\",  # Local mode with in-memory storage only\n",
    "    collection_name=\"test\",\n",
    ")\n",
    "retriever = vectorstore.as_retriever()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ar4bpzJtRrio"
   },
   "source": [
    "The nice thing is that the vector store also does the similarity searches for us:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "tiSqastIP1oT"
   },
   "outputs": [],
   "source": [
    "query = \"What is Chain of Thought doing?\"\n",
    "docs = vectorstore.similarity_search_by_vector(\n",
    "    base_embeddings.embed_query(query)\n",
    ")  # will rank the splits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "k7CO0lvSR0MA",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "85fb6c8b-7daa-4765-e8da-6c333cb1740f"
   },
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "[Document(page_content='the model’s thinking process.', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', '_id': 'c441d03b159d4dd689f7aac012651613', '_collection_name': 'test'}),\n",
       " Document(page_content='[1] Wei et al. “Chain of thought prompting elicits reasoning in large language models.” NeurIPS 2022', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', '_id': 'c35878417fbc4c19b66abd58c86dde38', '_collection_name': 'test'}),\n",
       " Document(page_content='the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', '_id': '8f56e0b635684f0d8e4b40292ceca0fb', '_collection_name': 'test'}),\n",
       " Document(page_content='Chain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', '_id': '7a1a81ea8b4d4790b02bf92d8299d6ba', '_collection_name': 'test'})]"
      ]
     },
     "metadata": {},
     "execution_count": 16
    }
   ],
   "source": [
    "docs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yyypf-VZRsxg"
   },
   "source": [
    "Looks good! We have an ordered list of documents that seem to relate to the question. That is what we need.\n",
    "\n",
    "The last major component is the actual LLM."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lDDom9EbKXCX"
   },
   "source": [
    "###2.4. The LLM\n",
    "\n",
    "We will use one Open Source Model (`mistralai/Mistral-7B-Instruct-v0.1`) and one Proprietery Model (`Cohere`) for our tests. Let's first set up the OS model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "YooxnCPNOoQ7",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 386,
     "referenced_widgets": [
      "61bde2cf87764e98b7b6e4d2bef74635",
      "4b26ea2f2a8947d6989d687952ef90f9",
      "c313f2d0b0e64b14b6ce52a89f3481fc",
      "01814c150eab4edb941b02411e6fd904",
      "aa8b54944432416486cbc27babffc3fb",
      "c73633ddc1ab40c0a89cc36f4099f550",
      "490036306fb14b5da2f0b74822d35e0a",
      "cdf608c489e74aeab10bc384cee0ce43",
      "dcfe120f20aa42a0853db39f9dccaf99",
      "11843f8af3174008ac15d7823330b1b6",
      "3905358e2ee6402fb14f59e45d62535f",
      "889644558f4545baafebe515da640fdc",
      "8594b70918014ce18fe56b7d5869311b",
      "e731cdfea27f4407bfb1964a26af7b2a",
      "290253241b04490fb6a801336aca0053",
      "4eaae0855a174b75807af821a530a50c",
      "9f21918403fe443bb07fe015e034a2eb",
      "1491f89bcec1471c875c1b779ed2de33",
      "f2a31a3cc3d24b4aa7a14bc30b62b4bc",
      "597d99f7772f4e9a89d8130aeca45912",
      "ebb1be8d68284638abbeb8ba19e5e8bd",
      "d24b30669c9a4b1bba753226f179d212",
      "fb36ca2e6f66461dbc1054ec1fb9ab0f",
      "fc24cd8378944ba3881080fe9d225493",
      "625c961f94f94be998d063272f90ffd5",
      "337136d190a4499aa12c72de4dce0a69",
      "e118f57460b24beea9279a3de53c9e8f",
      "77b1efde2db943bca74242090a3b66dc",
      "3f70d6e135e044ceb51be6914ff42385",
      "bcf6012285b34feaa1b9144b591370b3",
      "0770d6701f95429e81db462502a001dc",
      "706922f202ca4dffa4a4d6ffc123fc05",
      "56beafbff551493d82570b507d6f8fbc",
      "507fd096fb0e43b895ba4d5ad3566c33",
      "43bb1c701401416e93ae65e069ef9143",
      "a8fc6d6ebb3545ff8d472565d2b8d788",
      "fe783695f6e441d3bd2788176be8b214",
      "761d86bcd9d940f6b402bda5af974b1b",
      "db47ed472db44dd2a3688810c5941a9c",
      "4f4e8575a81f469886f95d465fe56996",
      "d3e5282742f74e30b0e83fcc4d24e8c0",
      "ddce6b8c73d846e7ad8fa7d41e67bd9b",
      "24dddd7b7f954619b59d5c72107a315b",
      "62b545e1a0714810a379ad3d9ada8530",
      "476cae2e8fd040cbb69141d88dfe96d1",
      "ed8e4e559ac6460d80adf9df95e115b0",
      "559585f0be7a4ae2984bb007374fb83c",
      "2d40f47ff6e7409882edfef7e2a46588",
      "af37875552af48498f5440962bfc0135",
      "274d9d1d796a458e8059b7eff510b4a2",
      "ad5416de1aaa4a329443c093787c1fb1",
      "402307636b59421a82248dd433132617",
      "a812d758e10e45ada68bca3f7659fe8d",
      "83e2cc7ee9aa4892bd6342c32bfabc9c",
      "98ac1600cba14c26adda5f3d5e9599af",
      "9b455df848f04c17ac5022dba1cde781",
      "0a7d8e5a349441318167f9095268d1af",
      "453f436dc1aa4686bb4b3e3a90db6c03",
      "14e8d395b56c42f4b5628366db68b280",
      "6292aa65d3c04dd7b0705a2748a6bcaa",
      "ca14381c0fb547fcb840cfc64d77c18e",
      "9f815762f4c34ddeacea219313b59417",
      "741e5ba8e416488ab4537d403a1ff97c",
      "529031d7699743b78566fc3c0da7416a",
      "58a9b6c104aa40c1917a36ddccc6d408",
      "fa585bdf39cc418996fc6317906f6ad7",
      "5cd896948ecd4cf59f20ea2a6485f0fb",
      "059e676192b84a6288d78df790767adb",
      "143639dd1d10425dbac8aa3f619b76bc",
      "3ed76de4a01c46618a3482d0f8c5eeed",
      "ef68ffccdc9244f09eeb001a0c529ccc",
      "d4ba6c8d120340c990d8b297e28cf3d6",
      "752fe6fddd514490bd932724c3eda477",
      "3e9e402c493a4176a360e0d5d1601818",
      "69c1695056c64314b38b5716da58e684",
      "f2cc787744ec46d5bf85630f6f679da8",
      "3d95a1cc89e343a6ac29f81ed4cb5945",
      "35f3f7cb3a70420495e55ae6721e55a4",
      "612ce55e7cdc4d028a402bab0d0d22b8",
      "67330f73c19e493ca8036a471646e093",
      "c702a3fce244413695685ee042805e5e",
      "ce192cdf6653461abed37aebe0509a55",
      "5f5617b2a18244f78974fa3771bcdb6d",
      "c423ac30b643486a98b4eff882ef4396",
      "5b0cdce971bb4e9aa39a3c73c7e88f01",
      "773b4d9c7287424ebbbca4eabc1383a6",
      "4173ada84f9d4d6f91e65cdb2f0364a4",
      "17451c57711e433daae16d309fe7e997",
      "0da19d911f8c4e409798166ebca5f5ea",
      "952fc87c2aba45569ab04c84b4ec36d4",
      "9e750c7129794f1aa9b92094757a9e70",
      "5e8e2b7c2e9945c8862fa9fbe5bcf754",
      "21c6627657c74f4a9b1369470583b579",
      "ca5cf69da5354d0b85170d0c4e3d7209",
      "6a8f4ad9edb6434182c9988391032c68",
      "d49bead556784f66b195572c0103762d",
      "8ae0a183e1d940a3857f5c95041e76df",
      "8c9fd541aded4ea4af997015b589fb06",
      "4af3a0699a094fae9cca8458a5804e05",
      "9020bbbca52541d4aae58df85fc6278d",
      "3c051e183766421782510d3a5872952d",
      "3aa456496426433ca499c47bc7f16353",
      "b9091cb89166441a8d3cda2bbf6a1e70",
      "f0d67326b4c14c5ba37d42b9e4d8a7da",
      "147ae93de43941b3be92053b96e83526",
      "424c1786e02947ce82b74e15588cbbab",
      "db03405d42de4065ad3fbbca8e3bfee2",
      "f4e129c465714709be9b1339d1b80ad2",
      "2bdbcc5258ee4684917dabb337a24905",
      "404d6f4424ed45e38a0ab31d1db142e1",
      "e897f8bd26fa421585a0d2efe7515c09",
      "c69c9577417c4f58bd4c85d6a6a1b739",
      "e28994ac1b464c4cace1416713c02cd6",
      "71c51aca8e944a6b824ef8b6ead4628a",
      "88123e66271d401dada14b92c3837ff4",
      "0d53925aa50446ae85d09b6bdda148f1",
      "24bf9fa741f047319547ccabef6472b1",
      "49027c6b8db34c0da48402dd36c26b4f",
      "3cac471e2eb54ca8b9bfd7246f4f8236",
      "06c3498fc6724dc284d6e15c4cf862f3",
      "4604a5e0932c466e9af36debbece4960"
     ]
    },
    "outputId": "bd256906-1b2a-40d1-b660-b4e787c2664b"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Unused kwargs: ['llm_int4_enable_fp32_cpu_offload']. These kwargs are not used in <class 'transformers.utils.quantization_config.BitsAndBytesConfig'>.\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "config.json:   0%|          | 0.00/571 [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "61bde2cf87764e98b7b6e4d2bef74635"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model.safetensors.index.json:   0%|          | 0.00/25.1k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "889644558f4545baafebe515da640fdc"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "fb36ca2e6f66461dbc1054ec1fb9ab0f"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model-00001-of-00002.safetensors:   0%|          | 0.00/9.94G [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "507fd096fb0e43b895ba4d5ad3566c33"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model-00002-of-00002.safetensors:   0%|          | 0.00/4.54G [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "476cae2e8fd040cbb69141d88dfe96d1"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "9b455df848f04c17ac5022dba1cde781"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "generation_config.json:   0%|          | 0.00/116 [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "5cd896948ecd4cf59f20ea2a6485f0fb"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/1.47k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "35f3f7cb3a70420495e55ae6721e55a4"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "tokenizer.model:   0%|          | 0.00/493k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "0da19d911f8c4e409798166ebca5f5ea"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/1.80M [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "9020bbbca52541d4aae58df85fc6278d"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "special_tokens_map.json:   0%|          | 0.00/72.0 [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "e897f8bd26fa421585a0d2efe7515c09"
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "# @title LLM Setup - `Mistral7B`\n",
    "# %%capture\n",
    "\n",
    "quantization_config = BitsAndBytesConfig(\n",
    "    load_in_4bit=True, llm_int4_enable_fp32_cpu_offload=True\n",
    ")\n",
    "\n",
    "\n",
    "llm_mistral_model = AutoModelForCausalLM.from_pretrained(\n",
    "    \"mistralai/Mistral-7B-Instruct-v0.1\",\n",
    "    torch_dtype=torch.float32,\n",
    "    device_map=\"auto\",\n",
    "    quantization_config=quantization_config,\n",
    ")\n",
    "\n",
    "llm_mistral_tokenizer = AutoTokenizer.from_pretrained(\n",
    "    \"mistralai/Mistral-7B-Instruct-v0.1\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZVGOr2cV_q6I"
   },
   "source": [
    "We use the model first to generate a Hugging Face pipeline. A pipeline simplifies the process of actually generating responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "Rvxo5OKwvjNN"
   },
   "outputs": [],
   "source": [
    "# @title `Mistral7B` Transformers Pipeline\n",
    "mistral_pipe = pipeline(\n",
    "    \"text-generation\",\n",
    "    model=llm_mistral_model,\n",
    "    tokenizer=llm_mistral_tokenizer,\n",
    "    max_length=1000,\n",
    "    temperature=0.6,\n",
    "    top_p=0.95,\n",
    "    do_sample=True,\n",
    "    repetition_penalty=1.2,\n",
    ")\n",
    "mistral_pipe.model.config.pad_token_id = mistral_pipe.model.config.eos_token_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kOHYPzDiTRIK"
   },
   "source": [
    "Does it work?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "wAUHEvq_TRSv",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "583ee5c7-238b-4b2f-cf1c-516f485abee4"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[{'generated_text': '[INST]Give me a two-sentence story about an apple![/INST] Once upon a time, there was a beautiful red apple that grew on a tree in the orchard. One day, a little girl came and picked the apple, and it became her favorite fruit.'}]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[{'generated_text': '[INST]What is the capital of France[/INST] The capital city of France is Paris.'}]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[{'generated_text': '[INST]What is RAG (Retrieval Augmented Generation)[/INST] Retrieval-Augmented Generation (RAG) refers to a type of natural language processing technique used in text generation. In this approach, the system uses both retrieval and generative processes to generate new text.\\n\\nThe retrieval process involves searching for relevant information from existing sources such as databases or texts that can be incorporated into the generated text. The generative process involves creating new text based on the input data using machine learning algorithms such as sequence-to-sequence models or transformers.\\n\\nIn RAG systems, the output of the retrieval process is fed into the generative model as an additional input. This allows the system to leverage the strengths of both approaches - the ability to incorporate relevant information through retrieval, and the flexibility and creativity of generative models to create new content.\\n\\nRAG has shown promise in generating high-quality text in various applications such as chatbots, customer service agents, and news articles.'}]\n",
      "[{'generated_text': '[INST]What are the best places to dine-out in Bangalore?[/INST] Bangalore has a diverse and vibrant food scene, with many great options for dining out. Some of the best places to eat in Bangalore include:\\n\\n1. Toit Brewpub - This is a popular spot for craft beer lovers, offering a wide variety of beers on tap as well as delicious pub fare.\\n2. Pump House Restaurant & Bar - Located in Whitefield, this restaurant offers a stunning view of the city skyline and serves up international cuisine with an Indian twist.\\n3. Humming Tree - A lively bar and live music venue, Humming Tree also serves up some of the best street food in town.\\n4. The Permit Room - This speakeasy-style establishment offers a unique cocktail menu and small plates made from locally sourced ingredients.\\n5. The Humming Tree Kitchen - Another branch of the Humming Tree brand, this location focuses more on food than drinks and serves up dishes inspired by different regions of India.'}]\n"
     ]
    }
   ],
   "source": [
    "q: list[str] = [\n",
    "    \"[INST]Give me a two-sentence story about an apple![/INST]\",\n",
    "    \"[INST]What is the capital of France[/INST]\",\n",
    "    \"[INST]What is RAG (Retrieval Augmented Generation)[/INST]\",\n",
    "    \"[INST]What are the best places to dine-out in Bangalore?[/INST]\",\n",
    "]\n",
    "\n",
    "for each_q in q:\n",
    "    print(mistral_pipe(each_q))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Sh-60AEBTODZ"
   },
   "source": [
    "Reasonable!\n",
    "\n",
    "We will also use a Cohere model, but will create this below as part of the LangChain framework.\n",
    "\n",
    "###2.5 Testing the LLM in a LangChain Chain\n",
    "\n",
    "#### Langchain Components\n",
    "To build a RAG using LangChain, you'll primarily interact with the following components:\n",
    "\n",
    "* `Language Models`: These are the core components used for generating text. In LangChain, you can integrate various LLMs, including custom models trained or fine-tuned for specific tasks.\n",
    "\n",
    "* `Retrievers`: LangChain supports integrating various retrieval systems that can fetch relevant information from a knowledge base. This is critical for the RAG's first step.\n",
    "\n",
    "* `Chains`: These are sequences of operations (like retrieving then answering) that LangChain can execute. Building a RAG typically involves creating a chain that first retrieves relevant documents and then uses a language model to generate an answer based on the retrieved content.\n",
    "\n",
    "* `Adapters`: These allow LangChain to connect to different backends for both language models and databases, facilitating the integration of specific models or data sources.\n",
    "\n",
    "LangChain provides a high-level API that abstracts much of the complexity involved in setting up these components, allowing you to focus more on tailoring the solution to your needs\n",
    "\n",
    "---\n",
    "\n",
    "Chains will be defined and discussed in Week 11. In short, they are convenient programmatic ways to deal with 'chains' of actions that involve LLMs. For example, a list of events like 'here is a city name. Plug that city name into prompt template, then generate a story about that city. Lastly, format the model output as a string' can be easily handled by LangChain's Chain framework. In this case, the Chain would consist of the prompt template, the LLM, and the String Formatter. The parameter (the city in this case) will be provided at run time by invocation of the Chain. Let's test that.\n",
    "\n",
    "To use a Hugging Face model in a LangChain environment, we need to wrap the model into a LangChain pipeline object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "P1LSyyFmTOYP"
   },
   "outputs": [],
   "source": [
    "# @title `Mistral7B` Langchain Pipeline\n",
    "# Create a langchain instance of the mistral transformer pipeline\n",
    "mistral_llm_lc = HuggingFacePipeline(pipeline=mistral_pipe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3FkQde8yZYqZ"
   },
   "source": [
    "Next, we need to define a template and create a corresponding prompt template that can take any questiion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "dh3R9445K5ct"
   },
   "outputs": [],
   "source": [
    "test_llm_template: str = (\n",
    "    \"\"\"[INST] Give me a two-sentence story about {object}! [/INST]\"\"\"\n",
    ")\n",
    "test_llm_prompt_template = PromptTemplate(\n",
    "    template=test_llm_template, input_variables=[\"object\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_V6PO2LISDbT"
   },
   "source": [
    "Let's define a Chain, a static flow of actions that (usually) involve at least a definition of the variables used in the chain, one or more templates, LLM step(s) and potentially other actions. This would be a chain that declares the variable 'object' to be expected when the chain is invoked, then inserts it into the template, and passes this to our mistral model pipeline (wrapped as a LangChain object):    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "aSvSoD7hZb4Z"
   },
   "outputs": [],
   "source": [
    "test_llm_chain_short = (\n",
    "    {\"object\": RunnablePassthrough()} | test_llm_prompt_template | mistral_llm_lc\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "jH8lUWWSZ-9n",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "f4b12882-3934-4d16-9628-33822043ab57"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[INST] Give me a two-sentence story about an apple! [/INST] Once upon a time, there was a small red apple that grew on the branch of an old oak tree. One day, a curious little girl plucked it from the tree and discovered that inside was a tiny note written in delicate cursive: \"To whoever finds this - may you always have joy in your heart.\"\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[INST] Give me a two-sentence story about a planet! [/INST] Once upon a time, on a distant planet far beyond our solar system, there lived an intelligent species of beings who thrived in their lush and colorful world. But one day, a terrible natural disaster struck the planet, threatening to destroy everything they had built and cherished.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[INST] Give me a two-sentence story about a mountain flower! [/INST] As the sun rose over the mountain, the delicate petals of the flower slowly unfurled to reveal its beauty. Despite the harsh conditions and rocky terrain, the flower thrived with resilience, brightening up the barren landscape.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[INST] Give me a two-sentence story about a beach! [/INST] The sun was setting over the endless horizon, painting the sky with shades of orange and pink as Sarah walked barefoot on the warm sand of the beach, feeling the waves gently kissing her toes. She closed her eyes, taking in the peaceful sound of the ocean and the salty breeze that rustled through her hair, feeling at peace in this moment.\n",
      "[INST] Give me a two-sentence story about a skyscraper! [/INST] Once a symbol of human ambition and progress, the towering skyscraper now stood as an abandoned relic of a bygone era, its once gleaming glass facade tarnished with age and neglect. As the sun set behind it, casting long shadows across the deserted streets below, the skyscraper whispered stories of the people who had once worked and lived within its walls, their hopes and dreams now lost to the sands of time.\n"
     ]
    }
   ],
   "source": [
    "obs: list[str] = [\n",
    "    \"an apple\",\n",
    "    \"a planet\",\n",
    "    \"a mountain flower\",\n",
    "    \"a beach\",\n",
    "    \"a skyscraper\",\n",
    "]\n",
    "for ob in obs:\n",
    "    print(test_llm_chain_short.invoke(ob))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SYeKmOoMaSKS"
   },
   "source": [
    "Works too. We will use this notation moving forward.\n",
    "\n",
    "Next, how would we do this with a Cohere Chat Model instead of Mistral?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "JROGul-gZcAU"
   },
   "outputs": [],
   "source": [
    "# @title Cohere LLM Setup\n",
    "cohere_chat_model = ChatCohere(cohere_api_key=COHERE_API_KEY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-h3DW_bBfPsI"
   },
   "source": [
    "This can be plugged straight into the Chain:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "AJ5bzWTVaZdO"
   },
   "outputs": [],
   "source": [
    "test_cohere_llm_chain_short = (\n",
    "    {\"object\": RunnablePassthrough()} | test_llm_prompt_template | cohere_chat_model\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "ZldFGowOciCk",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "5386905b-8700-4df8-ddb0-dcb9ebcb58dd"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "content=\"The apple, once bitter and unappealing, fell from the tree and transformed into a sweet, juicy treat. Its once sour notes now a distant memory, it became a symbol of nature's surprising gifts.\" additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '49d35faf-84f6-46e9-b774-ca82f347c69b'} response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '49d35faf-84f6-46e9-b774-ca82f347c69b'} id='run-9feea12e-5889-437c-a35d-83d336b9394e-0'\n",
      "content='The planet Xylara, a lush, verdant world teeming with exotic life, orbits a distant star. Its peaceful existence is disrupted when an ancient evil, long dormant, stirs and threatens to consume all that is good and pure.' additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': 'f1db6708-bf05-4d25-bc06-7c65498cbd44'} response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': 'f1db6708-bf05-4d25-bc06-7c65498cbd44'} id='run-96d5e3ef-b111-4fa6-b999-5f0d6d7d7f77-0'\n",
      "content='A delicate mountain flower, rare and pristine, blooms amidst the rugged peaks. Its beauty inspires a sense of wonder and awe in all who are fortunate enough to behold it.' additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': 'b4864256-f939-45a6-882c-496a2af1ff13'} response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': 'b4864256-f939-45a6-882c-496a2af1ff13'} id='run-07ccd1b7-7575-4b13-8af5-3be1c930ae81-0'\n",
      "content='The sun shone brightly on the crystal-clear waters of the beach, as seagulls soared overhead, their calls mixing with the gentle crash of waves. Children built sandcastles, their laughter carrying on the breeze, while their parents relaxed nearby, soaking in the warm sun.' additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '182326a5-3ede-4f04-a946-14065f75c422'} response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '182326a5-3ede-4f04-a946-14065f75c422'} id='run-49551db7-ce43-46c4-8e9b-74cead3596c2-0'\n",
      "content='The towering skyscraper, a symbol of ambition, rose above the city. Its gleaming glass facade reflected the aspirations of those who built it, reaching for the sky.' additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '2c51053d-9d17-43b4-87df-deafb0c03b8a'} response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '2c51053d-9d17-43b4-87df-deafb0c03b8a'} id='run-704af2dc-3f07-456f-92bc-104f1b365bd8-0'\n"
     ]
    }
   ],
   "source": [
    "for ob in obs:\n",
    "    print(test_cohere_llm_chain_short.invoke(ob))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1CasSDdMnF8z"
   },
   "source": [
    "Works! (Note: you may want to review the format of the template. The one we used here is the one from Mistral, and the format may or may not be optimal for Cohere.)\n",
    "\n",
    "How can we get the output formatting under control? We can add a String Formatter to the chain:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "3rSDhR5AeR7_",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "9f4b4f05-00b1-4d3e-db13-dad781072a54"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "The apple, once bitter and unappealing, transformed into a sweet and juicy delight after a long, sunny summer. It was then that the farmer knew his hard work had paid off, and he could finally reap the rewards of his labor.\n",
      "The planet Xylara, a lush, verdant world teeming with exotic life, orbits a distant star. Its peaceful existence is threatened when an alien race arrives, seeking to exploit its natural resources, but the brave resistance fighters of Xylara rise up to defend their beloved home.\n",
      "A delicate mountain flower, rare and pristine, blooms amidst the rugged peaks. Its beauty inspires a sense of wonder and awe in all who are fortunate enough to behold it.\n",
      "The sun-soaked beach, with its golden sand and crystal-clear waters, was a haven of relaxation. With the gentle breeze carrying the scent of the ocean, it was the perfect place to unwind and forget the world.\n",
      "The towering skyscraper, a symbol of ambition and innovation, soared above the city. Its sleek glass and steel structure reflected the changing skies, a majestic testament to human ingenuity and the pursuit of progress.\n"
     ]
    }
   ],
   "source": [
    "output_parser = StrOutputParser()\n",
    "\n",
    "test_cohere_llm_chain_short_formatted = (\n",
    "    {\"object\": RunnablePassthrough()}\n",
    "    | test_llm_prompt_template\n",
    "    | cohere_chat_model\n",
    "    | output_parser\n",
    ")\n",
    "\n",
    "for ob in obs:\n",
    "    print(test_cohere_llm_chain_short_formatted.invoke(ob))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ry1S5W_ueSWt"
   },
   "source": [
    "###2.6 Setting Up a Simple RAG Chain\n",
    "\n",
    "For RAG, we will follow the same approach. Except... you will **later** need to change the chain to include the retrieval step.\n",
    "\n",
    "We first do a simple test: create a RAG template that takes a question and a pre-defined context as input, and generates the answer based on the provided context:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "qrSx3gFlncAO",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "f60bafaa-382c-4930-9f58-33318bb5a36b"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Human: \n",
      "[INST] Answer the question based only on the following context:\n",
      "{'context': 'Germany has won the World Cup 4 times.', 'question': 'How many times did Germany win the world cup?'}\n",
      "\n",
      "Question: {'context': 'Germany has won the World Cup 4 times.', 'question': 'How many times did Germany win the world cup?'}\n",
      "[/INST]\n",
      "Based on the provided context, Germany won the World Cup 4 times.\n"
     ]
    }
   ],
   "source": [
    "rag_template: str = \"\"\"\n",
    "[INST] Answer the question based only on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "[/INST]\n",
    "\"\"\"\n",
    "rag_prompt_template = ChatPromptTemplate.from_template(rag_template)\n",
    "\n",
    "base_rag_chain = (\n",
    "    {\"context\": RunnablePassthrough(), \"question\": RunnablePassthrough()}\n",
    "    | rag_prompt_template\n",
    "    | mistral_llm_lc\n",
    "    | output_parser\n",
    ")\n",
    "\n",
    "predefined_context = \"Germany has won the World Cup 4 times.\"\n",
    "question = \"How many times did Germany win the world cup?\"\n",
    "\n",
    "resp = base_rag_chain.invoke({\"context\": predefined_context, \"question\": question})\n",
    "\n",
    "print(resp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Maq5x1jDhJiX"
   },
   "source": [
    "That's great. But of course, the context needs to be created in an earlier retrieval step. More precisely, the documents will be first retrieved as a list, and then they will need to be formatted into one string to pass to the LLM in the context window.\n",
    "\n",
    "Here is a simple formatting function that can be hooked into the chain, which combines a list of chunks into one string:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "id": "3VUMkGithJtY"
   },
   "outputs": [],
   "source": [
    "def format_docs(docs: list[str]) -> str:\n",
    "    return \"\\n\\n\".join(doc.page_content for doc in docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xv0wLvGQhJ5R"
   },
   "source": [
    "So how could we build a simple chain? Let's first just get the retrieval done and the formatted retrieved data and the question inserted into the prompt template:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "_UiGHgRLhKEZ"
   },
   "outputs": [],
   "source": [
    "rag_template = (\n",
    "    \"\"\"Here is a context:\\n{context} \\n\\nand here is a question: \\n{question}\"\"\"\n",
    ")\n",
    "\n",
    "rag_prompt = ChatPromptTemplate.from_template(rag_template)\n",
    "\n",
    "rag_chain = (\n",
    "    # retriever from vector store declaration - piped with format docs function above\n",
    "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
    "    | rag_prompt\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "sVRdhLzwjYk6"
   },
   "outputs": [],
   "source": [
    "output = rag_chain.invoke(\"What is Chain of Thought?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m8UynrWikgPc"
   },
   "source": [
    "Ok... with some formatting... this looks good:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "UjlKfsrljYnn",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "4337c43a-226c-4f2c-c005-b42c2a361d22"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Here is a context:\n",
      "the model’s thinking process.\n",
      "\n",
      "[1] Wei et al. “Chain of thought prompting elicits reasoning in large language models.” NeurIPS 2022\n",
      "\n",
      "the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process\n",
      "\n",
      "Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes \n",
      "\n",
      "and here is a question: \n",
      "What is Chain of Thought?\n"
     ]
    }
   ],
   "source": [
    "print(output.messages[0].content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kx0ZWB7Ul4x8"
   },
   "source": [
    "Let's complete the RAG Chain by using a `StrOutputParser` directly, instead of indexing the output like before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "Qh9ZopW3jYqA"
   },
   "outputs": [],
   "source": [
    "output_parser = StrOutputParser()\n",
    "\n",
    "rag_template = \"\"\"\n",
    "[INST]Please answer the question below only based on the context information provided.\n",
    "\n",
    "Here is a context: {context}\n",
    "Here is a question: {question}.\n",
    "[/INST]\"\"\"\n",
    "\n",
    "rag_prompt = ChatPromptTemplate.from_template(rag_template)\n",
    "\n",
    "rag_chain = (\n",
    "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
    "    | rag_prompt\n",
    "    | mistral_llm_lc\n",
    "    | output_parser\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "id": "_e_gx7lMjYse",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 174
    },
    "outputId": "8af88d0b-97af-421d-d883-3b024556e891"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset\n",
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "'Human: \\n[INST]Please answer the question below only based on the context information provided.\\n\\nHere is a context: the model’s thinking process.\\n\\n[1] Wei et al. “Chain of thought prompting elicits reasoning in large language models.” NeurIPS 2022\\n\\nthe problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process\\n\\nTree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes\\nHere is a question: What is Chain of Thought?.\\n[/INST] Based on the given context information, \"Chain of Thought\" refers to a method used in natural language processing and machine learning that involves breaking down complex problems or tasks into smaller sub-problems or sub-tasks, and generating a sequence of intermediate solutions or ideas that lead to a final solution. This approach is often used in training large language models, such as those used in artificial intelligence applications, and can involve exploring multiple possible reasoning paths or outcomes at each stage.'"
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      }
     },
     "metadata": {},
     "execution_count": 34
    }
   ],
   "source": [
    "rag_chain.invoke(\"What is Chain of Thought?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KfNT72j2mZw-"
   },
   "source": [
    "What about the Cohere models?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "id": "JHye-j_MjYvL"
   },
   "outputs": [],
   "source": [
    "cohere_rag_chain = (\n",
    "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
    "    | rag_prompt\n",
    "    | cohere_chat_model\n",
    "    | output_parser\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "id": "YnVEnMmNmrgM",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 70
    },
    "outputId": "89b02eb1-c1e0-4e6a-ad30-4285f590e37f"
   },
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "\"Chain of Thought (CoT) is a prompting technique used to elicit reasoning and improve the performance of large language models on complex tasks. It involves decomposing a problem into a series of thought steps, generating multiple thoughts for each step, and then using these thoughts to guide the model's search process. By organizing the model's thinking process in a structured manner, CoT helps the model to provide more accurate and reasoned responses.\""
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      }
     },
     "metadata": {},
     "execution_count": 36
    }
   ],
   "source": [
    "cohere_rag_chain.invoke(\"What is Chain of Thought?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gtgVF-T8nN9o"
   },
   "source": [
    "Works too! Time to build the real thing and do experimentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dM7gS9kGNOJp"
   },
   "source": [
    "##3. The RAG Model & Experimentation\n",
    "\n",
    "With this we can get started. First, we need to acquire the data, chunk it, vectorize it, and store the embeddings (and in this simple case also the docs) in our Qdrant vector db.\n",
    "\n",
    "\n",
    "###3.1 The Vector Database\n",
    "\n",
    "We will start by creating our datastore, Qdrant. Usually, you would deploy the vector db as a server, but in this case let's simply put everything in memory. Also, in this case we will store not only the embeddings but the whole document in the vector store. We will seed the store with the splits from the blog post we had used before.\n",
    "\n",
    "We will also create the retriever, which defines the way the documents are being retrieved. The retriever parameters define for example which method is used, how many docs are retrieved, etc. See [this LangChain link ](https://python.langchain.com/docs/modules/data_connection/retrievers/vectorstore)for more information.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "id": "aXxjbq6RsKq3"
   },
   "outputs": [],
   "source": [
    "# @title Vectorstore Initialization - `Qdrant`\n",
    "qdrant_vectorstore = Qdrant.from_documents(\n",
    "    splits,\n",
    "    base_embeddings,\n",
    "    location=\":memory:\",  # Local mode with in-memory storage only\n",
    "    collection_name=\"rag_tech_db\",\n",
    "    force_recreate=True,\n",
    ")\n",
    "\n",
    "retriever = qdrant_vectorstore.as_retriever()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GC9z0o5ZsLML"
   },
   "source": [
    "###3.2 Data Acquisition, Chunking, and Vectorization\n",
    "\n",
    "Now where we have our store we need to get the data into it. We will need to retrieve the data, create the chunks, then vectorize them, and finally store the vectors (along with the docs in this case) in the vector db.\n",
    "\n",
    "Let us first set chunk size and overlap, as well as the type of splitter. These are starting parameters and you may want to experiment with them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "id": "V_BR4rgYrCGn"
   },
   "outputs": [],
   "source": [
    "# @title **Hyperparams**\n",
    "# Note that these defaults may or may not be ideal!\n",
    "\n",
    "# CHUNK_SIZE: int = 64\n",
    "# CHUNK_SIZE: int = 128\n",
    "CHUNK_SIZE: int = 256\n",
    "OVERLAP: int = 0\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=CHUNK_SIZE, chunk_overlap=OVERLAP\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b-vhTZ_tvIvD"
   },
   "source": [
    "Now let's work with an actual document collection.  We will work with four types of documents:\n",
    "\n",
    "* A few papers from the `ArXiv` on RAG and NLP\n",
    "* A few blogs from `Lily Weng` that talk about Open Domain Question Answering and related topics\n",
    "* A number of `Wikipedia` articles on that topic\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t_xFjAUSx7ES"
   },
   "source": [
    "First we'll grab some papers from ArXiv.  We'll grab the pdf files and get all of the pages as separate documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "id": "s6Jj2vLAyxic"
   },
   "outputs": [],
   "source": [
    "arxiv_numbers = (\n",
    "    \"2005.11401\",\n",
    "    \"2104.07567\",\n",
    "    \"2104.09864\",\n",
    "    \"2105.03011\",\n",
    "    \"2106.09685\",\n",
    "    \"2203.02155\",\n",
    "    \"2211.09260\",\n",
    "    \"2211.12561\",\n",
    "    \"2212.09741\",\n",
    "    \"2305.14314\",\n",
    "    \"2305.18290\",\n",
    "    \"2306.15595\",\n",
    "    \"2309.08872\",\n",
    "    \"2309.15217\",\n",
    "    \"2310.06825\",\n",
    "    \"2310.11511\",\n",
    "    \"2311.08377\",\n",
    "    \"2312.05708\",\n",
    "    \"2401.06532\",\n",
    "    \"2402.01306\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "id": "hq8ETxzzygTj",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 101,
     "referenced_widgets": [
      "68485f50f1fc4e528fa08beff8d03900",
      "a510a0d6b67849449452b2eecc165651",
      "f47f8eedfda34b6e97e33cc040b61888",
      "e0f20a0ec2be4736bfcd45b01eb458e9",
      "dc19f17e15ba4b2bafcbcc2e32105104",
      "4ccc8312f5ec4b69a9b8218d9d65a3d1",
      "5276348b709b4d72871b54d1c00468a1",
      "dd432188eb2a433ea7d43607b4f3aff0",
      "f59318cd060042daaca4b4fa26bf9925",
      "9e770af0c3f34486948b716c14727fc9",
      "ddeee64de59c4a8faae835b0fc0f598b"
     ]
    },
    "outputId": "49240758-88ba-4664-c198-53f63513bfce"
   },
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/20 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "68485f50f1fc4e528fa08beff8d03900"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "len(all_arxiv_pages)=420\n",
      "len(arxiv_numbers)=20\n",
      "global_doc_number-1=20\n"
     ]
    }
   ],
   "source": [
    "# @title Arxiv Data Loader\n",
    "all_arxiv_pages = []\n",
    "\n",
    "# assign a unique number to each document we ingest\n",
    "global_doc_number = 1\n",
    "\n",
    "# loop through the papers\n",
    "for identifier in tqdm(arxiv_numbers):\n",
    "    # Construct URL using the arXiv unique identifier\n",
    "    arx_url: str = f\"https://arxiv.org/pdf/{identifier}.pdf\"\n",
    "\n",
    "    # Extract pages from the document and add them to the list of pages\n",
    "    arx_loader = PyMuPDFLoader(arx_url)\n",
    "    arx_pages = arx_loader.load()\n",
    "    for page_num in range(len(arx_pages)):\n",
    "        page = arx_pages[page_num]\n",
    "\n",
    "        # CHANGED\n",
    "        page.metadata[\"page_num\"] = page_num\n",
    "        page.metadata[\"doc_num\"] = global_doc_number\n",
    "        page.metadata[\"doc_source\"] = \"ArXiv\"\n",
    "        all_arxiv_pages.append(page)\n",
    "\n",
    "    global_doc_number += 1\n",
    "\n",
    "print(f\"{len(all_arxiv_pages)=}\")\n",
    "print(f\"{len(arxiv_numbers)=}\")\n",
    "print(f\"{global_doc_number-1=}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "id": "f6Qr75rnvLKJ",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "e1410370-0148-41a3-fffc-aae17ab21d2c"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Table 1: Open-Domain QA Test Scores. For TQA,\n",
      "left column uses the standard test set for Open-\n",
      "Domain QA, right column uses the TQA-Wiki\n",
      "test set. See\n"
     ]
    }
   ],
   "source": [
    "print(all_arxiv_pages[5].page_content[:150])  # all pages of the Document content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Czx-h_Aeg_NM"
   },
   "source": [
    "Now we need to split the docs into chunks.  LangChain provides a couple of ways to do that.  We'll use for now the `RecursiveCharacterTextSplitter`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "id": "8Ln3nmeVvLBI",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "950a56ac-a898-4d46-f4f8-782a04fa1c76"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Number of splits/chunks:  6712\n"
     ]
    }
   ],
   "source": [
    "# index doc chunks\n",
    "splits = text_splitter.split_documents(all_arxiv_pages)\n",
    "for idx, text in enumerate(splits):\n",
    "    splits[idx].metadata[\"split_id\"] = idx\n",
    "\n",
    "print(\"Number of splits/chunks: \", len(splits))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "id": "gVEpjCBLE50m",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "d50307db-fe42-4e78-d86b-bb95fbc0f74e"
   },
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "Document(page_content='Retrieval-Augmented Generation for\\nKnowledge-Intensive NLP Tasks\\nPatrick Lewis†‡, Ethan Perez⋆,\\nAleksandra Piktus†, Fabio Petroni†, Vladimir Karpukhin†, Naman Goyal†, Heinrich Küttler†,', metadata={'source': 'https://arxiv.org/pdf/2005.11401.pdf', 'file_path': 'https://arxiv.org/pdf/2005.11401.pdf', 'page': 0, 'total_pages': 19, 'format': 'PDF 1.5', 'title': '', 'author': '', 'subject': '', 'keywords': '', 'creator': 'LaTeX with hyperref', 'producer': 'pdfTeX-1.40.21', 'creationDate': 'D:20210413004838Z', 'modDate': 'D:20210413004838Z', 'trapped': '', 'page_num': 0, 'doc_num': 1, 'doc_source': 'ArXiv', 'split_id': 0})"
      ]
     },
     "metadata": {},
     "execution_count": 43
    }
   ],
   "source": [
    "splits[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u6DUxHWuozJB"
   },
   "source": [
    "Let's add the vectors to the datastore and see whether we can retrieve a nearest neighbor to a query. Let's look at the second closest match:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "KSJmDkj6SvQQ"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "vs_exception: bool = True\n",
    "\n",
    "try:\n",
    "    qdrant_vectorstore.add_documents(documents=splits)\n",
    "except Exception as exc:\n",
    "    print(exc)\n",
    "    vs_exception = False"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "assert vs_exception, \"There was an error storing all documents in the vectorstore\""
   ],
   "metadata": {
    "id": "Y5LJ6iXlkFGA"
   },
   "execution_count": 45,
   "outputs": []
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "id": "4xOhcgKd5ckk"
   },
   "outputs": [],
   "source": [
    "query = \"How can we train a model for preferences?\"\n",
    "found_docs = qdrant_vectorstore.similarity_search_with_score(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "id": "QDelZRCF5Ite",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "1b62ece0-e485-449f-e11f-df01d6f69311"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "preferences at once, or where everyone would endorse the tradeoffs.\n",
      "One path forward could be to train models that can be conditioned on the preferences of certain\n",
      "0.7036675973413791\n",
      "a large text dataset. While the most straightforward approach to preference learning is supervised\n",
      "fine-tuning on human demonstrations of high quality responses, the most successful class of methods\n",
      "0.6703005536978632\n"
     ]
    }
   ],
   "source": [
    "print(found_docs[0][0].page_content)\n",
    "print(found_docs[0][1])\n",
    "print(found_docs[1][0].page_content)\n",
    "print(found_docs[1][1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ipT82FOghpr6"
   },
   "source": [
    "Next, let's get some information from Wikipedia on our main topic -- Gen AI.  LangChain provides a `DocumentLoader` that accesses the `Wikipedia API`."
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Wikipedia Data Loader\n",
    "wiki_queries: list[str] = [\n",
    "    \"Generative Artificial Intelligence\",\n",
    "    \"Information Retrieval\",\n",
    "    \"Large Language Models\",\n",
    "]\n",
    "\n",
    "for wq in tqdm(wiki_queries):\n",
    "    wiki_docs = WikipediaLoader(query=wq, load_max_docs=4).load()\n",
    "\n",
    "    for idx, text in enumerate(wiki_docs):\n",
    "        wiki_docs[idx].metadata[\"doc_num\"] = global_doc_number\n",
    "        wiki_docs[idx].metadata[\"doc_source\"] = \"Wikipedia\"\n",
    "\n",
    "    global_doc_number += 1\n",
    "\n",
    "    print(f\"Number of documents {wq}: \", len(wiki_docs))\n",
    "\n",
    "    # index docs\n",
    "\n",
    "    wiki_splits = text_splitter.split_documents(wiki_docs)\n",
    "    for idx, text in enumerate(wiki_splits):\n",
    "        wiki_splits[idx].metadata[\"split_id\"] = idx\n",
    "\n",
    "    print(f\"Number of splits/chunks {wq}: \", len(wiki_splits))\n",
    "    try:\n",
    "        qdrant_vectorstore.add_documents(documents=wiki_splits)\n",
    "    except Exception as exc:\n",
    "        print(exc)\n",
    "        vs_exception = False\n",
    "\n",
    "    assert (\n",
    "        vs_exception\n",
    "    ), f\"{wq}: There was an error storing all documents in the vectorstore\""
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 153,
     "referenced_widgets": [
      "417ac7d2c28f4d3b82d01349e5e6af34",
      "8eaf9375b40140be9e20a7fde5f99985",
      "677a929849474e188db2ebecab975938",
      "c84789a976a444bdb002aa6347721edd",
      "86da482d7870400c8e8938214889043f",
      "e454dfe23a4a426da3e03f19b957fde4",
      "e3a56a80e2304a158e7e66388d3e4317",
      "0c753dd7107440bc8403c8fbe329cf7b",
      "c9af9c4c2f534f72ad066d28b1fb33e6",
      "d44b1932c69e4589bf6ae3067560d332",
      "40c660d4ac0247928c153df91ae4eb78"
     ]
    },
    "id": "WCxoTgN2nfUa",
    "outputId": "3c06fa0a-8b66-4903-96e2-fdd623e7aade"
   },
   "execution_count": 48,
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "417ac7d2c28f4d3b82d01349e5e6af34"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Number of documents Generative Artificial Intelligence:  4\n",
      "Number of splits/chunks Generative Artificial Intelligence:  80\n",
      "Number of documents Information Retrieval:  4\n",
      "Number of splits/chunks Information Retrieval:  94\n",
      "Number of documents Large Language Models:  4\n",
      "Number of splits/chunks Large Language Models:  86\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Hfs8ro_Ziekd"
   },
   "source": [
    "We'll also augment our collection with some blog entries about Open Domain Question Answering, of which RAG is an approach, and some related topics in case users want to ask how the new Search system works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "id": "K5LdggJw8nBF",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "30b919ac-8e70-4d19-f07e-8f1a5f53abf7"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Number of documents:  3\n",
      "Number of splits:  625\n"
     ]
    }
   ],
   "source": [
    "# @title Online Blog Data Loader\n",
    "web_loader = WebBaseLoader(\n",
    "    web_paths=(\n",
    "        \"https://lilianweng.github.io/posts/2020-10-29-odqa/\",\n",
    "        \"https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/\",\n",
    "        \"https://lilianweng.github.io/posts/2018-06-24-attention/\",\n",
    "    ),\n",
    "    bs_kwargs=dict(\n",
    "        parse_only=bs4.SoupStrainer(\n",
    "            class_=(\"post-content\", \"post-title\", \"post-header\")\n",
    "        )\n",
    "    ),\n",
    ")\n",
    "\n",
    "web_documents = web_loader.load()\n",
    "\n",
    "for idx, text in enumerate(web_documents):\n",
    "    web_documents[idx].metadata[\"doc_num\"] = global_doc_number\n",
    "    web_documents[idx].metadata[\"doc_source\"] = \"WWW\"\n",
    "\n",
    "global_doc_number += 1\n",
    "\n",
    "print(\"Number of documents: \", len(web_documents))\n",
    "\n",
    "web_splits = text_splitter.split_documents(web_documents)\n",
    "\n",
    "for idx, text in enumerate(web_splits):\n",
    "    web_splits[idx].metadata[\"split_id\"] = idx\n",
    "\n",
    "print(\"Number of splits: \", len(web_splits))\n",
    "\n",
    "try:\n",
    "    qdrant_vectorstore.add_documents(documents=web_splits)\n",
    "except Exception as exc:\n",
    "    print(exc)\n",
    "    vs_exception = False"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "joblib.dump(\n",
    "    qdrant_vectorstore,\n",
    "    f\"mids_290_a5/qdrant_vectorstore_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\",\n",
    ")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "X-GQ4sdorvBm",
    "outputId": "726f9e7c-97c6-4688-91cf-9e47ee88ca94"
   },
   "execution_count": 50,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "['mids_290_a5/qdrant_vectorstore_CHUNK_256_OVR_0.pkl']"
      ]
     },
     "metadata": {},
     "execution_count": 50
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4KVUbsDa3BBF"
   },
   "source": [
    "---\n",
    "### 3.3 The Test Data\n",
    "\n",
    "You will want to test the system that you (will) have built. Below we give you a validation set that you could take as labeled data (imagine, your user personas would have had these questions and deemed the answers to be good). We also will give you a test set that only contains questions. (This is the set that we will use to get a feel for how well your RAG system corresponds to our Gold model).\n",
    "\n",
    "Here are is the gold validation set and the test questions. **DO NOT CHANGE OR DELETE!!**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "id": "mxMnyWig7R-T",
    "cellView": "form"
   },
   "outputs": [],
   "source": [
    "# @title\n",
    "validation_questions_answers = {\n",
    "    0: {\n",
    "        \"question\": \"What purpose do large language models serve in the field of natural language processing?\",\n",
    "        \"gold_answer_research\": \"Large language models (LLMs) serve the purpose of enabling general-purpose language generation and other natural language processing tasks such as classification. They achieve this by learning statistical relationships from text documents during computationally intensive self-supervised and semi-supervised training. LLMs can be used for text generation by predicting the next token or word, making them valuable for tasks like speech recognition, machine translation, and information retrieval. Additionally, LLMs have superseded previous models like recurrent neural networks, showcasing their efficiency and effectiveness in NLP tasks.\",\n",
    "        \"gold_answer_marketing\": \"Large language models serve the purpose of improving performance in various natural language processing tasks, such as speech recognition, machine translation, natural language generation, optical character recognition, handwriting recognition, grammar induction, and information retrieval.\",\n",
    "    },\n",
    "    1: {\n",
    "        \"question\": \"How does a large language model learn from text during training?\",\n",
    "        \"gold_answer_research\": \"A large language model learns from text during training by first going through an unsupervised generative 'pretraining' stage where it sets initial parameters using a language modeling objective. Then, it goes through a supervised discriminative 'fine-tuning' stage where it refines its parameters based on annotated examples or task demonstrations. This dual-stage approach allows the model to learn statistical relationships from text documents in a computationally intensive process, enabling it to achieve general-purpose language generation and natural language processing tasks.\",\n",
    "        \"gold_answer_marketing\": \"A large language model learns from text during training by first pretraining on a diverse dataset to acquire general language knowledge, and then fine-tuning on specific tasks or demonstrations to adapt its parameters for more targeted performance.\",\n",
    "    },\n",
    "    2: {\n",
    "        \"question\": \"What are some key architectures behind the development of large language models?\",\n",
    "        \"gold_answer_research\": \"Key architectures behind the development of large language models include the use of self-attention mechanisms, such as those seen in Transformer decoders. These architectures have been applied to tasks like autoregressive language modeling and have led to the dominance of Transformer-based language models in NLP. Models like BERT and GPT-2 have further advanced this paradigm, showcasing the power of large Transformer language models in achieving state-of-the-art results across various NLP tasks. Additionally, architectures like neural-retriever-in-the-loop generative-based models have shown improvements in tasks like open-domain QA and knowledge-grounded dialogue, emphasizing the importance of consistent and engaging responses in long-form generation and multi-turn conversations.\",\n",
    "        \"gold_answer_marketing\": \"Key architectures behind the development of large language models include Transformer-based models such as BERT and GPT-2, which utilize self-attention mechanisms for tasks like autoregressive language modeling and knowledge-grounded dialogue. These models have shown significant success in NLP tasks and have led to advancements in general-purpose language generation and natural language processing.\",\n",
    "    },\n",
    "    3: {\n",
    "        \"question\": \"Can you name some specific large language models and the companies or organizations that have developed them?\",\n",
    "        \"gold_answer_research\": \"Some specific large language models include GPT-3 by OpenAI, Chinchilla by DeepMind, and BERT by Google. OpenAI developed GPT-3, DeepMind developed Chinchilla, and Google developed BERT. These models have been significant advancements in the field of natural language processing.\",\n",
    "        \"gold_answer_marketing\": \"Chinchilla by DeepMind, GPT-3 by OpenAI.\",\n",
    "    },\n",
    "    7: {\n",
    "        \"question\": \"What licensing models have been adopted for the distribution of source-available language models?\",\n",
    "        \"gold_answer_research\": \"Based on the provided context, it seems that licensing models for the distribution of source-available language models have not been explicitly discussed in the referenced papers. However, it is crucial to consider potential licensing options such as open-source licenses (e.g., GPL, MIT) or proprietary licenses when distributing language models to ensure legal compliance and control over usage rights. Additionally, considering the implications of different licensing models on accessibility, collaboration, and commercialization is essential for determining the most suitable approach for sharing language models with the community. Further research or consultation with legal experts may be necessary to explore specific licensing strategies for source-available language models.\",\n",
    "        \"gold_answer_marketing\": \"Answer: Some organizations choose open-sourcing, while others restrict access to a few organizations with resources or offer end-to-end deployment via API.\",\n",
    "    },\n",
    "    8: {\n",
    "        \"question\": \"What are language models and what is their purpose in natural language processing?\",\n",
    "        \"gold_answer_research\": \"Language models are probabilistic models of natural language that help predict or correct text. Their purpose in natural language processing is to assist in various tasks such as speech recognition, machine translation, natural language generation, and information retrieval. By analyzing the performance of human subjects, language models improve the understanding and generation of human-like text.\",\n",
    "        \"gold_answer_marketing\": \"Language models are probabilistic models of natural language that are used in tasks such as speech recognition, machine translation, and natural language generation in natural language processing.\",\n",
    "    },\n",
    "    9: {\n",
    "        \"question\": \"How have language models evolved in terms of architecture, from the 1980s to present times?\",\n",
    "        \"gold_answer_research\": \"Language models have evolved significantly in terms of architecture from the 1980s to present times. In the 1980s, the first statistical language model was proposed, leading to experiments by IBM that identified areas for improvement by observing human subjects. However, it wasn't until 2017 when the transformer architecture was introduced by Google, revolutionizing the field. This development paved the way for models like BERT in 2018, which marked a shift towards large-scale transformer-based language models. These modern architectures, based on self-attention mechanisms, have dominated the field of natural language processing, achieving state-of-the-art performance in various tasks.\",\n",
    "        \"gold_answer_marketing\": \"Language models have evolved from early statistical models in the 1980s to modern transformer architectures, such as BERT and GPT-2, which use self-attention mechanisms and have become dominant in natural language processing tasks.\",\n",
    "    },\n",
    "    11: {\n",
    "        \"question\": \"Can you explain how maximum entropy language models work and what the partition function signifies?\",\n",
    "        \"gold_answer_research\": \"Maximum entropy language models use feature functions to encode the relationship between a word and its n-gram history, aiming to maximize reward while satisfying a KL-constrained objective. The partition function, denoted as Z(x), is crucial in normalizing the probabilities of all possible outputs given the input. It represents the sum of the exponential of the reward function over all possible output sequences, making it computationally expensive to estimate but essential for accurate modeling. The partition function ensures that the model's predicted probabilities sum up to 1, providing a foundation for effective language modeling.\",\n",
    "        \"gold_answer_marketing\": \"Maximum entropy language models encode the relationship between a word and the n-gram history using feature functions. The partition function in this context represents the total probability of all possible outcomes, making it a crucial factor in determining the optimal solution for the reward maximization objective.\",\n",
    "    },\n",
    "    12: {\n",
    "        \"question\": \"What is the benefit of using continuous space embeddings in recurrent neural network language models?\",\n",
    "        \"gold_answer_research\": \"Continuous space embeddings in recurrent neural network language models help alleviate the curse of dimensionality by representing words as non-linear combinations of weights in the embedding space. This approach helps address the data sparsity problem caused by the exponential increase in possible word sequences with vocabulary size. By utilizing continuous space embeddings, neural networks can effectively capture semantic relationships and meaning within the language model.\",\n",
    "        \"gold_answer_marketing\": \"Continuous space embeddings in recurrent neural network language models help alleviate the curse of dimensionality caused by the exponential increase in possible word sequences, reducing data sparsity issues.\",\n",
    "    },\n",
    "    13: {\n",
    "        \"question\": \"What challenges do large language models face in mirroring human cognitive patterns?\",\n",
    "        \"gold_answer_research\": \"Large language models face challenges in mirroring human cognitive patterns because they sometimes learn patterns that humans do not learn, while also failing to learn patterns that humans typically learn. This discrepancy suggests that the models may not be plausible cognitive models, despite matching human performance in some tasks. Further research is needed to address these limitations and improve the alignment of large language models with human cognitive patterns.\",\n",
    "        \"gold_answer_marketing\": \"Large language models sometimes learn patterns that humans do not learn and fail to learn patterns that humans typically do learn.\",\n",
    "    },\n",
    "    16: {\n",
    "        \"question\": \"What factors influenced the development of generative language models by Anthropic?\",\n",
    "        \"gold_answer_research\": \"Several factors influenced the development of generative language models by Anthropic, including the limitations in coding, math, and reasoning capabilities of the initial version Claude, the partnerships with companies like Notion and Quora to enhance the model's capabilities, and the need to address biases, unsafe content, and ethical considerations in training data. Additionally, the reliance on supervised learning and the need for controlled generation in generative models played a role in shaping the development of Anthropic's language models.\",\n",
    "        \"gold_answer_marketing\": \"Factors that influenced the development of generative language models by Anthropic include partnerships with companies like Notion and Quora, limitations in coding, math, and reasoning capabilities in initial models like Claude, and the need to address biases and unsafe content in training datasets.\",\n",
    "    },\n",
    "    17: {\n",
    "        \"question\": \"What is Constitutional AI and how does it affect the functionality of AI systems?\",\n",
    "        \"gold_answer_research\": \"Constitutional AI is an approach developed by Anthropic for training AI systems, particularly language models like Claude, to be harmless and helpful without relying on extensive human feedback. It involves two phases: supervised learning, where the model generates responses to prompts and self-critiques based on a set of guiding principles, and reinforcement learning, where the model is trained with AI-generated feedback according to constitutional principles. This approach enables the training of AI assistants that are both helpful and harmless, with the ability to explain objections to harmful requests, enhancing transparency and reducing the need for human supervision.\",\n",
    "        \"gold_answer_marketing\": \"Constitutional AI is an approach developed by Anthropic for training AI systems, particularly language models like Claude, to be harmless and helpful without relying on extensive human feedback. It involves supervised learning and reinforcement learning phases to guide the model's responses based on a set of guiding principles (a 'constitution'). This approach aims to create AI systems that are both helpful and transparent in their decision-making process, reducing the need for constant human supervision.\",\n",
    "    },\n",
    "    18: {\n",
    "        \"question\": \"How do advances in AI models impact their ability to interact with different types of data, such as images?\",\n",
    "        \"gold_answer_research\": \"Advances in AI models, such as multimodal models like RA-CM3, have significantly improved their ability to interact with different types of data, such as images. These models can refer to external memory, like web data, to increase their knowledge capacity, allowing them to generate correct images from entity-rich captions. Additionally, these models can perform image editing and manually specify examples in-context for better results. The use of large language models, combined with larger datasets and neural networks, has also enhanced their performance in tasks like image generation and text generation.\",\n",
    "        \"gold_answer_marketing\": \"Advances in AI models, such as multimodal models like RA-CM3, allow for better interaction with different types of data, like images, by accessing external memory for increased knowledge capacity and improving performance in tasks like image generation and image editing.\",\n",
    "    },\n",
    "    19: {\n",
    "        \"question\": \"What are the potential trade-offs between AI system alignment with ethical guidelines and practical utility?\",\n",
    "        \"gold_answer_research\": \"The potential trade-offs between AI system alignment with ethical guidelines and practical utility include the risk of reduced performance and usability due to stringent ethical alignment measures, as seen with Claude 2. Users may face limitations and refusal of assistance for benign requests, leading to debates over the 'alignment tax' in AI development. Balancing ethical considerations with practical functionality is crucial to ensure alignment with ethical guidelines without compromising the practical utility of AI systems. Research is needed to find a middle ground that prioritizes ethical alignment while maintaining usability and performance.\",\n",
    "        \"gold_answer_marketing\": \"The potential trade-offs between AI system alignment with ethical guidelines and practical utility include balancing stringent ethical alignment that may reduce usability and performance, ensuring transparency and fairness in alignment processes, and addressing the alignment tax that may impact adoption of AI systems.\",\n",
    "    },\n",
    "    20: {\n",
    "        \"question\": \"How has the token handling capacity changed between different versions of the Claude model?\",\n",
    "        \"gold_answer_research\": \"The token handling capacity has increased with each new version of the Claude model. Claude Instant has a context length of 100,000 tokens, Claude 2.1 doubled this to 200,000 tokens, and Claude 3 Opus default version has a context window of 200,000 tokens but can be expanded to 1 million for specific use cases. This progression shows a trend towards handling larger amounts of text data for improved performance and capabilities.\",\n",
    "        \"gold_answer_marketing\": \"The token handling capacity has increased from Claude to Claude Instant to Claude 2.1, with Claude Instant having a input context length of 100,000 tokens, Claude 2.1 having a context window of 200,000 tokens, and Claude 3 Opus having a context window of 1 million tokens.\",\n",
    "    },\n",
    "    22: {\n",
    "        \"question\": \"In what ways has the Claude model's ability to self-critique and revise its responses enhanced its transparency?\",\n",
    "        \"gold_answer_research\": \"The Claude model's ability to self-critique and revise its responses has enhanced its transparency by allowing for iterative improvements based on past actions and mistakes. Through self-reflection, the model can refine its output by learning from feedback and generating special tokens to signal the need for retrieval or confirm the relevance, support, or completeness of its responses. This process ensures that the model's statements about the world are truthful and accurate, ultimately increasing transparency in its decision-making and reasoning processes.\",\n",
    "        \"gold_answer_marketing\": \"The Claude model's ability to self-critique and revise its responses has enhanced its transparency by allowing it to generate text informed by retrieved passages, criticize the output, and signal the need for retrieval or confirm the output's relevance, support, or completeness. This self-reflection process helps improve the model's accuracy and reliability in generating responses.\",\n",
    "    },\n",
    "    23: {\n",
    "        \"question\": \"How do subsequent versions of Claude compare in terms of their likelihood to produce false statements?\",\n",
    "        \"gold_answer_research\": \"Claude Instant is a faster and lighter version of Claude, with an input context length of 100,000 tokens. In contrast, Claude 3 has faced criticism for its stringent ethical alignment, leading to a debate over the 'alignment tax' in AI development. Users have been refused assistance with benign requests, which has sparked discussions on balancing ethical considerations and practical functionality. This suggests that Claude Instant may have a lower likelihood of producing false statements compared to Claude 3 due to its focus on usability and performance.\",\n",
    "        \"gold_answer_marketing\": \"Claude Instant is a faster, less expensive, and lighter version of Claude with a shorter input context length. Claude 3 has faced criticism for ethical alignment issues that may affect usability and performance.\",\n",
    "    },\n",
    "    24: {\n",
    "        \"question\": \"Who developed the language model family known as Chinchilla?\",\n",
    "        \"gold_answer_research\": \"The Chinchilla language model family was developed by the research team at DeepMind and presented in March 2022. It is named 'Chinchilla' as an advancement over the previous Gopher model family. The Chinchilla family has been trained to investigate the scaling laws of large language models and is designed to outperform GPT-3.\",\n",
    "        \"gold_answer_marketing\": \"The research team at DeepMind developed the language model family known as Chinchilla.\",\n",
    "    },\n",
    "    25: {\n",
    "        \"question\": \"What benchmark did Chinchilla achieve an average accuracy of 67.5% on?\",\n",
    "        \"gold_answer_research\": \"Chinchilla achieved an average accuracy of 67.5% on the MMLU benchmark (Measuring Massive Multitask Language Understanding).\",\n",
    "        \"gold_answer_marketing\": \"Chinchilla achieved an average accuracy of 67.5% on the MMLU benchmark (Measuring Massive Multitask Language Understanding).\",\n",
    "    },\n",
    "    27: {\n",
    "        \"question\": \"What is the relationship between Chinchilla and the Gopher language model families?\",\n",
    "        \"gold_answer_research\": \"The Chinchilla family of transformer models is essentially the same as the Gopher family, with minor modifications and different training optimizers. Chinchilla uses AdamW optimizer while Gopher uses Adam optimizer. Additionally, Chinchilla uses relative positional encoding and RMSNorm instead of absolute positional encoding and LayerNorm used by Gopher. Chinchilla has 70B parameters and outperforms Gopher on the MMLU benchmark by 7%, showcasing an improvement in performance. Both families follow similar naming conventions and were developed to investigate the scaling laws of large language models.\",\n",
    "        \"gold_answer_marketing\": \"Chinchilla is a family of transformer models developed by DeepMind, which is a further development over a previous model family named Gopher. Both model families were trained to investigate the scaling laws of large language models.\",\n",
    "    },\n",
    "    28: {\n",
    "        \"question\": \"What distinguishes the architectures of the Chinchilla and Gopher family models in terms of optimization techniques used?\",\n",
    "        \"gold_answer_research\": \"The main distinction in optimization techniques between the Chinchilla and Gopher family models lies in the choice of optimizers. The Gopher family utilizes the Adam optimizer, whereas the Chinchilla family is trained using the AdamW optimizer. Additionally, the Gopher family employs RMSNorm instead of LayerNorm, and relative positional encoding rather than absolute positional encoding. These differences in optimization techniques contribute to the unique characteristics and performance of each model family.\",\n",
    "        \"gold_answer_marketing\": \"The Chinchilla family uses AdamW optimizer, while the Gopher family uses the Adam optimizer.\",\n",
    "    },\n",
    "    30: {\n",
    "        \"question\": \"What is the recommended strategy for training large autoregressive language models with limited compute resources, as contributed by the Chinchilla team?\",\n",
    "        \"gold_answer_research\": \"The Chinchilla team recommends that the number of training tokens should be doubled for every model size doubling to achieve better results on downstream tasks. They also suggest using larger, higher-quality training datasets to improve performance. Additionally, they mention the importance of balancing model size and efficiency to address computational costs and inference latency limitations. It is advised to focus on Transformer language models and consider sharing model parameters for quick task-switching when deploying as a service.\",\n",
    "        \"gold_answer_marketing\": \"The Chinchilla team recommends doubling the number of training tokens for every model size doubling and using larger, higher-quality training datasets to achieve better results on downstream tasks.\",\n",
    "    },\n",
    "    33: {\n",
    "        \"question\": \"What are some key areas of research in the field of artificial intelligence as reflected in recent academic literature?\",\n",
    "        \"gold_answer_research\": \"Recent academic literature in the field of artificial intelligence reflects key areas of research such as natural language processing with state-of-the-art transformers, feature learning in infinite-width neural networks, diverse beam search for complex scene description, and the development of generative AI models capable of generating text and images. Additionally, research focuses on human preferences in dueling bandits, the use of few-shot learners in language models, and the exploration of knowledge-grounded neural conversation models. These areas of research highlight the advancements in AI technology and its applications across various domains.\",\n",
    "        \"gold_answer_marketing\": \"Some key areas of research in artificial intelligence include natural language processing, deep neural networks, generative AI, AI safety, AI art, reinforcement learning, and language agents alignment.\",\n",
    "    },\n",
    "    34: {\n",
    "        \"question\": \"What are some of the limitations of traditional position encoding methods in the architecture of pre-trained language models (PLMs), and what novel approach does the paper propose to address these issues?\",\n",
    "        \"gold_answer_research\": \"One limitation of traditional position encoding methods in PLMs is that they may not enable length extrapolation of pre-existing models, leading to the need for substantial pre-training costs. The paper proposes a novel approach called Position Interpolation, which extends existing PLMs without deviating far from existing definitions of position encoding or attention mechanisms. This method allows for much extended context windows for text modeling, leading to significant perplexity gains and improved model performance.\",\n",
    "        \"gold_answer_marketing\": \"Traditional position encoding methods in PLMs have limitations in enabling length extrapolation and adapting to extended context windows. The paper proposes a novel approach called Position Interpolation, which generates strong models that can effectively make use of much extended context windows. This method allows for substantial pre-training cost savings and preserves the quality of the original models, even for small context window tasks.\",\n",
    "    },\n",
    "    35: {\n",
    "        \"question\": \"How does the Rotary Position Embedding (RoPE) approach in Transformers differ from the traditional additive method of position embedding with respect to encoding position information?\",\n",
    "        \"gold_answer_research\": \"The RoPE approach in Transformers differs from the traditional additive method of position embedding by being multiplicative instead of additive. While traditional methods add position encoding to context representations, RoPE incorporates relative position information through rotation matrix product. This means that RoPE naturally includes relative position dependency in the self-attention formulation, without altering terms in the expanded formulation like the additive method does. Additionally, RoPE's properties show that it decays as the relative distance between positions increases, providing a clear theoretical interpretation of how position information is encoded.\",\n",
    "        \"gold_answer_marketing\": \"The RoPE approach in Transformers differs from the traditional additive method of position embedding by incorporating relative position information through rotation matrix product instead of altering terms in the expanded formulation of additive position encoding.\",\n",
    "    },\n",
    "    36: {\n",
    "        \"question\": \"What is the significance of comparing the normalized subspace similarity between ∆Wq, ∆Wv, and random Gaussian matrices when analyzing the adaptation of pre-trained language models?\",\n",
    "        \"gold_answer_research\": \"Comparing the normalized subspace similarity between ∆Wq, ∆Wv, and random Gaussian matrices provides insight into the underlying mechanism for adapting pre-trained language models. It helps determine the intrinsic rank of the adaptation matrix ∆W and sheds light on the connection between ∆W and the original weight matrix W. By analyzing these similarities, we can understand how much of the adaptation is specific to the task at hand and how much is influenced by the pre-trained model. This comparison is crucial for optimizing the adaptation process and maximizing downstream performance in NLP tasks.\",\n",
    "        \"gold_answer_marketing\": \"Comparing the normalized subspace similarity between ∆Wq, ∆Wv, and random Gaussian matrices helps understand the underlying mechanism for adapting pre-trained language models. It reveals the intrinsic rank and common singular value directions learned by different runs, shedding light on the fundamental principles of using pre-trained language models for downstream tasks in NLP.\",\n",
    "    },\n",
    "    38: {\n",
    "        \"question\": \"What issues are associated with the homogeneity of language model training contractors, and how might it affect the behavior of the models?\",\n",
    "        \"gold_answer_research\": \"The issues associated with the homogeneity of language model training contractors include potential biases in the labeling process, lack of diverse perspectives leading to limited coverage of sensitive content, and reduced robustness in model performance across different tasks. This homogeneity can affect the behavior of the models by reinforcing certain biases, increasing the risk of harmful content generation, and limiting the models' ability to generalize effectively. To address these issues, it is important to ensure diversity among labelers, incorporate varied perspectives in training data, and implement measures to enhance model robustness and performance across a range of tasks.\",\n",
    "        \"gold_answer_marketing\": \"The homogeneity of language model training contractors can lead to biased or limited perspectives in the data, which may result in the models producing harmful content, gaming objectives, or lacking sensitivity to diverse viewpoints. This can affect the behavior of the models by reinforcing stereotypes, increasing toxicity, and reducing their ability to accurately represent under-represented groups.\",\n",
    "    },\n",
    "    39: {\n",
    "        \"question\": \"What are common research topics and themes found in recent publications about artificial intelligence and natural language processing?\",\n",
    "        \"gold_answer_research\": \"Recent publications in artificial intelligence and natural language processing have covered topics such as transformer models, feature learning in neural networks, attention mechanisms, multi-task benchmark platforms, semantic search using sentence embeddings, cross-task generalization, and question generation for question answering. Themes commonly explored include machine comprehension of text, reinforcement learning algorithms, sentence embeddings, semantic compositionality, reasoning with language models and knowledge graphs, and the gap between neural text and human text. These publications also delve into deep language understanding, retrieval-augmented transformers, image captioning, and open datasets for image-text pairs.\",\n",
    "        \"gold_answer_marketing\": \"Common research topics and themes in recent publications on artificial intelligence and natural language processing include transformer models, attention mechanisms, semantic search, sentence embeddings, and question answering using language models and knowledge graphs.\",\n",
    "    },\n",
    "    41: {\n",
    "        \"question\": \"Question: When conducting demographic and technical assessments of teams or research subjects, what types of data categories are typically collected and analyzed to ensure a comprehensive understanding of the group's composition and the methods used?\",\n",
    "        \"gold_answer_research\": \"When conducting demographic and technical assessments of teams or research subjects, it is important to collect and analyze data categories such as age, gender, education level, professional background, and expertise in specific areas. By gathering information on these categories, you can ensure a comprehensive understanding of the group's composition and the methods used in your assessments. Additionally, it may be helpful to consider factors like cultural background, language proficiency, and geographical location to capture a more nuanced picture of the group being assessed. This detailed approach to data collection and analysis can provide valuable insights for making informed decisions and recommendations based on the gathered information.\",\n",
    "        \"gold_answer_marketing\": \"Answer: Demographic data such as age, gender, education level, and technical data related to skills and experience are typically collected and analyzed for comprehensive understanding.\",\n",
    "    },\n",
    "    43: {\n",
    "        \"question\": \"What kind of tasks can be performed using the datasets described in the provided text, and what are some common features of these datasets?\",\n",
    "        \"gold_answer_research\": \"The datasets described in the provided text can be used for tasks such as question answering, duplicate question retrieval, entity retrieval, citation prediction, query understanding, document understanding, passage retrieval, text summarization, fact verification, and code search. Common features of these datasets include diverse task categories, comprehensive instructions, a wide range of synthetic user personalities and interaction patterns, and a focus on enhancing comprehension of documents to deliver accurate results. Additionally, the datasets cover a variety of domains such as public health, scientific exams, climate, and general knowledge.\",\n",
    "        \"gold_answer_marketing\": \"The datasets described in the provided text can be used for tasks such as question answering, document summarization, duplicate question retrieval, code search, sentence simplification, dialogue generation, body retrieval, caption generation, fact verification, and more. Some common features of these datasets include diverse input-output pairs, incorporation of various knowledge-intensive datasets, and a focus on generating high-quality synthetic data points.\",\n",
    "    },\n",
    "    44: {\n",
    "        \"question\": \"What conclusions can be drawn about the relationship between input prompt toxicity and output toxicity when using different language models and prompts?\",\n",
    "        \"gold_answer_research\": \"Based on the findings presented in the results section, it can be concluded that the relationship between input prompt toxicity and output toxicity varies depending on the language model used and the specific prompt given. When instructed to produce a safe and respectful output, InstructGPT models generate less toxic outputs compared to GPT-3, but this advantage disappears when the respectful prompt is removed. On the other hand, when explicitly prompted to produce a toxic output, InstructGPT outputs are much more toxic than GPT-3 outputs. Additionally, the toxicity of the model outputs is highly correlated with the toxicity of the input prompt, as shown in Figure 39.\",\n",
    "        \"gold_answer_marketing\": \"The study found that when instructed to produce a safe and respectful output, InstructGPT models generate less toxic outputs compared to GPT-3. However, this advantage disappears when the respectful prompt is removed. Interestingly, when explicitly prompted to produce a toxic output, InstructGPT outputs are much more toxic than GPT-3. This suggests that the toxicity of the output is highly correlated with the toxicity of the input prompt.\",\n",
    "    },\n",
    "    45: {\n",
    "        \"question\": \"What are some challenges in training retrieval systems and how are negative samples used to address them?\",\n",
    "        \"gold_answer_research\": \"Training retrieval systems face challenges such as redundancy in retrieved documents and lack of diversity in retrieval. Negative samples, including randomly sampled negatives, denoised hard negatives, and instruction-unfollowing negatives, are crucial for improving system performance. Carefully designed negative samples help the system effectively learn the task, but they can also lead to performance drops in out-of-domain datasets. Combining random samples and challenging negatives during training is key to building a competitive system for both in-domain and out-of-domain retrieval.\",\n",
    "        \"gold_answer_marketing\": \"Some challenges in training retrieval systems include high cost of annotating datasets for new tasks and improving performance in zero-shot settings. Negative samples, such as denoised hard negative documents and instruction-unfollowing negative documents, are used to train retrieval systems effectively and address performance drops in out-of-domain datasets.\",\n",
    "    },\n",
    "    46: {\n",
    "        \"question\": \"What factors have been found to potentially impact the ability of models to follow instructions, based on the analysis provided?\",\n",
    "        \"gold_answer_research\": \"Based on the analysis provided, factors that have been found to potentially impact the ability of models to follow instructions include the human feedback obtained from contractors, which may be influenced by their beliefs, cultural backgrounds, and personal history. Additionally, the model's behavior can be affected by false premises in instructions, tendencies to hedge, and performance degradation with multiple explicit constraints in instructions. The models are also not fully aligned or safe, as they can generate toxic or biased outputs, make up facts, and fail to generate reasonable outputs in some cases.\",\n",
    "        \"gold_answer_marketing\": \"Factors that may impact the ability of models to follow instructions include false premises in instructions, models hedging unnecessarily, performance degradation with multiple constraints in instructions, generation of toxic or biased outputs, and over-generalization leading to refusal of innocuous instructions.\",\n",
    "    },\n",
    "    47: {\n",
    "        \"question\": \"What are some key factors to consider when building a successful multi-task instruction-following retrieval system as identified in the research?\",\n",
    "        \"gold_answer_research\": \"Some key factors to consider when building a successful multi-task instruction-following retrieval system include the need for cross-task interdependence for training a single retriever, the flexibility and zero-shot transfer enabled by instructions compared to task identifiers, and the elimination of the need for hosting multiple task-specific retrievers. Additionally, optimizing the mix and volume of instructional data for diverse tasks is crucial, as well as considering the impact of ranking strategy in data construction. Finally, the effectiveness of the dataset scale in retrieval and the importance of carefully designed negative samples should be taken into account for improved efficiency of instruction-following retrievers.\",\n",
    "        \"gold_answer_marketing\": \"Key factors to consider when building a successful multi-task instruction-following retrieval system include the effectiveness of the dataset scale in retrieval, the diversity in data and model scale, carefully designed negative samples, and the ability to adapt to new tasks via instructions.\",\n",
    "    },\n",
    "    48: {\n",
    "        \"question\": \"What are the benefits of using retrieval-augmented techniques in multimodal language modeling, as demonstrated by the performance of the RA-CM3 model in the document?\",\n",
    "        \"gold_answer_research\": \"The benefits of using retrieval-augmented techniques in multimodal language modeling, as demonstrated by the performance of the RA-CM3 model, include significantly better training efficiency with less training compute, outperforming existing models by using less training data, compute, and parameters. The retrieval augmentation allows the model to focus on learning how to use retrieved documents in context, leading to improved accuracy in classification tasks. Additionally, the RA-CM3 model achieves strong performance in image and caption generation, surpassing existing models like DALL-E and Flamingo despite using fewer resources.\",\n",
    "        \"gold_answer_marketing\": \"The benefits of using retrieval-augmented techniques in multimodal language modeling, as demonstrated by the performance of the RA-CM3 model in the document, include outperforming existing models by using less training data, compute, and parameters, achieving significantly better training efficiency, and improving accuracy in k-shot classification tasks. Additionally, retrieval augmentation allows the model to focus on learning how to use retrieved documents in context, leading to stronger performance in tasks such as image and caption generation.\",\n",
    "    },\n",
    "    50: {\n",
    "        \"question\": \"What methods are typically employed to create training data for embedding models that use task-specific instructions?\",\n",
    "        \"gold_answer_research\": \"To create training data for embedding models that use task-specific instructions, a common method is to combine datasets from different sources, such as the SuperNaturalInstructions dataset with existing collections designed for embedding training. The SuperNaturalInstructions dataset provides natural language instructions, which can be paired with positive and negative examples to form training samples. Additionally, for tasks like classification or similarity, training samples can be constructed by selecting text sequences associated with different classes or similarities. This diverse training data is essential for instruction-based finetuning, which enables the embedding model to learn from a wide range of tasks and domains.\",\n",
    "        \"gold_answer_marketing\": \"Training data for embedding models that use task-specific instructions is typically created by formulating a wide variety of tasks as text-to-text problems, distinguishing good/bad candidate outputs given an input text. This is done by combining datasets with natural language instructions and constructing positive and negative pairs for training.\",\n",
    "    },\n",
    "    51: {\n",
    "        \"question\": \"Question: What are some of the challenges and innovations associated with fine-tuning large language models, and how does the approach discussed in the referenced text aim to address them?\",\n",
    "        \"gold_answer_research\": \"Some challenges associated with fine-tuning large language models include limited access to and manipulation of knowledge, lagging performance on knowledge-intensive tasks, and the need for provenance in decision-making and updating world knowledge. The approach discussed in the referenced text aims to address these challenges by utilizing Retrieval Augmented Generation (RAG), which involves retrieving relevant passages from a corpus to feed to the language model for improved performance in tasks such as question-answering and dialogue. This iterative approach focuses on improving alignment with user intent and fine-tuning models to control sentiment and improve response quality in various language tasks.\",\n",
    "        \"gold_answer_marketing\": \"The challenges with fine-tuning large language models include aligning them with user intent and controlling the quality of generated outputs. The approach discussed in the referenced text aims to address these challenges by using Retrieval Augmented Generation (RAG) to retrieve relevant passages from a corpus and feed them to the language model, improving alignment and performance.\",\n",
    "    },\n",
    "    52: {\n",
    "        \"question\": \"What is a common technique used to address the outlier issue when applying block-wise k-bit quantization to input tensors, and how does it work?\",\n",
    "        \"gold_answer_research\": \"A common technique used to address the outlier issue when applying block-wise k-bit quantization to input tensors is to chunk the input tensor into blocks that are independently quantized, each with their own quantization constant. This approach involves dividing the input tensor into contiguous blocks of size B by flattening the tensor and slicing it into n blocks, where n is determined by the size of the blocks. Each block is then quantized independently using a quantization constant c, which helps prevent outlier values from causing performance degradation.\",\n",
    "        \"gold_answer_marketing\": \"A common technique used to address the outlier issue when applying block-wise k-bit quantization to input tensors is to chunk the input tensor into blocks that are independently quantized, each with their own quantization constant. This helps prevent performance degradation by reducing the impact of outliers on the quantization process.\",\n",
    "    },\n",
    "    54: {\n",
    "        \"question\": \"What considerations or techniques are commonly implemented when setting up finetuning experiments for machine learning models?\",\n",
    "        \"gold_answer_research\": \"When setting up finetuning experiments for machine learning models, it is common to use a two-stage approach. The initial stage involves setting the initial parameters using a language modeling objective. This is followed by a supervised discriminative 'fine-tuning' stage to adapt these parameters to the target task. Additionally, it is typical to train all models using the Adam optimizer and a triangular learning rate scheduler with 10% warmup. Experimentation with different hyperparameters such as number of epochs, peak learning rate, and batch size is also conducted to optimize model performance. Finally, utilizing a mixture of datasets and balancing the sizes of datasets can help improve the robustness and generalization of the finetuned models.\",\n",
    "        \"gold_answer_marketing\": \"Considerations for setting up finetuning experiments for machine learning models commonly include using a language modeling objective for initial parameter setting and supervised discriminative fine-tuning for adapting parameters to the target task. Techniques such as hyperparameter search, Adam optimizer with triangular learning rate scheduler, and balancing dataset sizes through mixing strategies are also commonly implemented. Additionally, freezing some model layers during fine-tuning and incorporating negative examples for contrastive learning can be effective strategies.\",\n",
    "    },\n",
    "    55: {\n",
    "        \"question\": \"What are the implications of the equivalence relation defined in the theoretical analysis of the DPO model for understanding the relationship between reward functions in reinforcement learning?\",\n",
    "        \"gold_answer_research\": \"The equivalence relation defined in the theoretical analysis of the DPO model implies that two reward functions are considered equivalent if they differ by a constant function. This means that the class of learned reward models is not constrained by this reparameterization, allowing for the exact recovery of the optimal policy. Understanding this relationship between reward functions in reinforcement learning helps in defining a unique reward function within each equivalence class, which is crucial for optimizing policies under existing models of human preferences. It also highlights the generality and flexibility in the reward model due to the proposed reparameterization.\",\n",
    "        \"gold_answer_marketing\": \"The equivalence relation defined in the theoretical analysis of the DPO model shows that two reward functions are considered equivalent if they differ by a fixed function. This implies that different reward functions can lead to the same optimal policy, allowing for flexibility in designing reward models in reinforcement learning.\",\n",
    "    },\n",
    "    59: {\n",
    "        \"question\": \"Considering the structure and content of the provided text, what guidelines should be used to evaluate the effectiveness of a summary or chatbot response in this context?\",\n",
    "        \"gold_answer_research\": \"To evaluate the effectiveness of a summary or chatbot response in this context, guidelines should include assessing the faithfulness of the answer to the retrieved context, the relevance of the answer to the question, and the focus of the retrieved context. Additionally, consider using quality metrics such as answer relevancy to rank responses based on how directly they address the question and avoid redundant or incomplete information. Lastly, take into account the performance of different tasks such as summarization, citation prediction, and passage ranking to determine the overall effectiveness of the response.\",\n",
    "        \"gold_answer_marketing\": \"Answer: Evaluate based on faithfulness, answer relevance, and context relevance.\",\n",
    "    },\n",
    "    60: {\n",
    "        \"question\": \"What are some recent methods and technologies that have been developed to enhance the capabilities and performance of natural language processing models?\",\n",
    "        \"gold_answer_research\": \"Recent methods and technologies developed to enhance natural language processing models include retrieval-augmented multimodal language modeling, which outperforms existing models with less training data and parameters. Another advancement is the use of feature learning in infinite-width neural networks to improve performance. Additionally, embedding techniques in NLP have been developed to map words or phrases to real number vectors, enhancing the model's understanding of language. These innovations have led to improvements in tasks like query reformulation, document ranking, and fine-tuning larger language models for various applications.\",\n",
    "        \"gold_answer_marketing\": \"Recent methods and technologies include retrieval-augmented language models, feature learning in infinite-width neural networks, and word embeddings.\",\n",
    "    },\n",
    "    61: {\n",
    "        \"question\": \"What are some potential directions for future work mentioned in the document related to enhancing question-answering techniques for document-oriented tasks?\",\n",
    "        \"gold_answer_research\": \"One potential direction for future work mentioned in the document is the development of multi-modal approaches that incorporate table and figure information into GPT-4 question-answering for documents. Another direction is to incorporate question type in the PDFTriage approach to improve the efficiency and efficacy of the approach. Additionally, the document suggests further research in document-grounded, information-seeking question answering, which the dataset is designed to facilitate.\",\n",
    "        \"gold_answer_marketing\": \"Some potential future directions mentioned in the document include developing multi-modal approaches that incorporate table and figure information into question-answering for documents, and incorporating question type in the PDFTriage approach to improve efficiency and efficacy.\",\n",
    "    },\n",
    "    62: {\n",
    "        \"question\": \"What information would you expect to find in section 2 of a document, based on the types of questions classified under Summarization?\",\n",
    "        \"gold_answer_research\": \"Based on the types of questions classified under Summarization, you would expect to find key takeaways, concise summaries, and specific content extraction related to different sections of the document in section 2. The section likely contains detailed summaries of specific parts of the document, along with structured metadata representation and instructions for summarizing the content effectively. It may also include guidelines for extracting specific information and rewriting text for clarity and conciseness.\",\n",
    "        \"gold_answer_marketing\": \"Based on the types of questions classified under Summarization, you would expect to find key takeaways, concise summaries, and specific content extraction related to the document in section 2.\",\n",
    "    },\n",
    "    63: {\n",
    "        \"question\": \"What are the main advantages and attention mechanisms that contribute to the enhanced performance and efficiency of the newly introduced language model as compared to its predecessors?\",\n",
    "        \"gold_answer_research\": \"The main advantages of the newly introduced language model include utilizing retrieval-augmentation to incorporate external knowledge, which improves prediction accuracy. Additionally, the model employs attention mechanisms that allow for better understanding of dependencies between source and target sequences, leading to more informed predictions. These attention mechanisms have been extended from machine translation to various other fields, enhancing the model's adaptability and performance across different tasks. Finally, the model's use of self-attention mechanisms enables better contextual representation learning, parallelization, and modeling of longer intra-token relations, improving efficiency and performance compared to previous models.\",\n",
    "        \"gold_answer_marketing\": \"The main advantages of the newly introduced language model include the use of retrieval-augmented mechanisms, attention mechanisms, and context representation learning, which contribute to enhanced performance and efficiency compared to its predecessors.\",\n",
    "    },\n",
    "    64: {\n",
    "        \"question\": \"What criteria are used to assess the quality of recommendations provided by different language models in a comparison study?\",\n",
    "        \"gold_answer_research\": \"In a comparison study of language models, criteria such as sentence relevance, lexical accuracy, and contextual understanding are used to assess the quality of recommendations. Different tasks may benefit from different evaluation measures, such as STRINC, LEXICAL, and CXMI. Additionally, template selection plays a vital role in the quality of recommendations, with deliberate template design being important for tasks like query suggestion. The overall quality of recommendations is often judged using a Likert scale, along with metadata collection for each model output.\",\n",
    "        \"gold_answer_marketing\": \"The criteria used to assess the quality of recommendations provided by different language models in a comparison study include comparing to human-created benchmarks, examining intrinsic character, comparing two models, investigating rate of learning, and analyzing learning curves.\",\n",
    "    },\n",
    "    65: {\n",
    "        \"question\": \"What approaches have been proposed to enhance the task performance of language models while considering the trade-offs such as runtime efficiency, robustness to irrelevant context, and attribution quality?\",\n",
    "        \"gold_answer_research\": \"Several approaches have been proposed to enhance the task performance of language models while considering trade-offs. These include using compression and selective augmentation methods to decrease the propensity of models to generate toxic or biased outputs. Adversarial setups have been suggested where labelers find worst-case behaviors of the model and add them to the dataset. Additionally, models like BART and T5 leverage bi-directional attention to achieve stronger performance on both discriminative and generative tasks. These methods aim to balance model performance with considerations such as runtime efficiency, robustness to irrelevant context, and attribution quality.\",\n",
    "        \"gold_answer_marketing\": \"Approaches proposed to enhance language model task performance include compression and selective augmentation, adversarial set-ups for labeling worst-case behaviors, retrieval-augmented models, and extending existing models to enable length extrapolation while maintaining quality.\",\n",
    "    },\n",
    "    67: {\n",
    "        \"question\": \"What metrics are commonly used to compare the performance of language models in various tasks, as outlined in an experimental results table?\",\n",
    "        \"gold_answer_research\": \"Common metrics used to compare the performance of language models in various tasks, as outlined in an experimental results table, include Exact Match and Unigram F1. These metrics have become standard in evaluating language models. Additionally, other metrics such as BLEU score, FactScore (factuality), precision, and recall are also commonly used to assess the performance of language models across different tasks. It is important to consider a variety of metrics to get a comprehensive understanding of the effectiveness of a language model in different contexts.\",\n",
    "        \"gold_answer_marketing\": \"The metrics commonly used to compare the performance of language models in various tasks are Exact Match and Unigram F1.\",\n",
    "    },\n",
    "    69: {\n",
    "        \"question\": \"What is the role of manual assessment in the validation of language model predictions according to the text provided?\",\n",
    "        \"gold_answer_research\": \"Manual assessment plays a crucial role in the validation of language model predictions. The engineers evaluate the quality of model outputs by having labelers rate them on test sets consisting of prompts from held-out customers. This manual assessment helps ensure that the models are aligned with a broad distribution of language tasks and can identify any behavioral issues that may arise from misalignment. Additionally, human annotators find that certain reflection token predictions are aligned with their assessments, providing valuable insights into the accuracy and effectiveness of the models.\",\n",
    "        \"gold_answer_marketing\": \"Answer: Manual assessment plays a key role in evaluating the quality of language model predictions by having labelers rate the model outputs and comparing them to prompts from held-out customers.\",\n",
    "    },\n",
    "    70: {\n",
    "        \"question\": \"What are the general steps outlined for training a language model in the document, and how is the training data for the generator language model collected and utilized?\",\n",
    "        \"gold_answer_research\": \"The document outlines the general steps for training a language model, including incorporating retrieved documents into the main input sequence and optimizing the loss function to train the generator. The training data for the generator language model is collected through various techniques such as supervised fine-tuning, critic learning, and custom retrievers for downstream tasks. The collected data is used to train the generator on specific tasks like summarization, machine reading comprehension, and natural language to SQL translation, improving performance on those tasks.\",\n",
    "        \"gold_answer_marketing\": \"The general steps for training a language model include fine-tuning on specific datasets, filtering pretraining data, and using critic learning. Training data for the generator language model is collected from open-access NLP papers and used for downstream conditional text generation tasks.\",\n",
    "    },\n",
    "    73: {\n",
    "        \"question\": \"What are the three main categories used to refine language model abilities in understanding and executing search tasks according to the given document?\",\n",
    "        \"gold_answer_research\": \"The three main categories used to refine language model abilities in understanding and executing search tasks are query understanding, document understanding, and query-document relationship understanding. Tasks within these categories focus on interpreting queries, comprehending documents, and understanding the relationships between queries and documents. This approach aims to enhance the models' performance in interpreting and responding to search-related instructions effectively, improving their utility in complex information retrieval scenarios.\",\n",
    "        \"gold_answer_marketing\": \"The three main categories used to refine language model abilities in understanding and executing search tasks are query understanding, document understanding, and query-document relationship understanding.\",\n",
    "    },\n",
    "    74: {\n",
    "        \"question\": \"What are some of the emerging research topics and challenges in the field of natural language processing and information retrieval according to recent academic conferences and publications?\",\n",
    "        \"gold_answer_research\": \"Recent academic conferences and publications have highlighted emerging research topics and challenges in natural language processing and information retrieval. Some key areas of focus include efficient retrieval augmented generation, unsupervised dense information retrieval with contrastive learning, citation-informed transformers, and knowledge refinement via interaction between search engines and large language models. Additionally, challenges such as zero-shot retrieval, semantic search using GPT sentence embeddings, and prompt-based effective input reformulation for legal case retrieval have been identified as important research directions. These topics reflect the ongoing advancements and complexities in the field, driving innovation and progress in NLP and IR research.\",\n",
    "        \"gold_answer_marketing\": \"Some emerging research topics and challenges in the field of natural language processing and information retrieval include efficient generation from unstructured knowledge, semantic code search evaluation, unsupervised dense information retrieval, context-aware document term weighting, knowledge refinement through interaction with large language models, and investigating the effectiveness of large language models in search re-ranking.\",\n",
    "    },\n",
    "    75: {\n",
    "        \"question\": \"Question: How do models with different fine-tuning strategies compare in terms of accuracy and F1 score for fact verification tasks?\",\n",
    "        \"gold_answer_research\": \"Models with different fine-tuning strategies are compared in terms of accuracy and F1 score for fact verification tasks. The introduction of LLMs has led to notable developments, with some studies leveraging prompting methods to apply LLMs in IR tasks. However, not all LLMs consistently outperform fine-tuned smaller models. For example, RankGPT based on gpt-3.5-turbo underperforms monoBERT in certain scenarios. Fine-tuning is not strictly necessary for models like GPT3, which has been evaluated on closed book question answering tasks without any updates or fine-tuning.\",\n",
    "        \"gold_answer_marketing\": \"Models with different fine-tuning strategies have shown mixed results in terms of accuracy and F1 score for fact verification tasks. Some studies have found that large language models (LLMs) outperform smaller fine-tuned models, while others have reported inconsistent performance. Factors such as task complexity and the need for prompt methods to apply LLMs in information retrieval tasks can also impact the comparison.\",\n",
    "    },\n",
    "    76: {\n",
    "        \"question\": \"What components does a fact verification task typically involve in order to assess the accuracy of a given statement?\",\n",
    "        \"gold_answer_research\": \"A fact verification task typically involves assessing the relationship between a claim and the evidence provided, analyzing if there is enough information for a conclusive judgment. This task requires a detailed understanding of the claim and evidence to determine if it is supported or refuted. The use of performance metrics based on including gold answers in model generations instead of exact matching can help search engines deliver accurate and relevant results. Additionally, incorporating lexical measures and verification functions can aid in determining the accuracy of statements.\",\n",
    "        \"gold_answer_marketing\": \"A fact verification task typically involves assessing the relationship between a claim and supporting evidence to determine accuracy.\",\n",
    "    },\n",
    "    78: {\n",
    "        \"question\": \"What are the key factors that determine the performance of HALO-aligned models compared to non-HALO models, according to the results presented in the analysis?\",\n",
    "        \"gold_answer_research\": \"According to the analysis presented, the key factors that determine the performance of HALO-aligned models compared to non-HALO models include the specific alignment method used (such as DPO and PPO variant), the model size (significant gap at 13B+ model sizes), and the ability to match or exceed the generation quality of SFT target sequences. Additionally, the study suggests that the cost of increasing model alignment is modest relative to pretraining, and that the modeling of human biases in HALOs may have practical benefits in improving overall performance.\",\n",
    "        \"gold_answer_marketing\": \"The key factor that determines the performance of HALO-aligned models compared to non-HALO models is the model size, with HALO-aligned models generally outperforming non-HALO models at larger sizes (13B+ model sizes).\",\n",
    "    },\n",
    "    80: {\n",
    "        \"question\": \"How does the performance of KTO compare to DPO in model alignment, and what are the potential implications for data usage and training efficiency?\",\n",
    "        \"gold_answer_research\": \"Based on the provided data and experiments, KTO consistently outperforms DPO in model alignment, even with restrictions such as using only one output per input. This suggests that KTO can achieve higher win rates and improve performance across various benchmarks compared to DPO. The implications of this performance difference include the ability to achieve quality generation results with significantly fewer desirable examples, potentially leading to more efficient data usage and training processes. This indicates that KTO may offer a more efficient and effective approach to model alignment compared to DPO.\",\n",
    "        \"gold_answer_marketing\": \"KTO outperforms DPO in model alignment with up to 90% fewer examples. This suggests that KTO can achieve high performance even with imbalanced data, potentially leading to more efficient training processes.\",\n",
    "    },\n",
    "    81: {\n",
    "        \"question\": \"What are some common approaches to building an open-domain question answering system?\",\n",
    "        \"gold_answer_research\": \"Some common approaches to building an open-domain question answering system include using the RAG model, which minimizes the negative log-likelihood of answers, and comparing it to extractive QA paradigms that rely on non-parametric knowledge retrieval. Another approach is to incorporate question rewriting techniques to make open-domain QA more conversational. Additionally, utilizing datasets like QASPER, which contain questions requiring complex reasoning, can improve the performance of the system. References to papers by Anantha et al. and Asai et al. provide further insights into building ODQA systems.\",\n",
    "        \"gold_answer_marketing\": \"Common approaches to building an open-domain question answering system include using retrieval over a knowledge base and incorporating the retrieved content as part of the prompt. Other methods involve pretraining models on large amounts of text data and fine-tuning them for question answering tasks.\",\n",
    "    },\n",
    "    82: {\n",
    "        \"question\": \"What is the difference between open-book and closed-book question answering?\",\n",
    "        \"gold_answer_research\": \"Open-book question answering involves the use of external sources of knowledge, such as Wikipedia, to retrieve information and generate a response. In contrast, closed-book question answering relies on pre-trained language models that have memorized factual knowledge within their parameters to generate responses without explicit context. Closed-book QA can be seen as analogous to a closed-book exam where no external resources are allowed. The key distinction lies in the reliance on external knowledge sources for open-book QA versus internal memorized knowledge for closed-book QA.\",\n",
    "        \"gold_answer_marketing\": \"Open-book question answering involves using external sources of knowledge to answer questions, while closed-book question answering relies on pre-trained language models to provide answers without explicit context.\",\n",
    "    },\n",
    "    84: {\n",
    "        \"question\": \"What are the basic components of the Retriever-Reader framework in open-domain QA?\",\n",
    "        \"gold_answer_research\": \"The basic components of the Retriever-Reader framework in open-domain QA include a retriever model, which fetches relevant information based on input prompts efficiently using FAISS. The retriever component is responsible for retrieving contextually relevant documents or evidence blocks based on the input question. The reader component then processes this retrieved information to generate answers to the questions posed. This framework combines information retrieval and machine reading comprehension to achieve state-of-the-art results in open-domain question answering tasks.\",\n",
    "        \"gold_answer_marketing\": \"The basic components of the Retriever-Reader framework in open-domain QA are the retriever and the reader components, which can be set up and trained independently or jointly trained end-to-end. The retriever component automatically fetches relevant information based on input prompts, while the reader component processes and comprehends the retrieved information to answer questions.\",\n",
    "    },\n",
    "    85: {\n",
    "        \"question\": \"How is the TF-IDF model used in question answering retrieval systems?\",\n",
    "        \"gold_answer_research\": \"In question answering retrieval systems, the TF-IDF model is used to represent queries and documents as bag-of-word vectors with terms weighted by term frequency multiplied by inverse document frequency. This allows for efficient non-learning-based search engine operations based on the vector space model. The TF-IDF model helps in calculating the relevance of documents to queries by measuring the importance of terms in the context of the entire document collection. This classic information retrieval approach aids in retrieving relevant information to answer questions accurately and efficiently.\",\n",
    "        \"gold_answer_marketing\": \"The TF-IDF model is used in question answering retrieval systems to weight terms in queries and documents based on their importance in determining relevance.\",\n",
    "    },\n",
    "    86: {\n",
    "        \"question\": \"Can neural networks enhance the process of information retrieval in QA systems?\",\n",
    "        \"gold_answer_research\": \"Neural networks, such as MLP, LSTM, and bidirectional LSTM, can be used to learn dense representations of text for information retrieval in QA systems. These approaches, known as 'Neural IR', are a new category of methods that can improve performance in retrieval problems. The introduction of neural retrievers in recent QA literature has shown to outperform traditional word-similarity-based architectures, such as BM25, and can scale to handle knowledge-grounded dialogue tasks effectively. Additionally, incorporating pre-trained retrievers in QA systems has been shown to enhance the performance of generative language models.\",\n",
    "        \"gold_answer_marketing\": \"Yes, neural networks can enhance the process of information retrieval in QA systems by improving performance in open-domain QA tasks and enabling the generation of more accurate answers.\",\n",
    "    },\n",
    "    87: {\n",
    "        \"question\": \"What is the importance of fine-tuning in the context of QA data for open-domain question answering models?\",\n",
    "        \"gold_answer_research\": \"Fine-tuning is important in the context of QA data for open-domain question answering models because it allows the model to adapt and improve its performance on specific QA datasets. By fine-tuning the model with common QA datasets, engineers can optimize the model's ability to answer questions accurately. However, there is a concern about the significant overlap between questions in the train and test sets of public QA datasets, which could affect the generalization ability of the fine-tuned models. Engineers should carefully consider this overlap and potentially explore ways to mitigate its impact during the fine-tuning process to ensure the model's effectiveness in real-world applications.\",\n",
    "        \"gold_answer_marketing\": \"Fine-tuning is important in the context of QA data for open-domain question answering models to improve search task performance and the ability to generalize to unseen datasets.\",\n",
    "    },\n",
    "    88: {\n",
    "        \"question\": \"How does pre-training with tasks like the Inverse Cloze Task benefit open-domain question answering models?\",\n",
    "        \"gold_answer_research\": \"Pre-training with tasks like the Inverse Cloze Task benefits open-domain question answering models by improving the retrieval process over a knowledge base. By predicting the context given a sentence, the model can better understand the relationship between the question and the evidence. This approach helps in incorporating retrieved content effectively into the prompt, leading to higher accuracy in the question answering task. Additionally, using models pretrained with ICT can enhance the overall performance of the QA system by providing a better understanding of the context.\",\n",
    "        \"gold_answer_marketing\": \"Pre-training with tasks like the Inverse Cloze Task benefits open-domain question answering models by improving retrieval and generation steps, ultimately enhancing the accuracy of the process.\",\n",
    "    },\n",
    "    89: {\n",
    "        \"question\": \"What is the main goal of prompt engineering in language models?\",\n",
    "        \"gold_answer_research\": \"The main goal of prompt engineering in language models is to effectively steer the behavior of the model towards desired outcomes without updating the model weights. This is achieved by composing and formatting prompts in a way that maximizes the model's performance on a specific task. Prompt engineering involves treating prompts as trainable parameters and optimizing them directly on the embedding space through methods like AutoPrompt, Prefix-Tuning, P-tuning, and Prompt-Tuning. The ultimate aim is to enhance the model's performance and alignment with user-defined tasks.\",\n",
    "        \"gold_answer_marketing\": \"The main goal of prompt engineering in language models is to steer the behavior of the model for desired outcomes without updating the model weights.\",\n",
    "    },\n",
    "    91: {\n",
    "        \"question\": \"What are some known biases that can affect the performance of few-shot classification in LLMs?\",\n",
    "        \"gold_answer_research\": \"Some known biases that can affect the performance of few-shot classification in LLMs include majority label bias, recency bias, and common token bias. Majority label bias occurs when the distribution of labels among examples is unbalanced, recency bias refers to the tendency for the model to repeat the label at the end, and common token bias indicates that LLM tends to produce common tokens more often than rare tokens. These biases can contribute to high variance in few-shot classification tasks and may impact the model's ability to generalize effectively.\",\n",
    "        \"gold_answer_marketing\": \"Some known biases that can affect the performance of few-shot classification in LLMs are majority label bias, recency bias, and common token bias.\",\n",
    "    },\n",
    "    92: {\n",
    "        \"question\": \"Why might increasing model size not reduce variance in model performance with varying prompts?\",\n",
    "        \"gold_answer_research\": \"Increasing model size may not necessarily reduce variance in model performance with varying prompts because the model's ability to generalize and adapt to different prompts is not solely dependent on its size. Factors such as the quality and relevance of the training examples, the learning rate or schedule, and the model's sensitivity to different hyperparameters can also play a significant role in determining performance variability. Additionally, the complexity of the task or dataset being used for training can impact how effectively the model scales with size. It is essential to consider these factors holistically when optimizing model performance rather than relying solely on increasing model size.\",\n",
    "        \"gold_answer_marketing\": \"Increasing model size may not reduce variance in model performance with varying prompts because the same order of prompts may work well for one model but poorly for another. Additionally, when the validation set is limited, choosing the order of prompts that prevents the model from producing extremely unbalanced predictions or being overconfident can also affect performance.\",\n",
    "    },\n",
    "    93: {\n",
    "        \"question\": \"What is the benefit of instruction-based finetuning in language models?\",\n",
    "        \"gold_answer_research\": \"Instruction-based finetuning improves models' ability to generalize to unseen domains and tasks by providing task-specific representations that can be used for many downstream language tasks without additional training. This method also allows pretrained language models to follow instructions provided in prompts, enabling them to generate the desired output given specific inputs. Additionally, instruction finetuning helps transform raw pretrained LLMs into chatbot-like models, making finetuning more accessible and common, particularly for researchers with limited resources. Overall, the benefit of instruction-based finetuning is improved model performance, enhanced generalizability, and reduced communication costs in aligning with human intentions.\",\n",
    "        \"gold_answer_marketing\": \"The benefit of instruction-based finetuning in language models is improved ability to generalize to unseen domains and tasks, without the need for additional training.\",\n",
    "    },\n",
    "    94: {\n",
    "        \"question\": \"Can you describe a situation where retrieval-based methods would be necessary to enhance language model performance?\",\n",
    "        \"gold_answer_research\": \"Retrieval-based methods are necessary to enhance language model performance in scenarios where the model needs to generate accurate and informative responses for entity-rich queries, such as 'George Washington standing in front of the Eiffel Tower.' In such cases, incorporating a retrieval module can provide additional context and relevant information to improve the model's understanding and generation of the desired output. Additionally, retrieval-based methods are crucial for question answering tasks, where the model needs to access external knowledge sources to provide accurate and comprehensive answers. By utilizing retrieval mechanisms, the language model can benefit from a wider range of information and improve its performance in handling complex and ambiguous queries effectively.\",\n",
    "        \"gold_answer_marketing\": \"Retrieval-based methods are necessary to enhance language model performance in tasks like question answering, where incorporating additional information from external sources can improve the model's ability to generate accurate and relevant responses.\",\n",
    "    },\n",
    "    95: {\n",
    "        \"question\": \"What is the Chain-of-Thought prompting technique and for which types of tasks is it particularly beneficial?\",\n",
    "        \"gold_answer_research\": \"Chain-of-Thought (CoT) prompting is a technique that generates reasoning chains or rationales step by step to lead to a final answer, benefiting complicated reasoning tasks using large models with more than 50B parameters. It can be implemented through iterative Monte Carlo search methods or through a three-step process called augment-prune-select. CoT is particularly beneficial for enhancing model performance on complex tasks by decomposing them into smaller and simpler steps, shedding light on the model's thinking process. Task decomposition in CoT can be done with simple prompting, task-specific instructions, or human inputs.\",\n",
    "        \"gold_answer_marketing\": \"Chain-of-Thought (CoT) prompting is a technique that generates reasoning chains or rationales step by step to lead to a final answer. It is particularly beneficial for complicated reasoning tasks when using large models with more than 50B parameters. Simple tasks only benefit slightly from CoT prompting.\",\n",
    "    },\n",
    "    96: {\n",
    "        \"question\": \"How do augmented language models with external tools differ from regular models in functionality?\",\n",
    "        \"gold_answer_research\": \"Augmented language models with external tools, such as TALM and Toolformer, are fine-tuned to learn how to use external tool APIs, expanding their capabilities beyond traditional language processing tasks. These models are trained to incorporate external tool API calls in order to improve the quality of their outputs, allowing them to perform tasks like speech recognition, machine translation, and information retrieval more effectively. By leveraging external tools, these models have the ability to access and utilize a wider range of resources and functionalities, enhancing their overall performance and versatility compared to regular language models.\",\n",
    "        \"gold_answer_marketing\": \"Augmented language models with external tools differ from regular models by fine-tuning a LM to use external tool APIs, expanding the dataset to improve model outputs and enhancing tasks like speech recognition, machine translation, and natural language generation.\",\n",
    "    },\n",
    "    97: {\n",
    "        \"question\": \"What can be inferred about the utilization of attention in neural networks?\",\n",
    "        \"gold_answer_research\": \"Attention mechanisms in neural networks play a crucial role in allowing models to focus on specific parts of input data when making predictions or generating outputs. By assigning importance weights to different elements, such as pixels in an image or words in a sentence, attention helps the model to attend to relevant information and make more accurate predictions. The use of attention can improve the interpretability of neural networks by showing which parts of the input data are being focused on during the prediction process. Additionally, attention mechanisms, like multi-head attention, can enhance model performance by allowing the model to jointly attend to information from different representation subspaces at different positions.\",\n",
    "        \"gold_answer_marketing\": \"Attention in neural networks allows the model to focus on specific parts of input data, such as images or text, in order to make predictions or generate output. It helps the model to learn relationships and correlations between different elements and improve performance in tasks like image captioning or language translation.\",\n",
    "    },\n",
    "    101: {\n",
    "        \"question\": \"Can the use of attention mechanisms in deep learning models be applied to both machine translation and computer vision?\",\n",
    "        \"gold_answer_research\": \"Yes, attention mechanisms in deep learning models have shown success in both machine translation and computer vision tasks. In machine translation, attention allows the model to capture dependencies between source and target sequences regardless of distance, leading to improved translation quality. Similarly, in computer vision, attention mechanisms have been used to focus on relevant parts of an image during caption generation, showcasing the ability to handle details and global dependencies effectively. Therefore, utilizing attention in both domains can enhance the performance of deep learning models significantly.\",\n",
    "        \"gold_answer_marketing\": \"Yes, attention mechanisms in deep learning models can be applied to both machine translation and computer vision.\",\n",
    "    },\n",
    "    102: {\n",
    "        \"question\": \"What are the potential benefits of incorporating self-attention mechanisms into Generative Adversarial Networks (GANs)?\",\n",
    "        \"gold_answer_research\": \"Incorporating self-attention mechanisms into GANs can help the generator and discriminator better model relationships between spatial regions, leading to improved generation of detailed and realistic images. This is particularly useful for capturing global dependencies and enhancing the performance of transformer architectures. Additionally, self-attention can enable the model to assess its own predictions after each generated segment, allowing for customizable decoding algorithms to meet specific constraints or user preferences. Overall, self-attention in GANs can enhance detail handling and overall performance.\",\n",
    "        \"gold_answer_marketing\": \"Incorporating self-attention mechanisms into GANs can help the generator and discriminator better model relationships between spatial regions, leading to improved performance in handling details and capturing global dependencies.\",\n",
    "    },\n",
    "    103: {\n",
    "        \"question\": \"How does the transformer model variate from traditional sequence-aligned recurrent architectures?\",\n",
    "        \"gold_answer_research\": \"The transformer model differs from traditional sequence-aligned recurrent architectures by not having a recurrent or convolutional structure. Instead, it heavily relies on self-attention mechanisms for processing sequences. This lack of recurrence and convolution, even with positional encoding, weakly incorporates sequential order, which can be a drawback for tasks sensitive to positional dependencies. Additionally, the transformer's architecture includes embedding layers, sinusoid-wave-based positional encoding, and softmax and linear layers in the final decoder output to maintain position information and facilitate processing of long sequences efficiently.\",\n",
    "        \"gold_answer_marketing\": \"The transformer model differs from traditional sequence-aligned recurrent architectures by not having a recurrent or convolutional structure, and instead making heavy use of self-attention. This allows for handling very long sequences efficiently and achieving better performance on tasks involving long texts.\",\n",
    "    },\n",
    "    104: {\n",
    "        \"question\": \"What implications does the concept of a Neural Turing Machine have for the theoretical power of neural networks?\",\n",
    "        \"gold_answer_research\": \"The concept of a Neural Turing Machine (NTM) expands the theoretical power of neural networks by incorporating external memory storage, allowing for more complex computations and tasks. This mimics the Turing machine tape, enabling the neural network to control operation heads for reading and writing to the tape. However, the finite memory in NTM suggests it may resemble more of a 'Neural von Neumann Machine,' limiting its mathematical limitlessness seen in traditional Turing machines. Overall, the addition of external memory in NTM enhances the capabilities and potential applications of neural networks in solving more advanced problems.\",\n",
    "        \"gold_answer_marketing\": \"The concept of a Neural Turing Machine suggests that neural networks can be equipped with external memory storage for more complex operations, potentially increasing their theoretical power.\",\n",
    "    },\n",
    "}\n",
    "\n",
    "\n",
    "test_questions = {\n",
    "    4: {\n",
    "        \"question\": \"When was the transformer architecture introduced, and by which organization?\"\n",
    "    },\n",
    "    5: {\n",
    "        \"question\": \"How has the accessibility of powerful language models, such as GPT-3 and GPT-4, been controlled by their developers?\"\n",
    "    },\n",
    "    6: {\n",
    "        \"question\": \"What benchmarks or ratings are used to compare the capabilities of different language models?\"\n",
    "    },\n",
    "    10: {\n",
    "        \"question\": \"What are some of the primary applications for language models in technology and computing?\"\n",
    "    },\n",
    "    14: {\n",
    "        \"question\": \"How are language models typically evaluated and what benchmarks are used for this purpose?\"\n",
    "    },\n",
    "    15: {\n",
    "        \"question\": \"What datasets are available for evaluating language processing systems?\"\n",
    "    },\n",
    "    21: {\n",
    "        \"question\": \"What collaborations with other companies have contributed to the development of Claude's capabilities?\"\n",
    "    },\n",
    "    26: {\n",
    "        \"question\": \"According to DeepMind, how should the number of training tokens change relative to the model size?\"\n",
    "    },\n",
    "    29: {\"question\": \"How do the sizes of models in the Gopher family range?\"},\n",
    "    31: {\n",
    "        \"question\": \"What type of model architecture do the Gopher and Chinchilla families belong to?\"\n",
    "    },\n",
    "    32: {\n",
    "        \"question\": \"Can you name the author who wrote the novels A Farewell to Arms and The Sun Also Rises?\"\n",
    "    },\n",
    "    37: {\n",
    "        \"question\": \"What are the key advantages of InstructGPT models over GPT-3 models according to the findings in the research?\"\n",
    "    },\n",
    "    40: {\n",
    "        \"question\": \"What metrics are used to compare the performance of different models on training and validation splits according to the document provided?\"\n",
    "    },\n",
    "    42: {\n",
    "        \"question\": \"What types of evaluation metrics are commonly used to assess the accuracy of answers in AI-driven question and answer datasets?\"\n",
    "    },\n",
    "    49: {\n",
    "        \"question\": \"What factors contribute to the performance improvement in retrieval-augmented language models compared to non-retrieval-augmented models?\"\n",
    "    },\n",
    "    56: {\n",
    "        \"question\": \"What are the benchmarks used to evaluate the performance of the Deep Policy Optimization (DPO) method compared to other preference learning algorithms in the document provided?\"\n",
    "    },\n",
    "    57: {\n",
    "        \"question\": \"What methodologies have been evaluated for training language models to align with human preferences, and how do they compare in terms of effectiveness?\"\n",
    "    },\n",
    "    58: {\n",
    "        \"question\": \"What methods have been discussed in the literature for improving the alignment of language models with human preferences or feedback?\"\n",
    "    },\n",
    "    66: {\n",
    "        \"question\": \"What are some of the evaluation metrics used for assessing different types of text generation tasks presented in the study?\"\n",
    "    },\n",
    "    68: {\n",
    "        \"question\": \"Consider a document related to research in natural language processing or artificial intelligence. Can you name some of the recent topics or methods that have been discussed or introduced in the field according to the document?\"\n",
    "    },\n",
    "    71: {\n",
    "        \"question\": \"What is the significance of using reflection tokens in a model like SELF-RAG?\"\n",
    "    },\n",
    "    72: {\n",
    "        \"question\": \"How does the inclusion of selected context as opposed to appending all retrieved text spans impact computational cost during both training and inference times in language model generation tasks?\"\n",
    "    },\n",
    "    77: {\n",
    "        \"question\": \"What are the benefits of modeling human biases in Human-Aware Loss Optimizations (HALOs), and how do they compare to non-HALOs on the same datasets?\"\n",
    "    },\n",
    "    79: {\n",
    "        \"question\": \"What are the modifications made to the traditional Kahneman-Tversky model to adapt it for optimizing language model performance?\"\n",
    "    },\n",
    "    83: {\n",
    "        \"question\": \"How does a model's ability to answer questions relate to its exposure to specific types of questions during training?\"\n",
    "    },\n",
    "    90: {\n",
    "        \"question\": \"How can adding examples to a prompt affect the performance of language models?\"\n",
    "    },\n",
    "    98: {\n",
    "        \"question\": \"What are the main components of a Neural Turing Machine (NTM) architecture?\"\n",
    "    },\n",
    "    99: {\n",
    "        \"question\": \"How might a seq2seq model's limitations be addressed in natural language processing tasks?\"\n",
    "    },\n",
    "    100: {\n",
    "        \"question\": \"What differentiates hard attention from soft attention in image processing algorithms?\"\n",
    "    },\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jC86ggPEuipg"
   },
   "source": [
    "###3.3 Running the RAG System\n",
    "\n",
    "Let's have a quick look at the validation and test data:"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "validation_df: pd.DataFrame = pd.DataFrame(validation_questions_answers).transpose()\n",
    "validation_df.to_pickle(\"mids_290_a5/val_df_base.pkl\")\n",
    "print(validation_df.shape)\n",
    "validation_df.head()"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 223
    },
    "id": "1kZXi6q_E8gz",
    "outputId": "a74a5f2e-c85e-439c-f108-ef5551f58f1f"
   },
   "execution_count": 52,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "(75, 3)\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                                            question  \\\n",
       "0  What purpose do large language models serve in...   \n",
       "1  How does a large language model learn from tex...   \n",
       "2  What are some key architectures behind the dev...   \n",
       "3  Can you name some specific large language mode...   \n",
       "7  What licensing models have been adopted for th...   \n",
       "\n",
       "                                gold_answer_research  \\\n",
       "0  Large language models (LLMs) serve the purpose...   \n",
       "1  A large language model learns from text during...   \n",
       "2  Key architectures behind the development of la...   \n",
       "3  Some specific large language models include GP...   \n",
       "7  Based on the provided context, it seems that l...   \n",
       "\n",
       "                               gold_answer_marketing  \n",
       "0  Large language models serve the purpose of imp...  \n",
       "1  A large language model learns from text during...  \n",
       "2  Key architectures behind the development of la...  \n",
       "3           Chinchilla by DeepMind, GPT-3 by OpenAI.  \n",
       "7  Answer: Some organizations choose open-sourcin...  "
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-fda463db-a804-4ca6-92d3-9fd9ecfadb1e\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>question</th>\n",
       "      <th>gold_answer_research</th>\n",
       "      <th>gold_answer_marketing</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>What purpose do large language models serve in...</td>\n",
       "      <td>Large language models (LLMs) serve the purpose...</td>\n",
       "      <td>Large language models serve the purpose of imp...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>How does a large language model learn from tex...</td>\n",
       "      <td>A large language model learns from text during...</td>\n",
       "      <td>A large language model learns from text during...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>What are some key architectures behind the dev...</td>\n",
       "      <td>Key architectures behind the development of la...</td>\n",
       "      <td>Key architectures behind the development of la...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Can you name some specific large language mode...</td>\n",
       "      <td>Some specific large language models include GP...</td>\n",
       "      <td>Chinchilla by DeepMind, GPT-3 by OpenAI.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>What licensing models have been adopted for th...</td>\n",
       "      <td>Based on the provided context, it seems that l...</td>\n",
       "      <td>Answer: Some organizations choose open-sourcin...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fda463db-a804-4ca6-92d3-9fd9ecfadb1e')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-fda463db-a804-4ca6-92d3-9fd9ecfadb1e button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-fda463db-a804-4ca6-92d3-9fd9ecfadb1e');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-d10ea391-c549-460a-9b78-54765876f324\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d10ea391-c549-460a-9b78-54765876f324')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-d10ea391-c549-460a-9b78-54765876f324 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "dataframe",
       "variable_name": "validation_df",
       "summary": "{\n  \"name\": \"validation_df\",\n  \"rows\": 75,\n  \"fields\": [\n    {\n      \"column\": \"question\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"What licensing models have been adopted for the distribution of source-available language models?\",\n          \"What is the main goal of prompt engineering in language models?\",\n          \"What factors influenced the development of generative language models by Anthropic?\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"gold_answer_research\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"Based on the provided context, it seems that licensing models for the distribution of source-available language models have not been explicitly discussed in the referenced papers. However, it is crucial to consider potential licensing options such as open-source licenses (e.g., GPL, MIT) or proprietary licenses when distributing language models to ensure legal compliance and control over usage rights. Additionally, considering the implications of different licensing models on accessibility, collaboration, and commercialization is essential for determining the most suitable approach for sharing language models with the community. Further research or consultation with legal experts may be necessary to explore specific licensing strategies for source-available language models.\",\n          \"The main goal of prompt engineering in language models is to effectively steer the behavior of the model towards desired outcomes without updating the model weights. This is achieved by composing and formatting prompts in a way that maximizes the model's performance on a specific task. Prompt engineering involves treating prompts as trainable parameters and optimizing them directly on the embedding space through methods like AutoPrompt, Prefix-Tuning, P-tuning, and Prompt-Tuning. The ultimate aim is to enhance the model's performance and alignment with user-defined tasks.\",\n          \"Several factors influenced the development of generative language models by Anthropic, including the limitations in coding, math, and reasoning capabilities of the initial version Claude, the partnerships with companies like Notion and Quora to enhance the model's capabilities, and the need to address biases, unsafe content, and ethical considerations in training data. Additionally, the reliance on supervised learning and the need for controlled generation in generative models played a role in shaping the development of Anthropic's language models.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"gold_answer_marketing\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"Answer: Some organizations choose open-sourcing, while others restrict access to a few organizations with resources or offer end-to-end deployment via API.\",\n          \"The main goal of prompt engineering in language models is to steer the behavior of the model for desired outcomes without updating the model weights.\",\n          \"Factors that influenced the development of generative language models by Anthropic include partnerships with companies like Notion and Quora, limitations in coding, math, and reasoning capabilities in initial models like Claude, and the need to address biases and unsafe content in training datasets.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
      }
     },
     "metadata": {},
     "execution_count": 52
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "test_df: pd.DataFrame = pd.DataFrame(test_questions).transpose()\n",
    "print(test_df.shape)\n",
    "test_df.head()"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 223
    },
    "id": "TcHjIlv7FVMV",
    "outputId": "3f78db1c-8615-44c7-a59e-09a0a22d23f0"
   },
   "execution_count": 53,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "(29, 1)\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                                             question\n",
       "4   When was the transformer architecture introduc...\n",
       "5   How has the accessibility of powerful language...\n",
       "6   What benchmarks or ratings are used to compare...\n",
       "10  What are some of the primary applications for ...\n",
       "14  How are language models typically evaluated an..."
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-5cc8ce17-ff97-468a-9242-80c50cf9a7a2\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>question</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>When was the transformer architecture introduc...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>How has the accessibility of powerful language...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>What benchmarks or ratings are used to compare...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>What are some of the primary applications for ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>How are language models typically evaluated an...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5cc8ce17-ff97-468a-9242-80c50cf9a7a2')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-5cc8ce17-ff97-468a-9242-80c50cf9a7a2 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-5cc8ce17-ff97-468a-9242-80c50cf9a7a2');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-d4fa7366-6d82-44f5-b467-e7301270799e\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d4fa7366-6d82-44f5-b467-e7301270799e')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-d4fa7366-6d82-44f5-b467-e7301270799e button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "dataframe",
       "variable_name": "test_df",
       "summary": "{\n  \"name\": \"test_df\",\n  \"rows\": 29,\n  \"fields\": [\n    {\n      \"column\": \"question\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 29,\n        \"samples\": [\n          \"How might a seq2seq model's limitations be addressed in natural language processing tasks?\",\n          \"What methodologies have been evaluated for training language models to align with human preferences, and how do they compare in terms of effectiveness?\",\n          \"What metrics are used to compare the performance of different models on training and validation splits according to the document provided?\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
      }
     },
     "metadata": {},
     "execution_count": 53
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2FQbNGECSq1g"
   },
   "source": [
    "Let's now use the data to ask questions against it. So we need to define our prompt templates, the RAG Chain, etc.\n",
    "\n",
    "We have two types of User Personas we need to support:\n",
    "\n",
    "1. The engineers, who require pretty detailed information when they ask questions  \n",
    "2. The marketing team and supporting staff who also will ask questions around GenAI in order to better understand the products and the field as a whole, but a lot more high level answers would likely be in order\n",
    "\n",
    "**Below, please build your RAG pipeline including the relevant prompts. This is free form so you will need to create your own cells, text documentation as you need, etc.**\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# Verify vectorstore contains relevant info\n",
    "pd.DataFrame(\n",
    "    qdrant_vectorstore.client.get_collection(collection_name=\"rag_tech_db\").dict()\n",
    ")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 258
    },
    "id": "1dTlQa5Xe_2x",
    "outputId": "26ed7b5f-633d-46a0-db2c-af5d90456fd8"
   },
   "execution_count": 54,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                    status optimizer_status  vectors_count  \\\n",
       "params               green               ok           8041   \n",
       "hnsw_config          green               ok           8041   \n",
       "optimizer_config     green               ok           8041   \n",
       "wal_config           green               ok           8041   \n",
       "quantization_config  green               ok           8041   \n",
       "\n",
       "                     indexed_vectors_count  points_count  segments_count  \\\n",
       "params                                   0          8041               1   \n",
       "hnsw_config                              0          8041               1   \n",
       "optimizer_config                         0          8041               1   \n",
       "wal_config                               0          8041               1   \n",
       "quantization_config                      0          8041               1   \n",
       "\n",
       "                                                                config  \\\n",
       "params               {'vectors': {'size': 768, 'distance': 'Cosine'...   \n",
       "hnsw_config          {'m': 16, 'ef_construct': 100, 'full_scan_thre...   \n",
       "optimizer_config     {'deleted_threshold': 0.2, 'vacuum_min_vector_...   \n",
       "wal_config            {'wal_capacity_mb': 32, 'wal_segments_ahead': 0}   \n",
       "quantization_config                                               None   \n",
       "\n",
       "                     payload_schema  \n",
       "params                          NaN  \n",
       "hnsw_config                     NaN  \n",
       "optimizer_config                NaN  \n",
       "wal_config                      NaN  \n",
       "quantization_config             NaN  "
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-240aba28-751e-4c27-a9ad-b0725d746aed\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>status</th>\n",
       "      <th>optimizer_status</th>\n",
       "      <th>vectors_count</th>\n",
       "      <th>indexed_vectors_count</th>\n",
       "      <th>points_count</th>\n",
       "      <th>segments_count</th>\n",
       "      <th>config</th>\n",
       "      <th>payload_schema</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>params</th>\n",
       "      <td>green</td>\n",
       "      <td>ok</td>\n",
       "      <td>8041</td>\n",
       "      <td>0</td>\n",
       "      <td>8041</td>\n",
       "      <td>1</td>\n",
       "      <td>{'vectors': {'size': 768, 'distance': 'Cosine'...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>hnsw_config</th>\n",
       "      <td>green</td>\n",
       "      <td>ok</td>\n",
       "      <td>8041</td>\n",
       "      <td>0</td>\n",
       "      <td>8041</td>\n",
       "      <td>1</td>\n",
       "      <td>{'m': 16, 'ef_construct': 100, 'full_scan_thre...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>optimizer_config</th>\n",
       "      <td>green</td>\n",
       "      <td>ok</td>\n",
       "      <td>8041</td>\n",
       "      <td>0</td>\n",
       "      <td>8041</td>\n",
       "      <td>1</td>\n",
       "      <td>{'deleted_threshold': 0.2, 'vacuum_min_vector_...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wal_config</th>\n",
       "      <td>green</td>\n",
       "      <td>ok</td>\n",
       "      <td>8041</td>\n",
       "      <td>0</td>\n",
       "      <td>8041</td>\n",
       "      <td>1</td>\n",
       "      <td>{'wal_capacity_mb': 32, 'wal_segments_ahead': 0}</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quantization_config</th>\n",
       "      <td>green</td>\n",
       "      <td>ok</td>\n",
       "      <td>8041</td>\n",
       "      <td>0</td>\n",
       "      <td>8041</td>\n",
       "      <td>1</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-240aba28-751e-4c27-a9ad-b0725d746aed')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-240aba28-751e-4c27-a9ad-b0725d746aed button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-240aba28-751e-4c27-a9ad-b0725d746aed');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-208dc307-5bd0-453b-9cd3-20b056602035\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-208dc307-5bd0-453b-9cd3-20b056602035')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-208dc307-5bd0-453b-9cd3-20b056602035 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "dataframe",
       "summary": "{\n  \"name\": \"pd\",\n  \"rows\": 5,\n  \"fields\": [\n    {\n      \"column\": \"status\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"green\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"optimizer_status\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"ok\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"vectors_count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 8041,\n        \"max\": 8041,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          8041\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"indexed_vectors_count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 0,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"points_count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 8041,\n        \"max\": 8041,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          8041\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"segments_count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 1,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"config\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"payload_schema\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": null,\n        \"min\": null,\n        \"max\": null,\n        \"num_unique_values\": 0,\n        \"samples\": [],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
      }
     },
     "metadata": {},
     "execution_count": 54
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "The number of vectors should be approximately equivalent to the number of splits. This seems to add up, since 14k of the splits would come from `arxiv`"
   ],
   "metadata": {
    "id": "Brq14qRMhdPx"
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# Check if documents from each source are present in the vectorstore\n",
    "print(qdrant_vectorstore.similarity_search(\"arxiv.org\")[0])\n",
    "print(qdrant_vectorstore.similarity_search(\"wikipedia.org large language models\")[0])\n",
    "print(qdrant_vectorstore.similarity_search(\"lilianweng.github.io\")[0])"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "p-hOUc-3hFok",
    "outputId": "ddff5a1c-db32-4025-bef6-5d51b35bca94"
   },
   "execution_count": 55,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "page_content='arXiv:2104.09864v5  [cs.CL]  8 Nov 2023' metadata={'source': 'https://arxiv.org/pdf/2104.09864.pdf', 'file_path': 'https://arxiv.org/pdf/2104.09864.pdf', 'page': 0, 'total_pages': 14, 'format': 'PDF 1.5', 'title': 'RoFormer', 'author': '', 'subject': '', 'keywords': '', 'creator': 'LaTeX with hyperref', 'producer': 'pdfTeX-1.40.25', 'creationDate': 'D:20231109011924Z', 'modDate': 'D:20231109011924Z', 'trapped': '', 'page_num': 0, 'doc_num': 3, 'doc_source': 'ArXiv', 'split_id': 726, '_id': '732f8b3bbefc4b58a908ef256db6783a', '_collection_name': 'rag_tech_db'}\n",
      "page_content='=== Large language models ===' metadata={'title': 'Language model', 'summary': 'A language model is a probabilistic model of a natural language. In 1980, the first significant statistical language model was proposed, and during the decade IBM performed ‘Shannon-style’ experiments, in which potential sources for language modeling improvement were identified by observing and analyzing the performance of human subjects in predicting or correcting text.Language models are useful for a variety of tasks, including speech recognition (helping prevent predictions of low-probability (e.g. nonsense) sequences), machine translation, natural language generation (generating more human-like text), optical character recognition, handwriting recognition, grammar induction, and information retrieval.Large language models, currently their most advanced form, are a combination of larger datasets (frequently using scraped words from the public internet), feedforward neural networks, and transformers. They have superseded recurrent neural network-based models, which had previously superseded the pure statistical models, such as word n-gram language model.', 'source': 'https://en.wikipedia.org/wiki/Language_model', 'doc_num': 23, 'doc_source': 'Wikipedia', 'split_id': 35, '_id': 'f4e17fc5879041ffbd39671d2ed202a1', '_collection_name': 'rag_tech_db'}\n",
      "page_content='url     = \"https://lilianweng.github.io/posts/2023-06-23-agent/\"\\n}\\nReferences#' metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', '_id': 'e0bfdec9eb8f4af6a83a1b84f710abb5', '_collection_name': 'rag_tech_db'}\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Iteration 1: Baseline - Naive Solution\n",
    "\n",
    "We first try to use the existing `rag_chain` with the basic prompt of providing context and a question to see how well model's answers perform compared to the validation set. This will behave as a baseline from where we can improve."
   ],
   "metadata": {
    "id": "9ZM3p-izkgZM"
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Mistral Raw Response\n",
    "validation_df[\"naive_response_raw\"] = validation_df[\"question\"].progress_apply(\n",
    "    rag_chain.invoke\n",
    ")\n",
    "validation_df[\"naive_answer\"] = (\n",
    "    validation_df[\"naive_response_raw\"]\n",
    "    .dropna()\n",
    "    .apply(lambda cell: cell.split(\"[/INST]\")[-1])\n",
    ")"
   ],
   "metadata": {
    "id": "JAlDfWc-jqRw"
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Cohere Raw Response\n",
    "validation_df[\"naive_response_raw_cohere\"] = validation_df[\"question\"].progress_apply(\n",
    "    cohere_rag_chain.invoke\n",
    ")"
   ],
   "metadata": {
    "id": "PMjD6AmXI1Xw",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "0149fe82ffc648c8b5ef154ff4fe05fe",
      "73fc3026a832486aa65f0131cd38d999",
      "a3a2ace2e43a42e59360eda9bd15feb0",
      "5f42a8d50e6642b8bfb194c3a67754fb",
      "d5ca95777bb44f36985b522b6da13fd9",
      "81f29be6cf00405f9c2709cbf1322382",
      "68f229a722304e0f8f663f708dcbb7b2",
      "c3ea93469c5945f6886e1d3c1769f8d5",
      "8156d53326314d4aa600de4eb50ddac6",
      "add29108855947feac91ae2941e9af84",
      "bfcdbca9e34b4a9e855ec36ba5409507"
     ]
    },
    "outputId": "8ab799be-7829-49f8-df25-74bac1e5fe8f"
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "0149fe82ffc648c8b5ef154ff4fe05fe"
      }
     },
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# Save checkpoint of data after iteration 1\n",
    "validation_df.to_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_1_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")"
   ],
   "metadata": {
    "id": "VIB18C3hqjVE"
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jJE1wexvV7ay"
   },
   "source": [
    "##4. Tests & Evaluations\n",
    "\n",
    "Here you should evaluate the results. First, you should define your evaluation metrics and then you should run evaluation tests. This is really your area, but key results to show are:\n",
    "\n",
    "1) Your metrics of choice  \n",
    "2) How  your various models compare to the labeled validation data.\n",
    "\n",
    "Make sure you look at the results for the marketing team and the research team separately.\n",
    "\n",
    "**Note:** You do not need to run all models against all labeled questions, as that may take some time. Just do that for a few models/configs, and test a larger set with a smaller subset.\n",
    "\n",
    "**This is free form so you will need to create your own cells, text documentation as you need, etc.**\n",
    "\n",
    "###4.1. Metrics\n",
    "\n",
    "Please define and motivate your metrics here. Please feel free to add more text and code cells as needed.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "We define the following metrics (more details in report):\n",
    "* Cosine Similarity of Base Embeddings\n",
    "* `BLEU` score\n",
    "* `ROUGE` score\n",
    "* `BERT` score\n",
    "\n",
    "We compare samples with the baseline results - since that would be the poorest performing metric. We expect low numbers when using this sample on metrics."
   ],
   "metadata": {
    "id": "89zQ-46muGlm"
   }
  },
  {
   "cell_type": "code",
   "source": [
    "validation_df: pd.DataFrame = pd.read_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_1_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")\n",
    "print(validation_df.shape)\n",
    "validation_df.head()"
   ],
   "metadata": {
    "id": "ITNkAIycMOt4"
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "id": "GjOCg-BHWBvY",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 177,
     "referenced_widgets": [
      "cf0eb0a3f0e449eebd522fdfc77431ca",
      "051ae4f420cd47628c19f0ef96803995",
      "e4d7a6143fca4847a827565566c305a2",
      "554c85e291984153a4fda3be0062eb08",
      "05e41fc637194280b6a2df80d5e07bce",
      "831c6e88707c41418d44a450fd168c8c",
      "bdc135c82d8549cda8dc8266b6d28744",
      "116ac1adfe6840e0b4b88b573e506351",
      "1da39059b4c24b689e747a8cfdaf77e6",
      "d56ba311aab143f2af9ad64fcd1a8731",
      "9d3c408f65054f40adf71aee594d0cd0",
      "d3aecc5dbdf943fc83a62b222ab3e701",
      "18e325e59b614c9680bd072f68a5d8f3",
      "a4779a6e714046a397c644c533854674",
      "c0b8bda14a7747dfb003662d95c73dfc",
      "7f32501b31334ddfae549be764dd0d71",
      "4c81895beec54f718ad80548bf1118d7",
      "a2d06eba88cf4d11a71c258e5ed67ddf",
      "5c05b2f3aa7c4fa1ae636f701ca0892b",
      "5a58e1c5ffc44de1af524826a28059cb",
      "88313eb879c742a290231489e8e317f9",
      "c81441f912a64129927744af70b8369e",
      "4a13fda5f49d497f8fca1ce8413ffd4b",
      "58e2c5cf163f4ee09e859bfb8c712512",
      "a25c47accf9d446180914c9066dd4d03",
      "99ee1547b9e34f9f88e77a4c8d688dbb",
      "a8ae735e4abb4234b1ee06ba4e55b241",
      "35a8cb8935c74298b49105539d71310e",
      "8b91f07b56bf443aa5343ba9ab3d1236",
      "5fcf21c145bc4ebfac26089e1ddcf1c3",
      "70ebb6ec218f40f1a2e66b83c1e6dcdf",
      "8d3bb7e843804a3cbdef41b1c93f2560",
      "b88a2b516eae4dcfbbe6830850fde735",
      "ab62a871485b4a50898595285cd6b162",
      "9a7da06768574645a46d0aa9e9fa2722",
      "d50bb1c95ec746aba820dcd9b1722401",
      "564f318ec0ea4045b7439032ffe97b7d",
      "3c93c5b03e964e339730fb0a8e3ccd3d",
      "ea5abbd055024ca98f03f2daf332284f",
      "13d0beb61d244723a9fe4e5fccfc7bcc",
      "ac6e1d90a6ae4504b17c1ccadbfc4e6e",
      "c9e94e2ab35f416c930cf5675e9a8c6b",
      "242a040e8b434513880cdc33aae097c2",
      "2d7f8772755a4e4b8508fda8c4bc7438",
      "ed7e1ef5655b4460ab7b5186f438c64f",
      "c7adb46dadd74f249647fdcb22ea40e7",
      "66ad787dbf5c49abad425f8eaadbb447",
      "51735840ede543669d7ed21aef805428",
      "ca4ef0b0550842b8b7770f0d337473b6",
      "cccf7a0295e74c9b94aca66bf4313752",
      "e55847dc6159494794395c7df07ffde7",
      "e33b05b166384fcda9fa9620da3f2c38",
      "e2c05f613fcc49f892cd384661a9935b",
      "04d5aea12628438590b96ecde03e46b9",
      "2f046d82889f474bac793b91809eb387"
     ]
    },
    "outputId": "9cc7c45b-8a1c-42f7-e5e0-64248e0da1e1"
   },
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "Downloading builder script:   0%|          | 0.00/5.94k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "cf0eb0a3f0e449eebd522fdfc77431ca"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "Downloading extra modules:   0%|          | 0.00/1.55k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "d3aecc5dbdf943fc83a62b222ab3e701"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "Downloading extra modules:   0%|          | 0.00/3.34k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "4a13fda5f49d497f8fca1ce8413ffd4b"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "Downloading builder script:   0%|          | 0.00/6.27k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "ab62a871485b4a50898595285cd6b162"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "Downloading builder script:   0%|          | 0.00/7.95k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "ed7e1ef5655b4460ab7b5186f438c64f"
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "# @title Load Metric Evaluators\n",
    "bleu = evaluate.load(\"bleu\")\n",
    "rouge = evaluate.load(\"rouge\")\n",
    "bertsc = evaluate.load(\"bertscore\")"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# Declare Sample Data\n",
    "x = validation_df.iloc[0][\"gold_answer_research\"]\n",
    "y = validation_df.iloc[0][\"naive_answer\"]\n",
    "z = validation_df.iloc[0][\"naive_response_raw_cohere\"]\n",
    "\n",
    "print(f\"{x=}\")\n",
    "print(f\"{y=}\")\n",
    "print(f\"{z=}\")"
   ],
   "metadata": {
    "id": "0CvWAZXMvmz0",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "71c078fe-4ab7-4c0d-927b-03381bf272b0"
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "x='Large language models (LLMs) serve the purpose of enabling general-purpose language generation and other natural language processing tasks such as classification. They achieve this by learning statistical relationships from text documents during computationally intensive self-supervised and semi-supervised training. LLMs can be used for text generation by predicting the next token or word, making them valuable for tasks like speech recognition, machine translation, and information retrieval. Additionally, LLMs have superseded previous models like recurrent neural networks, showcasing their efficiency and effectiveness in NLP tasks.'\n",
      "y=' In the field of natural language processing, large language models serve as controllers for building agents that can engage in deliberative problem solving by leveraging their ability to process and generate text. They are capable of learning from vast amounts of data, enabling them to understand complex language patterns and make accurate predictions or decisions. The use of large language models has shown promise in various applications, including scientific research, where they have demonstrated emergent autonomous capabilities.'\n",
      "z='Large language models (LLMs) have revolutionized the field of natural language processing (NLP) by providing advanced capabilities for understanding and generating human language. The main purpose of LLMs in NLP is to enable machines to process and interpret human language in a way that is similar to human cognition.\\n\\nLLMs are designed to learn from vast amounts of text data and capture the complexities of human language, including syntax, semantics, and context. They can be used for various NLP tasks such as language translation, text generation, sentiment analysis, question answering, and more.\\n\\nBy leveraging deep learning techniques and training on massive text corpora, LLMs have achieved remarkable performance in generating human-like responses, understanding context, and making inferences. This has led to their widespread adoption in applications such as virtual assistants, language translation services, content generation, and customer service chatbots.\\n\\nIn the research papers mentioned in the context, the authors explore the potential of using LLMs as a core component in building autonomous agents. They focus on the capabilities of LLMs to perform complex reasoning, problem-solving, and planning, highlighting their potential to revolutionize various industries and research fields.\\n\\nOverall, LLMs have become essential in the field of NLP, enabling machines to interpret and generate human language with unprecedented accuracy and flexibility, and opening up new possibilities for human-machine interaction and automation.'\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title BLEU Sample\n",
    "print(bleu.compute(predictions=[y], references=[x]))\n",
    "print(bleu.compute(predictions=[z], references=[x]))"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2y6s2GtFwUWb",
    "outputId": "14be753a-5c96-485b-f390-a2a8b850376f"
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "{'bleu': 0.0, 'precisions': [0.39285714285714285, 0.04819277108433735, 0.012195121951219513, 0.0], 'brevity_penalty': 0.8877655252065778, 'length_ratio': 0.8936170212765957, 'translation_length': 84, 'reference_length': 94}\n",
      "{'bleu': 0.05691366651073745, 'precisions': [0.20948616600790515, 0.0873015873015873, 0.035856573705179286, 0.016], 'brevity_penalty': 1.0, 'length_ratio': 2.6914893617021276, 'translation_length': 253, 'reference_length': 94}\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Rouge Sample\n",
    "print(rouge.compute(predictions=[y], references=[x]))\n",
    "print(rouge.compute(predictions=[z], references=[x]))"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qCaf6zsgxU2X",
    "outputId": "37597cd4-0996-4fe3-cf24-c46e80f9d623"
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "{'rouge1': 0.34355828220858897, 'rouge2': 0.049689440993788817, 'rougeL': 0.15950920245398773, 'rougeLsum': 0.15950920245398773}\n",
      "{'rouge1': 0.2977346278317152, 'rouge2': 0.10423452768729642, 'rougeL': 0.18122977346278318, 'rougeLsum': 0.22006472491909382}\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title BERT Score Sample\n",
    "print(bertsc.compute(predictions=[y], references=[x], lang=\"en\"))\n",
    "print(bertsc.compute(predictions=[z], references=[x], lang=\"en\"))"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hmT4xm1NLuGO",
    "outputId": "53fa8c0e-5905-4b2b-c232-2d6357b8ebdd"
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Some weights of RobertaModel were not initialized from the model checkpoint at roberta-large and are newly initialized: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "{'precision': [0.8741947412490845], 'recall': [0.8582159280776978], 'f1': [0.866131603717804], 'hashcode': 'roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.40.0.dev0)'}\n",
      "{'precision': [0.8594847917556763], 'recall': [0.8865609169006348], 'f1': [0.8728128671646118], 'hashcode': 'roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.40.0.dev0)'}\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Embeddings Cosine Similarity Sample\n",
    "csxy = cosine_similarity(\n",
    "    [np.array(base_embeddings.embed_query(y))],\n",
    "    [np.array(base_embeddings.embed_query(x))],\n",
    ")[0][0]\n",
    "\n",
    "csxz = cosine_similarity(\n",
    "    [np.array(base_embeddings.embed_query(z))],\n",
    "    [np.array(base_embeddings.embed_query(x))],\n",
    ")[0][0]\n",
    "\n",
    "print(f\"{csxy=:.2f}\")\n",
    "print(f\"{csxz=:.2f}\")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "GFZ7aLsK-dNI",
    "outputId": "2e70aba8-cc48-4f68-f192-f953ada37d5b"
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "csxy=0.75\n",
      "csxz=0.90\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "Based on a sample test of few metrics on the baseline response vs. gold responses -\n",
    "\n",
    "1. `Cohere` almost always performs better (which might be expected)\n",
    "2. `Cosine Similarity` and `BERT Score` tend to have much higher values, which would make threshold setting a lot harder, since the minimum for the baseline consists of already high scores."
   ],
   "metadata": {
    "id": "mMuL-f5GMjNW"
   }
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kKktTilyTy_z"
   },
   "source": [
    "###4.2. Evaluation Comparisons\n",
    "\n",
    "Document your key runs here. Feel free to add more text and code cells as needed."
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Evaluate Baseline (Iteration 1)\n",
    "\n",
    "gold_col_acronyms: dict[str, str] = {\n",
    "    \"gold_answer_research\": \"eng\",\n",
    "    \"gold_answer_marketing\": \"mkt\",\n",
    "}\n",
    "\n",
    "\n",
    "def assign_metrics(\n",
    "    dataframe: pd.DataFrame, iteration: int, col_names_iter: dict\n",
    ") -> pd.DataFrame:\n",
    "    df = dataframe.copy()\n",
    "\n",
    "    for model_choice in [\"mistral\", \"cohere\"]:\n",
    "        for ref_answer in [\"gold_answer_research\", \"gold_answer_marketing\"]:\n",
    "            acro = gold_col_acronyms[ref_answer]\n",
    "            df[\n",
    "                f\"{model_choice}_bleu_{gold_col_acronyms[ref_answer]}_it_{iteration}\"\n",
    "            ] = df.apply(\n",
    "                lambda row: bleu.compute(\n",
    "                    references=[row[ref_answer]],\n",
    "                    predictions=[row[col_names_iter[acro][model_choice]]],\n",
    "                )[\"bleu\"],\n",
    "                axis=1,\n",
    "            )\n",
    "\n",
    "            df[\n",
    "                f\"{model_choice}_rouge_{gold_col_acronyms[ref_answer]}_it_{iteration}\"\n",
    "            ] = df.progress_apply(\n",
    "                lambda row: rouge.compute(\n",
    "                    references=[row[ref_answer]],\n",
    "                    predictions=[row[col_names_iter[acro][model_choice]]],\n",
    "                )[\"rougeL\"],\n",
    "                axis=1,\n",
    "            )\n",
    "\n",
    "            df[\n",
    "                f\"{model_choice}_bert_{gold_col_acronyms[ref_answer]}_it_{iteration}\"\n",
    "            ] = df.progress_apply(\n",
    "                lambda row: bertsc.compute(\n",
    "                    references=[row[ref_answer]],\n",
    "                    predictions=[row[col_names_iter[acro][model_choice]]],\n",
    "                    lang=\"en\",\n",
    "                )[\"f1\"][0],\n",
    "                axis=1,\n",
    "            )\n",
    "    return df"
   ],
   "metadata": {
    "id": "vsvpqkmEQvVJ"
   },
   "execution_count": 57,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "validation_df: pd.DataFrame = pd.read_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_1_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")\n",
    "validation_df = assign_metrics(\n",
    "    validation_df,\n",
    "    iteration=1,\n",
    "    col_names_iter={\n",
    "        \"eng\": {\"mistral\": \"naive_answer\", \"cohere\": \"naive_response_raw_cohere\"},\n",
    "        \"mkt\": {\"mistral\": \"naive_answer\", \"cohere\": \"naive_response_raw_cohere\"},\n",
    "    },\n",
    ")\n",
    "validation_df[\n",
    "    [c for c in validation_df.columns if \"cohere\" in c or \"mistral\" in c]\n",
    "].describe().transpose().round(2)"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 681,
     "referenced_widgets": [
      "a976ce18edd44f09ace115f6ca4175be",
      "5da128b45b94482ba2b8cce91b93bd78",
      "eda0583a1eaf48fd88ae1b04b9d68add",
      "9cafa3eae2ef4e5babcda1f8b173558b",
      "c4c057a9b3cd493fa660250137793365",
      "4e369d779cdd47e297c375a7a9ead2bd",
      "c6695f5e87024b518a8dc045cba9aa9c",
      "e52a8688b5e34f7986f7f0a30c317411",
      "a521da5ce4544c0b8245bcaa96008f4e",
      "16afbe5850c547b6b64224f8f4b88242",
      "fd6de45284c84df795565c9acce7711b",
      "48a4e3941bbe4718a06f70671974f67a",
      "a8fc24041be649839040f067aa713ff9",
      "95e2b89be2c6402b87841efee285fbfd",
      "6e32da8ab47445ee8f25fe9a5a8d63e8",
      "081c2e2678e84628b16ea9b59aa2230c",
      "32024293d24045b98c165d3d48c6d59a",
      "b7baaacce5c14735bdcceff0eeeacd55",
      "1ce2daa108be4aefae08d1b6330037d7",
      "7e94c8603c194169a14823ef5a9fee0d",
      "0430d90708af45f68d31d3f5c9845fea",
      "60f11304ded1466ab1f7b2f22e238994",
      "e141faf3239543339ab325ca2af306af",
      "f271bfc517c04c36b24d1b3a0cb2dde1",
      "9b94c11b0438478286cf38964e48cf92",
      "41d88777ab6a4a5ba53d0a2bc4f155bc",
      "dd6d572d3c91436b8a20880c95d5eacf",
      "cb5ab4e93e6a4ded8500436ccd5ff74c",
      "1737688ec5074ab5bcaad2b2315ebe7b",
      "d6fb97e0f75949588b7b561804b46332",
      "9cfabb8292254e1c8b14585c377f3879",
      "0d5e5bafd5004d35a44688d90c218a5f",
      "fdaddc9124ed49dcbea3d5755e7d223e",
      "ca616bbb149f441d92256a17ce617b3b",
      "2401c5056fd247999edcd1b6176390a7",
      "d1a36dba11be48d98ce88e4f2055dae1",
      "a05853a983ed4bfb98659908e3f86b10",
      "beec8bad40724459866bcaf255d283c4",
      "334a14ca95d14b949f7608ec5bb9240c",
      "a5dfafa758574a37863fc007ce51a101",
      "d1e1f9633bab406f9f8530a0fb6367f1",
      "1f414cdc46504d50adb70360e422606c",
      "cbcfc49a593540dabf75d11a3f23329b",
      "6bfae7df0b4049d9926714a2b979a8be",
      "69f6c39f632444b69a0dd58056693399",
      "150bc4489b554dc99330ba959dc6e41c",
      "64b573aa115b40ca8e74036b8afb38c5",
      "4af9756239b74b9382e2f34c11c48aec",
      "2dace23764b644608443bb1e59ddba9d",
      "30780a4529474c8bb685919be6ea3953",
      "a4c95e2fd15e40e7b14538760f6b7969",
      "f92e3e68b2ec4b2ab4568d80c189d844",
      "7e6a713e919b475aac4a952d741ac82a",
      "9f63cf9941fd4495aea7862cfdbf2e68",
      "fae247cd8826438697e6d6dba227f8b3",
      "3b15da9197744637b728d07dd9596148",
      "be5242e90a84427b9d62694b55a272ac",
      "b9318fd216dc4689aa2cd24db76174b6",
      "aedcb55caadd4a7dafbf9ed4bdd0cec0",
      "eb0541cd87454eb1a45f4718e1efcd81",
      "3c404b683cee41029950d94f13f32c36",
      "f31cc652b1c54c6eb1207e8ecbba40e1",
      "263b2006a5314ae088127e1624c3a42c",
      "b682304c931f458eb2f1c2cf687ca042",
      "63b7d1158f7a46d88fafc231391fbd11",
      "0aca9c01b50241e6ad2cc35b58f77bad",
      "01a7ae757b9644bb8f51df6f3e040805",
      "131764a259134322bda8e402faac2574",
      "b66a2f3479ce4c02ba2f71953f8f1d21",
      "3587668bd9f2495d9550226870e6d459",
      "40a064a67338429dbc24c7e3d18a9a53",
      "bf706e39dde5491d82f131b0af469290",
      "db65392a4fb64586af0df7ea491bd94c",
      "36868620156c4089b252cfefec8d49dc",
      "8c3f6d77cc594ef8916300fb585c9c16",
      "af5cfbce278947d7905aeacdbdeba7bd",
      "ef91a28b9be345eab3db6d3485c1bd11",
      "b213a07145554c12b207dbd72c0d76c8",
      "b60300798fbc45df806be401450d68a2",
      "6c499589c869464cb975212ed6887358",
      "35a1e99a57c642e2b85c35135e2ebfdf",
      "0734b37b22174ec7b34b70deaf73f6e3",
      "918cc795c2cb4bd6b77dc15d41fbbe45",
      "de8526671a8646169200a16d11d71831",
      "751bad9743f547e791575bdad7d07932",
      "f82ec77de29b4576b8ac1bb94b092631",
      "5e415b151b894bc18a44a6f858c5439d",
      "3cdf780414ca481186f6fc195df7032f"
     ]
    },
    "id": "BAl9zcRkUU2f",
    "outputId": "ed284259-d034-4175-b003-ca49d280e7ab"
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "a976ce18edd44f09ace115f6ca4175be"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "48a4e3941bbe4718a06f70671974f67a"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "e141faf3239543339ab325ca2af306af"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "ca616bbb149f441d92256a17ce617b3b"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "69f6c39f632444b69a0dd58056693399"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "3b15da9197744637b728d07dd9596148"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "01a7ae757b9644bb8f51df6f3e040805"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "b213a07145554c12b207dbd72c0d76c8"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                        count  mean   std   min   25%   50%   75%   max\n",
       "mistral_bleu_eng_it_1    75.0  0.08  0.06  0.00  0.03  0.07  0.12  0.29\n",
       "mistral_rouge_eng_it_1   75.0  0.22  0.06  0.04  0.18  0.21  0.26  0.44\n",
       "mistral_bert_eng_it_1    75.0  0.87  0.02  0.81  0.86  0.87  0.88  0.91\n",
       "mistral_bleu_mkt_it_1    75.0  0.07  0.09  0.00  0.00  0.05  0.10  0.44\n",
       "mistral_rouge_mkt_it_1   75.0  0.22  0.11  0.00  0.14  0.22  0.27  0.64\n",
       "mistral_bert_mkt_it_1    75.0  0.87  0.02  0.81  0.85  0.87  0.89  0.93\n",
       "cohere_bleu_eng_it_1     75.0  0.06  0.06  0.00  0.02  0.05  0.08  0.43\n",
       "cohere_rouge_eng_it_1    75.0  0.20  0.08  0.07  0.14  0.19  0.23  0.67\n",
       "cohere_bert_eng_it_1     75.0  0.86  0.02  0.82  0.84  0.86  0.87  0.91\n",
       "cohere_bleu_mkt_it_1     75.0  0.04  0.06  0.00  0.00  0.02  0.05  0.43\n",
       "cohere_rouge_mkt_it_1    75.0  0.15  0.09  0.03  0.10  0.14  0.19  0.67\n",
       "cohere_bert_mkt_it_1     75.0  0.85  0.02  0.81  0.84  0.85  0.87  0.91"
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-8d3bfad0-3a59-4c4e-893a-f74190278e81\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>mistral_bleu_eng_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.03</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_rouge_eng_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.22</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.04</td>\n",
       "      <td>0.18</td>\n",
       "      <td>0.21</td>\n",
       "      <td>0.26</td>\n",
       "      <td>0.44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_bert_eng_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.81</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.88</td>\n",
       "      <td>0.91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_bleu_mkt_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.05</td>\n",
       "      <td>0.10</td>\n",
       "      <td>0.44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_rouge_mkt_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.22</td>\n",
       "      <td>0.11</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.22</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_bert_mkt_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.81</td>\n",
       "      <td>0.85</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.89</td>\n",
       "      <td>0.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_bleu_eng_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.05</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_rouge_eng_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.20</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.19</td>\n",
       "      <td>0.23</td>\n",
       "      <td>0.67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_bert_eng_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.82</td>\n",
       "      <td>0.84</td>\n",
       "      <td>0.86</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_bleu_mkt_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.04</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.05</td>\n",
       "      <td>0.43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_rouge_mkt_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.15</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.03</td>\n",
       "      <td>0.10</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.19</td>\n",
       "      <td>0.67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_bert_mkt_it_1</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.85</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.81</td>\n",
       "      <td>0.84</td>\n",
       "      <td>0.85</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.91</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8d3bfad0-3a59-4c4e-893a-f74190278e81')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-8d3bfad0-3a59-4c4e-893a-f74190278e81 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-8d3bfad0-3a59-4c4e-893a-f74190278e81');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-68e4e8ce-7b44-4d63-b5e7-121e990d2276\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-68e4e8ce-7b44-4d63-b5e7-121e990d2276')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-68e4e8ce-7b44-4d63-b5e7-121e990d2276 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "dataframe",
       "summary": "{\n  \"name\": \"validation_df[[c for c in validation_df\",\n  \"rows\": 12,\n  \"fields\": [\n    {\n      \"column\": \"count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.0,\n        \"min\": 75.0,\n        \"max\": 75.0,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          75.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"mean\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.3657733608604535,\n        \"min\": 0.04,\n        \"max\": 0.87,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          0.15\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"std\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.03165869118066518,\n        \"min\": 0.02,\n        \"max\": 0.11,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          0.02\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"min\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.39201403036115945,\n        \"min\": 0.0,\n        \"max\": 0.82,\n        \"num_unique_values\": 6,\n        \"samples\": [\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"25%\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.3841006812455991,\n        \"min\": 0.0,\n        \"max\": 0.86,\n        \"num_unique_values\": 9,\n        \"samples\": [\n          0.84\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"50%\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.37185855896938536,\n        \"min\": 0.02,\n        \"max\": 0.87,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          0.14\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"75%\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.3586707361490438,\n        \"min\": 0.05,\n        \"max\": 0.89,\n        \"num_unique_values\": 11,\n        \"samples\": [\n          0.89\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"max\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.2326315203114665,\n        \"min\": 0.29,\n        \"max\": 0.93,\n        \"num_unique_values\": 7,\n        \"samples\": [\n          0.29\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
      }
     },
     "metadata": {},
     "execution_count": 317
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "validation_df.to_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_1_METRICS_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")\n",
    "validation_df.describe().transpose().round(2).to_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_1_DESCRIBE_METRICS_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")"
   ],
   "metadata": {
    "id": "wLvZXTFb47j5"
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Iteration 2: Prompt Engineering\n",
    "\n",
    "In this iteration, we will use separate prompts for engineering and marketing. We will first construct the prompts by inspection of the validation dataset. And then, we will add keywords like `arxiv` for engineering team, and `wikipedia` for marketing so that they get relevant results."
   ],
   "metadata": {
    "id": "zrFUvcxIQeDX"
   }
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "id": "WXSFKKvtT0m1",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "7ade8220-aee9-4d90-9f43-6ad8b3f29109"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "count    75.000000\n",
      "mean     37.120000\n",
      "std      15.507592\n",
      "min       6.000000\n",
      "25%      27.000000\n",
      "50%      34.000000\n",
      "75%      47.500000\n",
      "max      75.000000\n",
      "Name: gold_answer_marketing, dtype: float64\n",
      "count     75.000000\n",
      "mean      85.853333\n",
      "std       15.619369\n",
      "min       16.000000\n",
      "25%       81.000000\n",
      "50%       87.000000\n",
      "75%       94.500000\n",
      "max      113.000000\n",
      "Name: gold_answer_research, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# @title Distributions of Tokens for Engineering and Marketing Gold Responses\n",
    "print(validation_df[\"gold_answer_marketing\"].str.split().apply(len).describe())\n",
    "print(validation_df[\"gold_answer_research\"].str.split().apply(len).describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "* Engineering has on average `80-100` tokens per gold response\n",
    "* Marketing has on average `30-50` tokens per gold response"
   ],
   "metadata": {
    "id": "vFE0p8nOZ3MD"
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Declare Mistral Pipeline for Prompting - Less Temperature (Fix Context)\n",
    "mistral_prompt_pipe = pipeline(\n",
    "    \"text-generation\",\n",
    "    model=llm_mistral_model,\n",
    "    tokenizer=llm_mistral_tokenizer,\n",
    "    max_length=1000,\n",
    "    temperature=0.1,\n",
    "    top_p=0.95,\n",
    "    do_sample=True,\n",
    "    repetition_penalty=1.2,\n",
    ")\n",
    "mistral_prompt_pipe.model.config.pad_token_id = (\n",
    "    mistral_prompt_pipe.model.config.eos_token_id\n",
    ")\n",
    "\n",
    "mistral_prompt_lc = HuggingFacePipeline(pipeline=mistral_prompt_pipe)"
   ],
   "metadata": {
    "id": "A6OZaJIgdt_J"
   },
   "execution_count": 59,
   "outputs": []
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "id": "vdOV07NeT0bC"
   },
   "outputs": [],
   "source": [
    "# @title Marketing Prompt\n",
    "rag_2_mkt_template = \"\"\"\n",
    "[INST]\n",
    "Please answer the `question` below only based on the `context` information provided, and the rules mentioned below.\n",
    "You are a chatbot for knowledge search on \"Large Language Models\" and \"Generative AI\".\n",
    "The target audience for this is marketing professionals.\n",
    "Provide a detailed response not exceeding sixty words and reduce use of adjectives.\n",
    "\n",
    "Here is a `context`: {context}\n",
    "Here is a `question`: {question}.\n",
    "[/INST]\"\"\"\n",
    "\n",
    "rag_2_mkt_prompt = ChatPromptTemplate.from_template(rag_2_mkt_template)\n",
    "\n",
    "rag_2_sample_q: str = validation_df.iloc[1][\"question\"]\n",
    "gold_2_mkt: str = validation_df.iloc[1][\"gold_answer_marketing\"]"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Mistral Marketing Sample\n",
    "rag_2_mistral_mkt_chain = (\n",
    "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
    "    | rag_2_mkt_prompt\n",
    "    | mistral_prompt_lc\n",
    ")\n",
    "\n",
    "response_2_mkt_mistral = rag_2_mistral_mkt_chain.invoke(rag_2_sample_q).split(\n",
    "    \"[/INST]\"\n",
    ")[-1]\n",
    "\n",
    "print(\"Mistral Response:\", response_2_mkt_mistral)\n",
    "len(response_2_mkt_mistral.split())\n",
    "print()\n",
    "print(\"Gold:\", gold_2_mkt)\n",
    "\n",
    "print(\"Mistral Iteration 2 Sample Metrics - Marketing\")\n",
    "print(bleu.compute(references=[gold_2_mkt], predictions=[response_2_mkt_mistral]))\n",
    "print(rouge.compute(references=[gold_2_mkt], predictions=[response_2_mkt_mistral]))\n",
    "print(\n",
    "    bertsc.compute(\n",
    "        references=[gold_2_mkt], predictions=[response_2_mkt_mistral], lang=\"en\"\n",
    "    )\n",
    ")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 402,
     "referenced_widgets": [
      "834c03b008544b39a8c41f92d91b9017",
      "a2c04366af704af580dff65644578e93",
      "f1b58582c1a74d5cbe567bcbadc92a0e",
      "08cdcc4236ab462d9ec860e7e83e6e45",
      "ccc477343005421ebfcf4eecc5c68f0f",
      "f3f70256597e4c6ba5441fe35f5ddef4",
      "e6aae83d83c44a7fa1cdd20925a839c2",
      "0d64578a4f11450991e36c892d7ff785",
      "eabb559f76914d709fae4ac97616813c",
      "f0a8bcad4f324495aa00891b85339d8d",
      "75b99a13350041e29e54fbdb47572f5a",
      "391da2f76d0f4be1bc5758b2a3bc3118",
      "dd2839a63681428c9264ffc628045f42",
      "c728d143846849ee97b25e992ea6b913",
      "a278f2e97d7a4d37ac0243ab55063e04",
      "ca294e7b1eb745999dee3ed877e4c099",
      "e7a1baf86a86474890b0de827a85c0af",
      "c42e8b6969684ae79d04764f2c6dead2",
      "e83e90890c29406489109c650b125a1d",
      "38026d179cbf4ad29e4f287192410a91",
      "c26bea6bf970423c8a48ff6267aae71f",
      "04c55e8b021949d59142203c390662de",
      "9133bd0248ef4f07a083b7c756ecac58",
      "f7e178beb7ad4c6cb3e1a9c728fa5c50",
      "124237f6ac514b5794a6ca967adc96b0",
      "ca0543c972394bcab6947d27e1b33b47",
      "fd3e86b41c024921bda45970c4449748",
      "31a761ae907245eeab75207e3576f821",
      "c3408de3d9ca42c9bba2572754f7d9a7",
      "0e55089cf2f648a8b76e99f1198f3851",
      "bf4c5734ec7d478487d32c4d7721b750",
      "87ffb501c4624d3bbc6bcbd0c234a23d",
      "9fe03839860c4bb3a684c3142f74a6a5",
      "d490c04d6f3e47328bf442eaeb5db7c1",
      "6604e705c356441fa3bcf44f749db803",
      "fdb4b65e48c14d6b814661a19022c952",
      "b6c032e0311f406c8d7d8f303950b21f",
      "2240383532fc4a0d86d530c297041a72",
      "7bcd95d4afbf41f7b6b7b46ff0f5d298",
      "6204a4d12ba24491bcab3f46b1ae837f",
      "34a578b037d444b299184b23e951d53c",
      "9d05c6a1515b4cff86819914c22db465",
      "e1009e5016c34ae78b3116b316938e93",
      "18c7ce83ed534293a9f2f2d2ab0df23a",
      "6a36b773606f47148c2e3f427956eda2",
      "89c9e05b87c44c689d4dfb5da7a1e9df",
      "cf1e3b1a708b4df1b2c368b563ae3d48",
      "0b41255ca2db4ff0b4f11d91589513d0",
      "54fa8a1f67734ecbbb34b717295db173",
      "b5aaa32e99464bc59cbf3c7f1fab0e36",
      "489209144d7c42ff88961adf48d22e20",
      "d5ca8f29ea1d4f229d413950c365938e",
      "cc3b808cd68041658fefb195ee47f117",
      "e4d9e01c6c87413281fe0220a00f07f8",
      "28dca91697ae4a489f72408b586bc930",
      "8f42fb4d6c82470e9f8499236d9b771f",
      "7773f7cdac984ce08e7fa77d4acc167c",
      "15a949c80d01436b8f5e05ffa6889cbf",
      "b15d7eeabf724b76a72e2ade1a00b6c5",
      "fd33f022f9504e308e193a7323f4b38c",
      "33f0a478237d445ebb84e17f3a86a502",
      "90f434d040f94167bad1ac152a45138f",
      "3ecaecd0b28749c8acccf74e1cb55be3",
      "3a6793b7c618449baa32b3b622a96c88",
      "d6f913d8dafd4603a93b582fe994e006",
      "0eeb8ae9055341f6b20d9a6a0899e2f1"
     ]
    },
    "id": "NyQciTC0ZObi",
    "outputId": "83fdaa99-f282-4c0b-fc10-1efe767dbcb5"
   },
   "execution_count": 61,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Mistral Response:  A large language model learns from text during training through a process called self-attention, which allows it to weigh the importance of different parts of the input text when generating output. This enables the model to capture complex relationships between words and phrases, as well as contextual clues that help it generate coherent and accurate responses.\n",
      "\n",
      "Gold: A large language model learns from text during training by first pretraining on a diverse dataset to acquire general language knowledge, and then fine-tuning on specific tasks or demonstrations to adapt its parameters for more targeted performance.\n",
      "Mistral Iteration 2 Sample Metrics - Marketing\n",
      "{'bleu': 0.14405591864498601, 'precisions': [0.25, 0.13559322033898305, 0.1206896551724138, 0.10526315789473684], 'brevity_penalty': 1.0, 'length_ratio': 1.5384615384615385, 'translation_length': 60, 'reference_length': 39}\n",
      "{'rouge1': 0.2736842105263158, 'rouge2': 0.17204301075268819, 'rougeL': 0.2526315789473684, 'rougeLsum': 0.2526315789473684}\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/25.0 [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "834c03b008544b39a8c41f92d91b9017"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "config.json:   0%|          | 0.00/482 [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "391da2f76d0f4be1bc5758b2a3bc3118"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "vocab.json:   0%|          | 0.00/899k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "9133bd0248ef4f07a083b7c756ecac58"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "d490c04d6f3e47328bf442eaeb5db7c1"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "6a36b773606f47148c2e3f427956eda2"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/1.42G [00:00<?, ?B/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "8f42fb4d6c82470e9f8499236d9b771f"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Some weights of RobertaModel were not initialized from the model checkpoint at roberta-large and are newly initialized: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "{'precision': [0.8771520853042603], 'recall': [0.8769662380218506], 'f1': [0.8770591616630554], 'hashcode': 'roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.40.0.dev0)'}\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Cohere Marketing Sample\n",
    "rag_2_cohere_mkt_chain = (\n",
    "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
    "    | rag_2_mkt_prompt\n",
    "    | cohere_chat_model\n",
    "    | output_parser\n",
    ")\n",
    "\n",
    "response_2_mkt_cohere = rag_2_cohere_mkt_chain.invoke(rag_2_sample_q)\n",
    "\n",
    "print(\"Cohere Response:\", response_2_mkt_cohere)\n",
    "print()\n",
    "print(\"Gold:\", gold_2_mkt)\n",
    "\n",
    "print(\"Cohere Iteration 2 Sample Metrics - Marketing\")\n",
    "print(bleu.compute(references=[gold_2_mkt], predictions=[response_2_mkt_cohere]))\n",
    "print(rouge.compute(references=[gold_2_mkt], predictions=[response_2_mkt_cohere]))\n",
    "print(\n",
    "    bertsc.compute(\n",
    "        references=[gold_2_mkt], predictions=[response_2_mkt_cohere], lang=\"en\"\n",
    "    )\n",
    ")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "iSht9PdOfOpP",
    "outputId": "57c2d52b-2572-433a-a9fe-3eef0ea76a44"
   },
   "execution_count": 62,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Cohere Response: Large language models are trained on vast amounts of text data, allowing them to learn patterns and memorize factual knowledge. This process involves unsupervised learning, where the model identifies patterns and relationships in the data without explicit guidance. The models' capacity to learn from text is due to their ability to memorize and generalize, forming the basis for their language understanding and generation capabilities.\n",
      "\n",
      "Gold: A large language model learns from text during training by first pretraining on a diverse dataset to acquire general language knowledge, and then fine-tuning on specific tasks or demonstrations to adapt its parameters for more targeted performance.\n",
      "Cohere Iteration 2 Sample Metrics - Marketing\n",
      "{'bleu': 0.0, 'precisions': [0.18571428571428572, 0.014492753623188406, 0.0, 0.0], 'brevity_penalty': 1.0, 'length_ratio': 1.794871794871795, 'translation_length': 70, 'reference_length': 39}\n",
      "{'rouge1': 0.23529411764705882, 'rouge2': 0.04, 'rougeL': 0.1568627450980392, 'rougeLsum': 0.1568627450980392}\n",
      "{'precision': [0.8714145421981812], 'recall': [0.8646404147148132], 'f1': [0.8680142760276794], 'hashcode': 'roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.40.0.dev0)'}\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "We see that both the `rougeL` and `bleu` scores have improved from the baseline - showcasing that the prompt appears to improve the model performance. We will now run it en-batch across all validation points"
   ],
   "metadata": {
    "id": "UZYrRbGUe-dT"
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Engineering Prompt and Model Declarations\n",
    "rag_2_eng_template = \"\"\"\n",
    "[INST]\n",
    "Please answer the `question` below only based on the `context` information provided, and the rules mentioned below.\n",
    "You are a chatbot for knowledge search on \"Large Language Models\" and \"Generative AI\".\n",
    "The target audience for this is engineering and resarch professionals. You can dive deep into technical details.\n",
    "Provide a detailed response not exceeding one hundred words and reduce use of adjectives.\n",
    "Prefer responses from arxiv.org\n",
    "\n",
    "Here is a `context`: {context}\n",
    "Here is a `question`: {question}.\n",
    "[/INST]\"\"\"\n",
    "\n",
    "rag_2_eng_prompt = ChatPromptTemplate.from_template(rag_2_eng_template)\n",
    "\n",
    "rag_2_mistral_eng_chain = (\n",
    "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
    "    | rag_2_eng_prompt\n",
    "    | mistral_prompt_lc\n",
    ")\n",
    "\n",
    "rag_2_cohere_eng_chain = (\n",
    "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
    "    | rag_2_eng_prompt\n",
    "    | cohere_chat_model\n",
    "    | output_parser\n",
    ")"
   ],
   "metadata": {
    "id": "KqE20r3X65rE"
   },
   "execution_count": 63,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "validation_df = pd.read_pickle(\"mids_290_a5/val_df_base.pkl\")"
   ],
   "metadata": {
    "id": "l8eiJGxt6MiD"
   },
   "execution_count": 103,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Mistral Raw Marketing Response - Iteration 2\n",
    "validation_df[\"mistral_prompt_response_mkt_raw\"] = validation_df[\n",
    "    \"question\"\n",
    "].progress_apply(rag_2_mistral_mkt_chain.invoke)\n",
    "validation_df[\"mistral_prompt_answer_mkt\"] = (\n",
    "    validation_df[\"mistral_prompt_response_mkt_raw\"]\n",
    "    .dropna()\n",
    "    .apply(lambda cell: cell.split(\"[/INST]\")[-1])\n",
    ")"
   ],
   "metadata": {
    "id": "SZgxkUm_6FUj"
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Mistral Raw Engineering Response - Iteration 2\n",
    "validation_df[\"mistral_prompt_response_eng_raw\"] = validation_df[\n",
    "    \"question\"\n",
    "].progress_apply(rag_2_mistral_eng_chain.invoke)\n",
    "validation_df[\"mistral_prompt_answer_eng\"] = (\n",
    "    validation_df[\"mistral_prompt_response_eng_raw\"]\n",
    "    .dropna()\n",
    "    .apply(lambda cell: cell.split(\"[/INST]\")[-1])\n",
    ")"
   ],
   "metadata": {
    "id": "C3iyr6cA6xiD"
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Cohere Raw Engineering Response - Iteration 2\n",
    "def safe_call_cohere(chain, q: str) -> str:\n",
    "    \"\"\"Cohere has a rate limit of 20 rpm\"\"\"\n",
    "    time.sleep(3)\n",
    "    return chain.invoke(q)\n",
    "\n",
    "\n",
    "validation_df[\"cohere_prompt_response_eng_raw\"] = validation_df[\n",
    "    \"question\"\n",
    "].progress_apply(lambda cell: safe_call_cohere(rag_2_cohere_eng_chain, cell))"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "c7fa5aebed914a7a8119446ffe3eace1",
      "6e5cee424c9744eca134522502ffbd50",
      "8dc9b127022a435ca36a980cc2cbf480",
      "af2f82fcabf240b29c24837f49634669",
      "0b9c60753dea46c19a63f3f22056a52d",
      "f3c09a3133ab432fb977dc88e7468bab",
      "51e65deb6872469da0bfe033daf43489",
      "6167e1e7720845fc9c209acb01d7a31c",
      "5ffe56cc26254574a40aeb55c1f06250",
      "20296844674746d0b28ae8695ecb7f3b",
      "4812fba6cc2f44e08e34c41b2acfea81"
     ]
    },
    "id": "lB5rLodf7ymx",
    "outputId": "d1bf3e5b-236e-4b97-9c98-6c474147ec7e"
   },
   "execution_count": 106,
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "c7fa5aebed914a7a8119446ffe3eace1"
      }
     },
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# @title Cohere Raw Marketing Response - Iteration 2\n",
    "validation_df[\"cohere_prompt_response_mkt_raw\"] = validation_df[\n",
    "    \"question\"\n",
    "].progress_apply(lambda cell: safe_call_cohere(rag_2_cohere_mkt_chain, cell))"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "976e1a4b685b4233b472e568705e7775",
      "825132085c2a4f8992bf2f301109d606",
      "c102d7d2b15a41808ee22b1da57d6665",
      "0403fdffcf2c4846a14ad4ff062bbe7a",
      "bcdf98dda3084383bd6856ae3f57e62c",
      "3e59d8afc712401590ed2bd915df3a00",
      "81f7c1411dc94120a0e5fa088e3e1517",
      "157b31bc02734196b24dfa7bb69638f5",
      "b57015603c684ec3bb96bd5bf180f60a",
      "a0c78826c3b646e29cb68e52eff1cb31",
      "1293a890dc224affa97865081453938f"
     ]
    },
    "id": "X7vT9rdC8NPE",
    "outputId": "654bbc95-47d3-473f-fd6a-5af47b3c2981"
   },
   "execution_count": 107,
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "976e1a4b685b4233b472e568705e7775"
      }
     },
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "# Save checkpoint of data after iteration 2\n",
    "validation_df.to_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_2_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")"
   ],
   "metadata": {
    "id": "ZeyH1iyn87Yx"
   },
   "execution_count": 108,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "print(validation_df.shape)\n",
    "validation_df.head()"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 643
    },
    "id": "SZSprUECD71C",
    "outputId": "b544ca7b-1114-4930-cda7-2c149ba21b28"
   },
   "execution_count": 109,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "(75, 9)\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                                            question  \\\n",
       "0  What purpose do large language models serve in...   \n",
       "1  How does a large language model learn from tex...   \n",
       "2  What are some key architectures behind the dev...   \n",
       "3  Can you name some specific large language mode...   \n",
       "7  What licensing models have been adopted for th...   \n",
       "\n",
       "                                gold_answer_research  \\\n",
       "0  Large language models (LLMs) serve the purpose...   \n",
       "1  A large language model learns from text during...   \n",
       "2  Key architectures behind the development of la...   \n",
       "3  Some specific large language models include GP...   \n",
       "7  Based on the provided context, it seems that l...   \n",
       "\n",
       "                               gold_answer_marketing  \\\n",
       "0  Large language models serve the purpose of imp...   \n",
       "1  A large language model learns from text during...   \n",
       "2  Key architectures behind the development of la...   \n",
       "3           Chinchilla by DeepMind, GPT-3 by OpenAI.   \n",
       "7  Answer: Some organizations choose open-sourcin...   \n",
       "\n",
       "                     mistral_prompt_response_mkt_raw  \\\n",
       "0  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "1  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "2  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "3  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "7  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "\n",
       "                           mistral_prompt_answer_mkt  \\\n",
       "0   Large language models serve as powerful tools...   \n",
       "1   Large language models learn through a process...   \n",
       "2   Some key architectures include transformer-ba...   \n",
       "3   Some specific large language models include B...   \n",
       "7   Licensing models for source-available languag...   \n",
       "\n",
       "                     mistral_prompt_response_eng_raw  \\\n",
       "0  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "1  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "2  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "3  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "7  Human: \\n[INST]\\nPlease answer the `question` ...   \n",
       "\n",
       "                           mistral_prompt_answer_eng  \\\n",
       "0   Large language models (LLMs) are used to gene...   \n",
       "1   Large language models learn from text through...   \n",
       "2   The development of large language models invo...   \n",
       "3   Some specific large language models include B...   \n",
       "7   There has been no specific licensing model ad...   \n",
       "\n",
       "                      cohere_prompt_response_eng_raw  \\\n",
       "0  Large language models (LLMs) have become integ...   \n",
       "1  Large language models learn from text through ...   \n",
       "2  Some key architectures that have played a sign...   \n",
       "3  Some prominent examples of large language mode...   \n",
       "7  The licensing of source-available language mod...   \n",
       "\n",
       "                      cohere_prompt_response_mkt_raw  \n",
       "0  Large language models (LLMs) are an essential ...  \n",
       "1  Large Language Models (LLMs) learn by ingestin...  \n",
       "2  Some key architectures that have driven the de...  \n",
       "3  Some prominent examples of large language mode...  \n",
       "7  The licensing models adopted for distributing ...  "
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-7d8cebae-fd1d-400e-90b6-7b73258764b0\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>question</th>\n",
       "      <th>gold_answer_research</th>\n",
       "      <th>gold_answer_marketing</th>\n",
       "      <th>mistral_prompt_response_mkt_raw</th>\n",
       "      <th>mistral_prompt_answer_mkt</th>\n",
       "      <th>mistral_prompt_response_eng_raw</th>\n",
       "      <th>mistral_prompt_answer_eng</th>\n",
       "      <th>cohere_prompt_response_eng_raw</th>\n",
       "      <th>cohere_prompt_response_mkt_raw</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>What purpose do large language models serve in...</td>\n",
       "      <td>Large language models (LLMs) serve the purpose...</td>\n",
       "      <td>Large language models serve the purpose of imp...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>Large language models serve as powerful tools...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>Large language models (LLMs) are used to gene...</td>\n",
       "      <td>Large language models (LLMs) have become integ...</td>\n",
       "      <td>Large language models (LLMs) are an essential ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>How does a large language model learn from tex...</td>\n",
       "      <td>A large language model learns from text during...</td>\n",
       "      <td>A large language model learns from text during...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>Large language models learn through a process...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>Large language models learn from text through...</td>\n",
       "      <td>Large language models learn from text through ...</td>\n",
       "      <td>Large Language Models (LLMs) learn by ingestin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>What are some key architectures behind the dev...</td>\n",
       "      <td>Key architectures behind the development of la...</td>\n",
       "      <td>Key architectures behind the development of la...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>Some key architectures include transformer-ba...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>The development of large language models invo...</td>\n",
       "      <td>Some key architectures that have played a sign...</td>\n",
       "      <td>Some key architectures that have driven the de...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Can you name some specific large language mode...</td>\n",
       "      <td>Some specific large language models include GP...</td>\n",
       "      <td>Chinchilla by DeepMind, GPT-3 by OpenAI.</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>Some specific large language models include B...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>Some specific large language models include B...</td>\n",
       "      <td>Some prominent examples of large language mode...</td>\n",
       "      <td>Some prominent examples of large language mode...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>What licensing models have been adopted for th...</td>\n",
       "      <td>Based on the provided context, it seems that l...</td>\n",
       "      <td>Answer: Some organizations choose open-sourcin...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>Licensing models for source-available languag...</td>\n",
       "      <td>Human: \\n[INST]\\nPlease answer the `question` ...</td>\n",
       "      <td>There has been no specific licensing model ad...</td>\n",
       "      <td>The licensing of source-available language mod...</td>\n",
       "      <td>The licensing models adopted for distributing ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7d8cebae-fd1d-400e-90b6-7b73258764b0')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-7d8cebae-fd1d-400e-90b6-7b73258764b0 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-7d8cebae-fd1d-400e-90b6-7b73258764b0');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-8d5e10db-a911-44f5-be05-e2e02ef39e17\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-8d5e10db-a911-44f5-be05-e2e02ef39e17')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-8d5e10db-a911-44f5-be05-e2e02ef39e17 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "dataframe",
       "variable_name": "validation_df",
       "summary": "{\n  \"name\": \"validation_df\",\n  \"rows\": 75,\n  \"fields\": [\n    {\n      \"column\": \"question\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"What licensing models have been adopted for the distribution of source-available language models?\",\n          \"What is the main goal of prompt engineering in language models?\",\n          \"What factors influenced the development of generative language models by Anthropic?\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"gold_answer_research\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"Based on the provided context, it seems that licensing models for the distribution of source-available language models have not been explicitly discussed in the referenced papers. However, it is crucial to consider potential licensing options such as open-source licenses (e.g., GPL, MIT) or proprietary licenses when distributing language models to ensure legal compliance and control over usage rights. Additionally, considering the implications of different licensing models on accessibility, collaboration, and commercialization is essential for determining the most suitable approach for sharing language models with the community. Further research or consultation with legal experts may be necessary to explore specific licensing strategies for source-available language models.\",\n          \"The main goal of prompt engineering in language models is to effectively steer the behavior of the model towards desired outcomes without updating the model weights. This is achieved by composing and formatting prompts in a way that maximizes the model's performance on a specific task. Prompt engineering involves treating prompts as trainable parameters and optimizing them directly on the embedding space through methods like AutoPrompt, Prefix-Tuning, P-tuning, and Prompt-Tuning. The ultimate aim is to enhance the model's performance and alignment with user-defined tasks.\",\n          \"Several factors influenced the development of generative language models by Anthropic, including the limitations in coding, math, and reasoning capabilities of the initial version Claude, the partnerships with companies like Notion and Quora to enhance the model's capabilities, and the need to address biases, unsafe content, and ethical considerations in training data. Additionally, the reliance on supervised learning and the need for controlled generation in generative models played a role in shaping the development of Anthropic's language models.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"gold_answer_marketing\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"Answer: Some organizations choose open-sourcing, while others restrict access to a few organizations with resources or offer end-to-end deployment via API.\",\n          \"The main goal of prompt engineering in language models is to steer the behavior of the model for desired outcomes without updating the model weights.\",\n          \"Factors that influenced the development of generative language models by Anthropic include partnerships with companies like Notion and Quora, limitations in coding, math, and reasoning capabilities in initial models like Claude, and the need to address biases and unsafe content in training datasets.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"mistral_prompt_response_mkt_raw\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"Human: \\n[INST]\\nPlease answer the `question` below only based on the `context` information provided, and the rules mentioned below.\\nYou are a chatbot for knowledge search on \\\"Large Language Models\\\" and \\\"Generative AI\\\".\\nThe target audience for this is marketing professionals.\\nProvide a detailed response not exceeding sixty words and reduce use of adjectives.\\n\\nHere is a `context`: the largest open-source language models\\n\\nIn the meantime, competing language models have for the most\\n\\nof generic language models for\\n\\nour current language model systems, we seek general and scalable methods that work for future AI\\nsystems (Leike et al., 2018). The systems we work with here are still fairly limited, but they are\\nHere is a `question`: What licensing models have been adopted for the distribution of source-available language models?.\\n[/INST] Licensing models for source-available language models include permissive licenses such as MIT or Apache, which allow for free use, modification, and redistribution of the codebase. Other options may involve commercial agreements or proprietary restrictions to protect intellectual property rights.\",\n          \"Human: \\n[INST]\\nPlease answer the `question` below only based on the `context` information provided, and the rules mentioned below.\\nYou are a chatbot for knowledge search on \\\"Large Language Models\\\" and \\\"Generative AI\\\".\\nThe target audience for this is marketing professionals.\\nProvide a detailed response not exceeding sixty words and reduce use of adjectives.\\n\\nHere is a `context`: or multimodality models. At its core, the goal of prompt\\n\\nPrompt Engineering\\n\\nbecause the language modeling objective\\n\\ncan be beneficial for instruction-finetuned language\\nmodels.\\nHere is a `question`: What is the main goal of prompt engineering in language models?.\\n[/INST] The main goal of prompt engineering in language models is to improve their ability to generate relevant responses to specific tasks by fine-tuning them with instructions.\",\n          \"Human: \\n[INST]\\nPlease answer the `question` below only based on the `context` information provided, and the rules mentioned below.\\nYou are a chatbot for knowledge search on \\\"Large Language Models\\\" and \\\"Generative AI\\\".\\nThe target audience for this is marketing professionals.\\nProvide a detailed response not exceeding sixty words and reduce use of adjectives.\\n\\nHere is a `context`: pre-trained, generative language models.\\n\\nClaude was the initial version of Anthropic's language model\\n\\nlanguage models from human preferences, including training generative models in other modalities.\\nAcknowledgements\\nEM gratefully acknowledges funding from a Knight-Hennessy Graduate Fellowship. CF and CM\\n\\nversatile generative language model\\nHere is a `question`: What factors influenced the development of generative language models by Anthropic?.\\n[/INST] The development of generative language models by Anthropic was influenced by their versatility and ability to generate text from human preferences. Claude, the initial version, demonstrated these capabilities and inspired further research into using language models in various modalities.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"mistral_prompt_answer_mkt\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \" Licensing models for source-available language models include permissive licenses such as MIT or Apache, which allow for free use, modification, and redistribution of the codebase. Other options may involve commercial agreements or proprietary restrictions to protect intellectual property rights.\",\n          \" The main goal of prompt engineering in language models is to improve their ability to generate relevant responses to specific tasks by fine-tuning them with instructions.\",\n          \" The development of generative language models by Anthropic was influenced by their versatility and ability to generate text from human preferences. Claude, the initial version, demonstrated these capabilities and inspired further research into using language models in various modalities.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"mistral_prompt_response_eng_raw\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"Human: \\n[INST]\\nPlease answer the `question` below only based on the `context` information provided, and the rules mentioned below.\\nYou are a chatbot for knowledge search on \\\"Large Language Models\\\" and \\\"Generative AI\\\".\\nThe target audience for this is engineering and resarch professionals. You can dive deep into technical details.\\nProvide a detailed response not exceeding one hundred words and reduce use of adjectives.\\nPrefer responses from arxiv.org\\n\\nHere is a `context`: the largest open-source language models\\n\\nIn the meantime, competing language models have for the most\\n\\nof generic language models for\\n\\nour current language model systems, we seek general and scalable methods that work for future AI\\nsystems (Leike et al., 2018). The systems we work with here are still fairly limited, but they are\\nHere is a `question`: What licensing models have been adopted for the distribution of source-available language models?.\\n[/INST] There has been no specific licensing model adopted for the distribution of source-available large language models in the context of research and engineering communities. However, many popular language models such as BERT and RoBERTa are released under permissive licenses like Apache 2.0 or MIT, which allow for free use, modification, and redistribution. These licenses encourage collaboration and innovation within the community while also ensuring that the models remain accessible to all users.\",\n          \"Human: \\n[INST]\\nPlease answer the `question` below only based on the `context` information provided, and the rules mentioned below.\\nYou are a chatbot for knowledge search on \\\"Large Language Models\\\" and \\\"Generative AI\\\".\\nThe target audience for this is engineering and resarch professionals. You can dive deep into technical details.\\nProvide a detailed response not exceeding one hundred words and reduce use of adjectives.\\nPrefer responses from arxiv.org\\n\\nHere is a `context`: or multimodality models. At its core, the goal of prompt\\n\\nPrompt Engineering\\n\\nbecause the language modeling objective\\n\\ncan be beneficial for instruction-finetuned language\\nmodels.\\nHere is a `question`: What is the main goal of prompt engineering in language models?.\\n[/INST] The main goal of prompt engineering in language models is to improve their ability to generate coherent and relevant text based on specific instructions or tasks. This involves designing prompts that effectively guide the model towards generating desired outputs, such as summarizing long documents or answering complex questions. By fine-tuning language models with task-specific prompts, researchers have shown significant improvements in performance compared to generic language models.\",\n          \"Human: \\n[INST]\\nPlease answer the `question` below only based on the `context` information provided, and the rules mentioned below.\\nYou are a chatbot for knowledge search on \\\"Large Language Models\\\" and \\\"Generative AI\\\".\\nThe target audience for this is engineering and resarch professionals. You can dive deep into technical details.\\nProvide a detailed response not exceeding one hundred words and reduce use of adjectives.\\nPrefer responses from arxiv.org\\n\\nHere is a `context`: pre-trained, generative language models.\\n\\nClaude was the initial version of Anthropic's language model\\n\\nlanguage models from human preferences, including training generative models in other modalities.\\nAcknowledgements\\nEM gratefully acknowledges funding from a Knight-Hennessy Graduate Fellowship. CF and CM\\n\\nversatile generative language model\\nHere is a `question`: What factors influenced the development of generative language models by Anthropic?.\\n[/INST] The development of generative language models by Anthropic was influenced by several factors such as advancements in natural language processing (NLP) techniques, availability of large amounts of text data, and the desire to create versatile models that could generate coherent and contextually relevant text. Additionally, the team at Anthropic aimed to push the boundaries of what was possible with NLP models and explore new applications and possibilities within the field.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"mistral_prompt_answer_eng\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \" There has been no specific licensing model adopted for the distribution of source-available large language models in the context of research and engineering communities. However, many popular language models such as BERT and RoBERTa are released under permissive licenses like Apache 2.0 or MIT, which allow for free use, modification, and redistribution. These licenses encourage collaboration and innovation within the community while also ensuring that the models remain accessible to all users.\",\n          \" The main goal of prompt engineering in language models is to improve their ability to generate coherent and relevant text based on specific instructions or tasks. This involves designing prompts that effectively guide the model towards generating desired outputs, such as summarizing long documents or answering complex questions. By fine-tuning language models with task-specific prompts, researchers have shown significant improvements in performance compared to generic language models.\",\n          \" The development of generative language models by Anthropic was influenced by several factors such as advancements in natural language processing (NLP) techniques, availability of large amounts of text data, and the desire to create versatile models that could generate coherent and contextually relevant text. Additionally, the team at Anthropic aimed to push the boundaries of what was possible with NLP models and explore new applications and possibilities within the field.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"cohere_prompt_response_eng_raw\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"The licensing of source-available language models is an important aspect of their distribution and usage. In the context of large language models and generative AI, there has been a trend towards open-sourcing these models to facilitate collaboration and improvement by the research community. \\n\\nOne popular license adopted for source-available language models is the MIT License, which is simple and permissive, allowing for usage in a wide range of projects with minimal restrictions. This license is used by prominent open-source language models like GPT-3's successor, GPT-Neo, and the GPT-J model. \\n\\nOther licenses adopted include the Apache 2.0 License and the Creative Commons licenses, which offer different levels of permissions and restrictions.\",\n          \"The main goal of prompt engineering in language models is to optimize the prompt, or input to a language model, to improve its performance and adaptability to specific tasks. Prompt engineering techniques aim to provide clear and effective instructions to the language model, enhancing its understanding and generation of responses, particularly for instruction-finetuned models. This field of engineering is an important area of research to improve the flexibility and applicability of language models in various real-world applications.\",\n          \"The development of generative language models by Anthropic was influenced by the idea of training such models based on human preferences. This is evident from the acknowledgment of EM's Knight-Hennessy Graduate Fellowship, which likely provided the resources and research freedom to explore this direction. Anthropic's initial language model, Claude, was likely a testbed for these ideas, with the ultimate goal of creating a versatile and adaptable generative language model.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"cohere_prompt_response_mkt_raw\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"The licensing models adopted for distributing source-available language models vary. Some models are released under open-source licenses, like the GPL or MIT license, fostering a community of contributors and enabling wider adoption. Other models use proprietary licenses, with restricted access and commercial terms for usage and distribution.\",\n          \"The main goal of prompt engineering in language models is to enhance the performance of language models by providing them with more effective prompts or input patterns. Prompt engineering techniques aim to improve the model's understanding and generation of language, making it more adaptable to different tasks and improving its overall language capabilities.\",\n          \"Funding and research in language models from human preferences influenced the development of generative language models by Anthropic. The initial version of their language model, Claude, was trained on a diverse set of human preferences to make it versatile and adaptable to different use cases.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
      }
     },
     "metadata": {},
     "execution_count": 109
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "validation_df: pd.DataFrame = pd.read_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_2_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")\n",
    "validation_df = assign_metrics(\n",
    "    validation_df,\n",
    "    iteration=2,\n",
    "    col_names_iter={\n",
    "        \"eng\": {\n",
    "            \"mistral\": \"mistral_prompt_answer_eng\",\n",
    "            \"cohere\": \"cohere_prompt_response_eng_raw\",\n",
    "        },\n",
    "        \"mkt\": {\n",
    "            \"mistral\": \"mistral_prompt_answer_mkt\",\n",
    "            \"cohere\": \"cohere_prompt_response_mkt_raw\",\n",
    "        },\n",
    "    },\n",
    ")\n",
    "validation_df.describe().transpose().round(2)"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 681,
     "referenced_widgets": [
      "5778db39bc114992a8f6c31b302bb44b",
      "712d7cffcc5f4495b88190ffa245c587",
      "923575169bcb4b7d8612028b1738a98a",
      "2aa6158ab8cf4663b2fb2e2a76d75774",
      "c58a08489d26472e84a2b237ff99d2ef",
      "77f990018462443c8f1afe29e7122c41",
      "86c0789f0d954fdcbceb9346a0fd50e1",
      "25baed0450454523aaef42522d2c25c2",
      "83d86f4a3e524feaaf967327ae91c133",
      "ace18903d734403a9e2b2979a3bb6ed3",
      "5afa6c6bc56a4b0aa4467c47c72b56fa",
      "72d2621aa2cc46128fb80e642870613e",
      "52e0d474b14e411d94d44471adc51d3e",
      "a3f0fdef77484329a0b85fd8b72a002f",
      "e0e08b2a73464e1fb770195ccbf63705",
      "adca83dd502744f0b2c3cb6c32d62c95",
      "902ae631c3504613ac48a03ce49cb3cd",
      "864454ad08b0462b8a1177ff976e097d",
      "5f57760bc4d84024a81fee52009586a9",
      "afdc394622a4403aa50be734a69c35d3",
      "f1576ba2ec794d55a59d7472691f9339",
      "75e1177de3db44eab0d0ac2ce216d4c4",
      "c3be67afe07d47e38ea9c45569939469",
      "5cf7acd34f1c4fadab40efc2d46cfae5",
      "f6562ce628ca4e14a0edbee346ba14ee",
      "2003ecee31914f43a81048562c25fbb9",
      "cd46cb9433c94d60ae560b279878c74c",
      "6a7714dce29740b593e6663a664b038a",
      "5c79112b538d44738ef1c60dc42b9c9b",
      "3679dcf8a39c49a8863ba16674bf64ca",
      "8e762196f7d94891a63b2a64800e59fc",
      "e52485bbbab44e0ab6f916bbc02e83c1",
      "fcdf13af13994b05a091a8026a1c0938",
      "8df40ee934674cad9d61f11ce3e0db21",
      "416ff5fc14f141e9801a7e62d71aca98",
      "13f885c14b4a4317a59f5c79ffedad9c",
      "38ccf2ce2bb7472f92d33949abec18d7",
      "3b646c078cc848419cbafd591a4081b1",
      "a0b603b86643417b923e91c6bd33a7ee",
      "75fa22125ffd48d58bde40b6d8ac4990",
      "d7580f74a21b40fdbaff386ca0bcb888",
      "85ab9557cb5d4f6aacb5c36f4186d9aa",
      "c391d01b0e1a43c7840175b51c0fbfbc",
      "cf72beb3ea604d38ab4da728c9419e5b",
      "e85c9670afe7406b8c94a1b9607371d0",
      "b132f55973fd4a42a8a563895c26b0dd",
      "3e36367514f0494c964834deebc0dd55",
      "f78e6da9b092440f9fb482d55ea091b8",
      "5b0f88ad2d1045df85ad4c9b36243c85",
      "d4d67fd96ac0426abe408d39ec73f162",
      "bee89ff017bc4dd8b928f33e0ea7eae9",
      "50089f650cc54427951644274bff6ea1",
      "1189d28a99534bd78ea42785c0406306",
      "132d799e7d984c16a8fe742aeba6e022",
      "cf280191ef2d44caa80ca561ba6b266b",
      "fce574ee4bd7427eb6e4e8f2afd587fb",
      "2b05f0a04cec459884f620fdd34ac7e4",
      "8e48d65637784406b250bf2b7f495fcb",
      "313422453c784d039a249f372ee2ca51",
      "91e8247ada37499481adf8674491855f",
      "db32d583fb8144b789910e6df68ca397",
      "a56df808bf7f4c30b1639906b285cea1",
      "c373d163413540fa885826f6d1367372",
      "11203e680c3740b182f09910410c979b",
      "9c33dc99b6524ec487075ef75511791e",
      "d972c0cf99144713a18a2c4afeacd7e8",
      "ed0a34cdf8cd4e5c9c3cfce43f9b25ca",
      "f1ec6e5a37954e02a70f4002979778c7",
      "96050a1be9db4178a24fb0a5da3cb9b8",
      "3c7e805965eb4922bda7b9d723d8bbbb",
      "b5a9251e0fae44168ba11bc78427e467",
      "f4ebe2b010744a3db4999b6d222d1224",
      "39cfffdd84b84131a3449f6aa58ad0c8",
      "b5a2cf6897324660b2e10b24d21cdabb",
      "ede15688e7b243fea67c88024c495ed2",
      "3d308556fa5a4ef196fe084981d59089",
      "6cdf6a5fad4041baa7028230d3485fb1",
      "b1f670c6832f45a8b5e3ad62d9005569",
      "6b517ad5f23643579e0b9e76dc00c588",
      "f42dc020ee814f39a33370a6780d82e8",
      "e4b463ba77af4fa38091da61ba9234b7",
      "a61471851fcf404eaf9e90d692034068",
      "e23cee15d88b45bcb22edf56b7066903",
      "0bca12f762344c039cf5850eefe2787e",
      "b2476e55bed24a8b8b18913ce29e2ff9",
      "a3033e42410945e189d3c70e15a5fde8",
      "1595a07cdc524855b60b8e20c78dba82",
      "7835ddde624c4dbdb02a016d19829dd9"
     ]
    },
    "id": "g_-fFPzLEb7t",
    "outputId": "c5dc6153-f642-4e74-8688-283b414ce3da"
   },
   "execution_count": 110,
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "5778db39bc114992a8f6c31b302bb44b"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "72d2621aa2cc46128fb80e642870613e"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "c3be67afe07d47e38ea9c45569939469"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "8df40ee934674cad9d61f11ce3e0db21"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "e85c9670afe7406b8c94a1b9607371d0"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "fce574ee4bd7427eb6e4e8f2afd587fb"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "ed0a34cdf8cd4e5c9c3cfce43f9b25ca"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "  0%|          | 0/75 [00:00<?, ?it/s]"
      ],
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "b1f670c6832f45a8b5e3ad62d9005569"
      }
     },
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                        count  mean   std   min   25%   50%   75%   max\n",
       "mistral_bleu_eng_it_2    75.0  0.10  0.08  0.00  0.05  0.10  0.14  0.30\n",
       "mistral_rouge_eng_it_2   75.0  0.26  0.07  0.12  0.20  0.25  0.30  0.45\n",
       "mistral_bert_eng_it_2    75.0  0.88  0.01  0.85  0.87  0.89  0.90  0.91\n",
       "mistral_bleu_mkt_it_2    75.0  0.11  0.12  0.00  0.00  0.09  0.15  0.56\n",
       "mistral_rouge_mkt_it_2   75.0  0.30  0.12  0.05  0.21  0.29  0.38  0.72\n",
       "mistral_bert_mkt_it_2    75.0  0.89  0.02  0.84  0.88  0.89  0.91  0.96\n",
       "cohere_bleu_eng_it_2     75.0  0.08  0.07  0.00  0.05  0.09  0.12  0.38\n",
       "cohere_rouge_eng_it_2    75.0  0.26  0.06  0.14  0.22  0.25  0.29  0.49\n",
       "cohere_bert_eng_it_2     75.0  0.88  0.02  0.84  0.87  0.88  0.89  0.93\n",
       "cohere_bleu_mkt_it_2     75.0  0.09  0.09  0.00  0.00  0.08  0.14  0.43\n",
       "cohere_rouge_mkt_it_2    75.0  0.28  0.10  0.04  0.21  0.29  0.35  0.62\n",
       "cohere_bert_mkt_it_2     75.0  0.89  0.02  0.83  0.87  0.89  0.90  0.96"
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-74e10db7-42e1-4a97-9287-77d07cc2a5b4\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>mistral_bleu_eng_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.10</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.05</td>\n",
       "      <td>0.10</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_rouge_eng_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.26</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.20</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0.30</td>\n",
       "      <td>0.45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_bert_eng_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.88</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.85</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.89</td>\n",
       "      <td>0.90</td>\n",
       "      <td>0.91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_bleu_mkt_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.11</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.15</td>\n",
       "      <td>0.56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_rouge_mkt_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.30</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.05</td>\n",
       "      <td>0.21</td>\n",
       "      <td>0.29</td>\n",
       "      <td>0.38</td>\n",
       "      <td>0.72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mistral_bert_mkt_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.89</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.84</td>\n",
       "      <td>0.88</td>\n",
       "      <td>0.89</td>\n",
       "      <td>0.91</td>\n",
       "      <td>0.96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_bleu_eng_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.05</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_rouge_eng_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.26</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.22</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0.29</td>\n",
       "      <td>0.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_bert_eng_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.88</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.84</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.88</td>\n",
       "      <td>0.89</td>\n",
       "      <td>0.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_bleu_mkt_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_rouge_mkt_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.10</td>\n",
       "      <td>0.04</td>\n",
       "      <td>0.21</td>\n",
       "      <td>0.29</td>\n",
       "      <td>0.35</td>\n",
       "      <td>0.62</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cohere_bert_mkt_it_2</th>\n",
       "      <td>75.0</td>\n",
       "      <td>0.89</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.83</td>\n",
       "      <td>0.87</td>\n",
       "      <td>0.89</td>\n",
       "      <td>0.90</td>\n",
       "      <td>0.96</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-74e10db7-42e1-4a97-9287-77d07cc2a5b4')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-74e10db7-42e1-4a97-9287-77d07cc2a5b4 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-74e10db7-42e1-4a97-9287-77d07cc2a5b4');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-a99479d6-7d9a-486e-8b5c-ad2e12db78c8\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a99479d6-7d9a-486e-8b5c-ad2e12db78c8')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-a99479d6-7d9a-486e-8b5c-ad2e12db78c8 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "dataframe",
       "summary": "{\n  \"name\": \"validation_df\",\n  \"rows\": 12,\n  \"fields\": [\n    {\n      \"column\": \"count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.0,\n        \"min\": 75.0,\n        \"max\": 75.0,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          75.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"mean\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.35331760971303616,\n        \"min\": 0.08,\n        \"max\": 0.89,\n        \"num_unique_values\": 9,\n        \"samples\": [\n          0.09\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"std\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.03965762565670407,\n        \"min\": 0.01,\n        \"max\": 0.12,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          0.07\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"min\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.39470261148762553,\n        \"min\": 0.0,\n        \"max\": 0.85,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          0.12\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"25%\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.38034573586130654,\n        \"min\": 0.0,\n        \"max\": 0.88,\n        \"num_unique_values\": 7,\n        \"samples\": [\n          0.05\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"50%\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.3569430393400792,\n        \"min\": 0.08,\n        \"max\": 0.89,\n        \"num_unique_values\": 7,\n        \"samples\": [\n          0.1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"75%\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.3389679076612452,\n        \"min\": 0.12,\n        \"max\": 0.91,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          0.89\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"max\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.2451020195755229,\n        \"min\": 0.3,\n        \"max\": 0.96,\n        \"num_unique_values\": 11,\n        \"samples\": [\n          0.96\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
      }
     },
     "metadata": {},
     "execution_count": 110
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "validation_df.to_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_2_METRICS_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")\n",
    "validation_df.describe().transpose().round(2).to_pickle(\n",
    "    f\"mids_290_a5/val_df_iteration_2_DESCRIBE_METRICS_CHUNK_{CHUNK_SIZE}_OVR_{OVERLAP}.pkl\"\n",
    ")"
   ],
   "metadata": {
    "id": "IzzmXWSoGpre"
   },
   "execution_count": 111,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JdOhlN23AYiA"
   },
   "source": [
    "##5. Results\n",
    "\n",
    "###5.1 Model Specifications\n",
    "\n",
    "Document the detailed specs of your choices. Also comment on how you valued the needs of the marketing tean vs the needs of the researchers, in case you had to make a trade-off.\n",
    "\n",
    "\n",
    "> Marketing and Engineering got separate chains per model, with separate prompts. The Engineering prompt valued technical details and a higher word limit, while marketing used a lower word count limit reducing technical details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "id": "DSoSdUriAa-C",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "6d3b68be-94b2-40ee-df07-402a943d5446"
   },
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "{'mistral_eng': {\n",
       "   context: VectorStoreRetriever(tags=['Qdrant', 'HuggingFaceEmbeddings'], vectorstore=<langchain_community.vectorstores.qdrant.Qdrant object at 0x7d55fafc0820>)\n",
       "            | RunnableLambda(format_docs),\n",
       "   question: RunnablePassthrough()\n",
       " }\n",
       " | ChatPromptTemplate(input_variables=['context', 'question'], messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['context', 'question'], template='\\n[INST]\\nPlease answer the `question` below only based on the `context` information provided, and the rules mentioned below.\\nYou are a chatbot for knowledge search on \"Large Language Models\" and \"Generative AI\".\\nThe target audience for this is engineering and resarch professionals. You can dive deep into technical details.\\nProvide a detailed response not exceeding one hundred words and reduce use of adjectives.\\nPrefer responses from arxiv.org\\n\\nHere is a `context`: {context}\\nHere is a `question`: {question}.\\n[/INST]'))])\n",
       " | HuggingFacePipeline(pipeline=<transformers.pipelines.text_generation.TextGenerationPipeline object at 0x7d55cd38f2b0>),\n",
       " 'mistral_mkt': {\n",
       "   context: VectorStoreRetriever(tags=['Qdrant', 'HuggingFaceEmbeddings'], vectorstore=<langchain_community.vectorstores.qdrant.Qdrant object at 0x7d55fafc0820>)\n",
       "            | RunnableLambda(format_docs),\n",
       "   question: RunnablePassthrough()\n",
       " }\n",
       " | ChatPromptTemplate(input_variables=['context', 'question'], messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['context', 'question'], template='\\n[INST]\\nPlease answer the `question` below only based on the `context` information provided, and the rules mentioned below.\\nYou are a chatbot for knowledge search on \"Large Language Models\" and \"Generative AI\".\\nThe target audience for this is marketing professionals.\\nProvide a detailed response not exceeding sixty words and reduce use of adjectives.\\n\\nHere is a `context`: {context}\\nHere is a `question`: {question}.\\n[/INST]'))])\n",
       " | HuggingFacePipeline(pipeline=<transformers.pipelines.text_generation.TextGenerationPipeline object at 0x7d55cd38f2b0>)}"
      ]
     },
     "metadata": {},
     "execution_count": 73
    }
   ],
   "source": [
    "{\n",
    "    \"mistral_eng\": rag_2_mistral_eng_chain,\n",
    "    \"mistral_mkt\": rag_2_mistral_mkt_chain,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L7J41pWiyh06"
   },
   "source": [
    "\n",
    "###5.2 Some Test Questions\n",
    "\n",
    "**QUESTIONS:**\n",
    "\n",
    "\n",
    "Please study the answers generated by your chosen setup for these specific test questions:\n",
    "\n",
    "1. \"What purpose do large language models serve in the field of natural language processing?\" (Question 0)\n",
    "\n",
    "2. \"What methods are typically employed to create training data for embedding models that use task-specific instructions?\" (Question 50)\n",
    "\n",
    "3. \"How does a model's ability to answer questions relate to its exposure to specific types of questions during training?\" (Question 83, no labeled answers)\n",
    "\n",
    "For each of the three questions above please provide:\n",
    "\n",
    "a) The RAG results (research and marketing response)  \n",
    "b) The context provided  \n",
    "c) The document sources for the context  \n",
    "d) Also discuss your metric(s) for the first two examples (for both responses) compared to the gold responses\n",
    "\n",
    "Then, for questions 1 and 2, comment on how well you feel your metrics captured the differences and similarities between your answer and the gold answer?\n",
    "\n",
    "Put your answers to these questions into the answers file as you have done on previous assignments. Please consult the answer file for further details.\n",
    "\n",
    "####5.2.1 Test Question 1\n",
    "\n",
    "Please run the query:\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "id": "fgJbWDR7Ab-K",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "e543478f-3749-4b5c-99e8-bd830247928b"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    }
   ],
   "source": [
    "tq1: str = validation_df.loc[0][\"question\"]\n",
    "print(tq1)\n",
    "tq1_eng_response: str = rag_2_mistral_eng_chain.invoke(tq1)\n",
    "tq1_mkt_response: str = rag_2_mistral_mkt_chain.invoke(tq1)"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(\"A: Engineering Response\", tq1_eng_response.split(\"[/INST]\")[-1])\n",
    "print(\"A: Marketing Response\", tq1_mkt_response.split(\"[/INST]\")[-1])"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KSBUvJEL9K-T",
    "outputId": "312c0080-4a18-463f-93c1-d312d071be3d"
   },
   "execution_count": 68,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "A: Engineering Response  Large language models serve as powerful tools in the field of natural language processing (NLP) by enabling various language tasks such as classification, summarization, question-answering, creative writing, and dialogue generation. These models are pre-trained on vast amounts of unsupervised textual data and possess the ability to memorize factual knowledge through their parameter weights. As a result, they can be used to perform a wide range of NLP tasks with high accuracy and efficiency.\n",
      "\n",
      "A: Marketing Response  Large language models serve as powerful tools in natural language processing by enabling various language tasks such as classification, summarization, question-answering, creative writing, and dialogue. They achieve this through being pre-trained on vast amounts of unsupervised textual data and possessing sufficient parameters to store factual knowledge.\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(\"B,C: Engineering Context\")\n",
    "print(tq1_eng_response.split(\"`question`:\")[0])"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dRQN2ZCk9Xpa",
    "outputId": "4561d28c-69dd-4cdb-e65d-3723e8185ede"
   },
   "execution_count": 71,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Human: \n",
      "[INST]\n",
      "Please answer the `question` below only based on the `context` information provided, and the rules mentioned below.\n",
      "You are a chatbot for knowledge search on \"Large Language Models\" and \"Generative AI\".\n",
      "The target audience for this is engineering and resarch professionals. You can dive deep into technical details.\n",
      "Provide a detailed response not exceeding one hundred words and reduce use of adjectives.\n",
      "Prefer responses from arxiv.org\n",
      "\n",
      "Here is a `context`: === Large language models ===\n",
      "\n",
      "limitations, and societal impact of large language models. arXiv preprint arXiv:2102.02503.\n",
      "Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos,\n",
      "\n",
      "among the largest language models today and we apply them on a wide range of language tasks,\n",
      "including classiﬁcation, summarization, question-answering, creative writing, dialogue, and others.\n",
      "\n",
      "Big language models have been pre-trained on a large collection of unsupervised textual corpus. Given enough parameters, these models are able to memorize some factual knowledge within parameter weights. Therefore, we can use these models to do\n",
      "Here is a \n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(\"B,C: Marketing Context\")\n",
    "print(tq1_mkt_response.split(\"`question`:\")[0])"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dGWSQc9j9z7l",
    "outputId": "5ebe3e1c-8dc8-4ddd-f757-03c404569974"
   },
   "execution_count": 72,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "B: Marketing Context\n",
      "Human: \n",
      "[INST]\n",
      "Please answer the `question` below only based on the `context` information provided, and the rules mentioned below.\n",
      "You are a chatbot for knowledge search on \"Large Language Models\" and \"Generative AI\".\n",
      "The target audience for this is marketing professionals.\n",
      "Provide a detailed response not exceeding sixty words and reduce use of adjectives.\n",
      "\n",
      "Here is a `context`: === Large language models ===\n",
      "\n",
      "limitations, and societal impact of large language models. arXiv preprint arXiv:2102.02503.\n",
      "Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos,\n",
      "\n",
      "among the largest language models today and we apply them on a wide range of language tasks,\n",
      "including classiﬁcation, summarization, question-answering, creative writing, dialogue, and others.\n",
      "\n",
      "Big language models have been pre-trained on a large collection of unsupervised textual corpus. Given enough parameters, these models are able to memorize some factual knowledge within parameter weights. Therefore, we can use these models to do\n",
      "Here is a \n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(\"D: Marketing Metrics\")\n",
    "print(\n",
    "    \"BLEU\",\n",
    "    bleu.compute(\n",
    "        predictions=[tq1_mkt_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[0][\"gold_answer_marketing\"]],\n",
    "    )[\"bleu\"],\n",
    ")\n",
    "print(\n",
    "    \"ROUGE\",\n",
    "    rouge.compute(\n",
    "        predictions=[tq1_mkt_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[0][\"gold_answer_marketing\"]],\n",
    "    )[\"rougeL\"],\n",
    ")\n",
    "print(\n",
    "    \"BERT\",\n",
    "    bertsc.compute(\n",
    "        predictions=[tq1_mkt_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[0][\"gold_answer_marketing\"]],\n",
    "        lang=\"en\",\n",
    "    )[\"f1\"][0],\n",
    ")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4JfzrRKJ99Yc",
    "outputId": "320c96dc-47c2-4869-997c-e43c6aff72cb"
   },
   "execution_count": 77,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "D: Marketing Metrics\n",
      "BLEU 0.08966592262979808\n",
      "ROUGE 0.2926829268292683\n",
      "BERT 0.8874030709266663\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(\"D: Engineering Metrics\")\n",
    "print(\n",
    "    \"BLEU\",\n",
    "    bleu.compute(\n",
    "        predictions=[tq1_eng_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[0][\"gold_answer_research\"]],\n",
    "    )[\"bleu\"],\n",
    ")\n",
    "print(\n",
    "    \"ROUGE\",\n",
    "    rouge.compute(\n",
    "        predictions=[tq1_eng_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[0][\"gold_answer_research\"]],\n",
    "    )[\"rougeL\"],\n",
    ")\n",
    "print(\n",
    "    \"BERT\",\n",
    "    bertsc.compute(\n",
    "        predictions=[tq1_eng_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[0][\"gold_answer_research\"]],\n",
    "        lang=\"en\",\n",
    "    )[\"f1\"][0],\n",
    ")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bbpeKRfh-6qp",
    "outputId": "ad1ace9f-eef4-4680-be13-1ac1aa12c8ab"
   },
   "execution_count": 78,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "D: Engineering Metrics\n",
      "BLEU 0.0736091550477613\n",
      "ROUGE 0.2484472049689441\n",
      "BERT 0.8888468742370605\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D6wrr5VUUJk1"
   },
   "source": [
    "* BERT Scored high on both engineering and marketing at `88`\n",
    "* ROUGE is reasonable for both"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PZdkNySoUDy3"
   },
   "source": [
    "####5.2.2 Test Question 2\n",
    "\n",
    "Please run the query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "f14ce23a-2a88-4ebf-9c8a-a261fb576156",
    "id": "uDiohH6G_aR1"
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "What methods are typically employed to create training data for embedding models that use task-specific instructions?\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
     ]
    }
   ],
   "source": [
    "tq1: str = validation_df.loc[50][\"question\"]\n",
    "print(tq1)\n",
    "tq1_eng_response: str = rag_2_mistral_eng_chain.invoke(tq1)\n",
    "tq1_mkt_response: str = rag_2_mistral_mkt_chain.invoke(tq1)"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(\"A: Engineering Response\", tq1_eng_response.split(\"[/INST]\")[-1])\n",
    "print(\"A: Marketing Response\", tq1_mkt_response.split(\"[/INST]\")[-1])"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "3d9f3b5b-6408-4e92-eb42-ef2f7e0ef262",
    "id": "ipHVApF-_aSG"
   },
   "execution_count": 96,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "A: Engineering Response  To create training data for embedding models using task-specific instructions, several methods are typically employed. One approach is to combine multiple datasets with task instructions, as demonstrated by Wang et al. (2022) through their construction of the Multi-Task Embeddings Data with Instructions (MEDI) dataset. Another method involves fine-tuning the model on specific tasks or domains, which has been shown to address challenges such as training a single model on diverse datasets (Sanh et al., 2022; Zhou et al.). Additionally, providing explicit context in prompts can be necessary when completing tasks that require up-to-date knowledge or access to internal/private knowledge bases.\n",
      "A: Marketing Response  To create training data for embedding models using task-specific instructions, methods such as fine-tuning and explicit prompting are often employed. Fine-tuning involves adjusting the model's parameters based on new task-specific data, while explicit prompting provides additional context to the model during inference.\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(\"D: Marketing Metrics\")\n",
    "print(\n",
    "    \"BLEU\",\n",
    "    bleu.compute(\n",
    "        predictions=[tq1_mkt_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[50][\"gold_answer_marketing\"]],\n",
    "    )[\"bleu\"],\n",
    ")\n",
    "print(\n",
    "    \"ROUGE\",\n",
    "    rouge.compute(\n",
    "        predictions=[tq1_mkt_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[50][\"gold_answer_marketing\"]],\n",
    "    )[\"rougeL\"],\n",
    ")\n",
    "print(\n",
    "    \"BERT\",\n",
    "    bertsc.compute(\n",
    "        predictions=[tq1_mkt_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[50][\"gold_answer_marketing\"]],\n",
    "        lang=\"en\",\n",
    "    )[\"f1\"][0],\n",
    ")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "3d03c5e5-b620-4835-b5a1-cbf913922563",
    "id": "EUzaDgng_aSH"
   },
   "execution_count": 97,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "D: Marketing Metrics\n",
      "BLEU 0.06043056408431284\n",
      "ROUGE 0.20202020202020202\n",
      "BERT 0.8703487515449524\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(\"D: Engineering Metrics\")\n",
    "print(\n",
    "    \"BLEU\",\n",
    "    bleu.compute(\n",
    "        predictions=[tq1_eng_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[50][\"gold_answer_research\"]],\n",
    "    )[\"bleu\"],\n",
    ")\n",
    "print(\n",
    "    \"ROUGE\",\n",
    "    rouge.compute(\n",
    "        predictions=[tq1_eng_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[50][\"gold_answer_research\"]],\n",
    "    )[\"rougeL\"],\n",
    ")\n",
    "print(\n",
    "    \"BERT\",\n",
    "    bertsc.compute(\n",
    "        predictions=[tq1_eng_response.split(\"[/INST]\")[-1].strip()],\n",
    "        references=[validation_df.loc[50][\"gold_answer_research\"]],\n",
    "        lang=\"en\",\n",
    "    )[\"f1\"][0],\n",
    ")"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "e1315737-ada1-42e6-db35-9ce718c69270",
    "id": "okfPO_gU_aSH"
   },
   "execution_count": 98,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "D: Engineering Metrics\n",
      "BLEU 0.10532164022986051\n",
      "ROUGE 0.25120772946859904\n",
      "BERT 0.8725976347923279\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VxDhzMXsT48H"
   },
   "source": [
    "####5.2.3 Test Question 3\n",
    "\n",
    "Please run the query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9WXJX_x3T4we"
   },
   "outputs": [],
   "source": [
    "tq1: str = \"How does a model's ability to answer questions relate to its exposure to specific types of questions during training?\"\n",
    "print(tq1)\n",
    "tq1_eng_response: str = rag_2_mistral_eng_chain.invoke(tq1)\n",
    "tq1_mkt_response: str = rag_2_mistral_mkt_chain.invoke(tq1)"
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(tq1_eng_response.split(\"[/INST]\")[-1])"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jScRq0y5Aevk",
    "outputId": "fdcf35ce-9f43-401a-ea50-96d1907a8958"
   },
   "execution_count": 91,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      " To create training data for embedding models using task-specific instructions, several methods are typically employed. One approach is to combine multiple datasets with task instructions, as demonstrated by Wang et al. (2022) through their construction of the Multi-Task Embeddings Data with Instructions (MEDI) dataset. Another method involves fine-tuning the model on specific tasks or domains, which has been shown to address challenges such as training a single model on diverse datasets (Sanh et al., 2022; Zhou et al.). Additionally, providing explicit context in prompts may be necessary when completing tasks that require up-to-date knowledge or access to internal/private knowledge bases.\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "print(tq1_mkt_response.split(\"[/INST]\")[-1])"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_CN5_IlQAjBz",
    "outputId": "ce31ace8-88e1-4cc4-c59d-1a8374998ad7"
   },
   "execution_count": 92,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      " To create training data for embedding models using task-specific instructions, methods such as combining multiple datasets from Super-NaturalInstructions (super-NI) and explicit provision of context through prompts are often used.\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "44h038A7AUdn"
   },
   "source": [
    "###5.3 Other Questions\n",
    "\n",
    "Below are a few questions that you should think about. Please answer them in the answer file directly (in a short paragraph) and also see whether they may be relevant for your final write-up.\n",
    "\n",
    "**QUESTION:**\n",
    "\n",
    "5.3.a. How would you expect your response quality to change if you had a chunk size of `50`?\n",
    "\n",
    "> Having tried a chunk size of `64`, the results were not as favourable as a larger chunk size between `128-256`. This could likely be because the particular subject matter in question (LLMs and Gen AI) may be more technical, and cannot be expressed in such a small chunk of characters, as you're almost always going to not capture the complete relevant information\n",
    "\n",
    "5.3.b. How would you expect your response quality to change if you had a chunk size of 5000?\n",
    "\n",
    "> I think a chunk size of `5000` would limit the number of documents the model can look at, since each split would contain 5000 characters in this case, rendering the breadth of search as being signficantly reduced. Larger chunks might improve detailed understanding of fewer documents but reduce the number of documents the system can consider simultaneously. Another issue with `5000` is that it might get close to the model's parameterized token limit\n",
    "\n",
    "5.3.c. If you had time, how do you think fine-tuning of the LLM could help?  What type of data would you want for that? And which training approach would you take?\n",
    "\n",
    "> Fine-tuning could enhance the LLM's ability to generate more relevant and context-specific answers, particularly for technical content or industry-specific jargon. We would use continued pretraining on the context from the gold validation set. This could be achieved through a training loop in `pytorch`, but continous retraining based on document updates may become tedious to maintain technically long term, as opposed to using a RAG.\n",
    "\n",
    "5.3.d. What was your design philosophy  of the prompts? How did they differ between engineering and marketing support?\n",
    "\n",
    "> The prompts were designed to elicit tailored responses suitable for the specific needs of the engineering and marketing departments. Engineering prompts were more technical, requiring detailed, longer responses, while marketing prompts were designed to be concise and to the point, focusing on clarity and accessibility. The word limit for each was deduced from a statistical distribution of the gold validation set answers.\n",
    "\n",
    "5.3.e. What are your average and peak load estimates for the system? Given that, would you suggest a pay-per-use deployment or one that reserves the LLM?\n",
    "\n",
    "> When starting out, and for internal use - a pay per use model may work better. This is because until adoption reaches a critical mass within the company, it's preferable to have it as pay-per-use / serverless. When the product goes customer-facing, it might be more prudent to reserve a managed LLM since it offer more predictable cost at scale.\n",
    "\n",
    "5.3.f. What type of limitations/risks would you see in using this system?\n",
    "\n",
    "> * `Quality and Freshness of Data`: The performance heavily depends on the quality and timeliness of the data fed into the system. Outdated or low-quality data can lead to irrelevant or incorrect answers.\n",
    "\n",
    "> * `Model Bias and Safety`: There is a risk of generating biased or unsafe responses, especially if the underlying LLM has not been adequately trained to handle sensitive topics safely.\n",
    "\n",
    "> * `System Complexity and Maintenance`: The sophisticated infrastructure might require significant maintenance and technical expertise, which could be a challenge for the operational team\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "6yhZpA9y1HyN"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "0149fe82ffc648c8b5ef154ff4fe05fe": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_73fc3026a832486aa65f0131cd38d999",
       "IPY_MODEL_a3a2ace2e43a42e59360eda9bd15feb0",
       "IPY_MODEL_5f42a8d50e6642b8bfb194c3a67754fb"
      ],
      "layout": "IPY_MODEL_d5ca95777bb44f36985b522b6da13fd9"
     }
    },
    "73fc3026a832486aa65f0131cd38d999": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_81f29be6cf00405f9c2709cbf1322382",
      "placeholder": "​",
      "style": "IPY_MODEL_68f229a722304e0f8f663f708dcbb7b2",
      "value": "100%"
     }
    },
    "a3a2ace2e43a42e59360eda9bd15feb0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c3ea93469c5945f6886e1d3c1769f8d5",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8156d53326314d4aa600de4eb50ddac6",
      "value": 75
     }
    },
    "5f42a8d50e6642b8bfb194c3a67754fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_add29108855947feac91ae2941e9af84",
      "placeholder": "​",
      "style": "IPY_MODEL_bfcdbca9e34b4a9e855ec36ba5409507",
      "value": " 75/75 [11:05&lt;00:00,  7.05s/it]"
     }
    },
    "d5ca95777bb44f36985b522b6da13fd9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "81f29be6cf00405f9c2709cbf1322382": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "68f229a722304e0f8f663f708dcbb7b2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c3ea93469c5945f6886e1d3c1769f8d5": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8156d53326314d4aa600de4eb50ddac6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "add29108855947feac91ae2941e9af84": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bfcdbca9e34b4a9e855ec36ba5409507": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a976ce18edd44f09ace115f6ca4175be": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5da128b45b94482ba2b8cce91b93bd78",
       "IPY_MODEL_eda0583a1eaf48fd88ae1b04b9d68add",
       "IPY_MODEL_9cafa3eae2ef4e5babcda1f8b173558b"
      ],
      "layout": "IPY_MODEL_c4c057a9b3cd493fa660250137793365"
     }
    },
    "5da128b45b94482ba2b8cce91b93bd78": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4e369d779cdd47e297c375a7a9ead2bd",
      "placeholder": "​",
      "style": "IPY_MODEL_c6695f5e87024b518a8dc045cba9aa9c",
      "value": "100%"
     }
    },
    "eda0583a1eaf48fd88ae1b04b9d68add": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e52a8688b5e34f7986f7f0a30c317411",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a521da5ce4544c0b8245bcaa96008f4e",
      "value": 75
     }
    },
    "9cafa3eae2ef4e5babcda1f8b173558b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_16afbe5850c547b6b64224f8f4b88242",
      "placeholder": "​",
      "style": "IPY_MODEL_fd6de45284c84df795565c9acce7711b",
      "value": " 75/75 [00:19&lt;00:00,  4.61it/s]"
     }
    },
    "c4c057a9b3cd493fa660250137793365": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4e369d779cdd47e297c375a7a9ead2bd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c6695f5e87024b518a8dc045cba9aa9c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e52a8688b5e34f7986f7f0a30c317411": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a521da5ce4544c0b8245bcaa96008f4e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "16afbe5850c547b6b64224f8f4b88242": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fd6de45284c84df795565c9acce7711b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "48a4e3941bbe4718a06f70671974f67a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a8fc24041be649839040f067aa713ff9",
       "IPY_MODEL_95e2b89be2c6402b87841efee285fbfd",
       "IPY_MODEL_6e32da8ab47445ee8f25fe9a5a8d63e8"
      ],
      "layout": "IPY_MODEL_081c2e2678e84628b16ea9b59aa2230c"
     }
    },
    "a8fc24041be649839040f067aa713ff9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_32024293d24045b98c165d3d48c6d59a",
      "placeholder": "​",
      "style": "IPY_MODEL_b7baaacce5c14735bdcceff0eeeacd55",
      "value": "100%"
     }
    },
    "95e2b89be2c6402b87841efee285fbfd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1ce2daa108be4aefae08d1b6330037d7",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7e94c8603c194169a14823ef5a9fee0d",
      "value": 75
     }
    },
    "6e32da8ab47445ee8f25fe9a5a8d63e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0430d90708af45f68d31d3f5c9845fea",
      "placeholder": "​",
      "style": "IPY_MODEL_60f11304ded1466ab1f7b2f22e238994",
      "value": " 75/75 [00:04&lt;00:00, 19.69it/s]"
     }
    },
    "081c2e2678e84628b16ea9b59aa2230c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "32024293d24045b98c165d3d48c6d59a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b7baaacce5c14735bdcceff0eeeacd55": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1ce2daa108be4aefae08d1b6330037d7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7e94c8603c194169a14823ef5a9fee0d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0430d90708af45f68d31d3f5c9845fea": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "60f11304ded1466ab1f7b2f22e238994": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e141faf3239543339ab325ca2af306af": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f271bfc517c04c36b24d1b3a0cb2dde1",
       "IPY_MODEL_9b94c11b0438478286cf38964e48cf92",
       "IPY_MODEL_41d88777ab6a4a5ba53d0a2bc4f155bc"
      ],
      "layout": "IPY_MODEL_dd6d572d3c91436b8a20880c95d5eacf"
     }
    },
    "f271bfc517c04c36b24d1b3a0cb2dde1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cb5ab4e93e6a4ded8500436ccd5ff74c",
      "placeholder": "​",
      "style": "IPY_MODEL_1737688ec5074ab5bcaad2b2315ebe7b",
      "value": "100%"
     }
    },
    "9b94c11b0438478286cf38964e48cf92": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d6fb97e0f75949588b7b561804b46332",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9cfabb8292254e1c8b14585c377f3879",
      "value": 75
     }
    },
    "41d88777ab6a4a5ba53d0a2bc4f155bc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0d5e5bafd5004d35a44688d90c218a5f",
      "placeholder": "​",
      "style": "IPY_MODEL_fdaddc9124ed49dcbea3d5755e7d223e",
      "value": " 75/75 [00:25&lt;00:00,  4.25it/s]"
     }
    },
    "dd6d572d3c91436b8a20880c95d5eacf": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cb5ab4e93e6a4ded8500436ccd5ff74c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1737688ec5074ab5bcaad2b2315ebe7b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d6fb97e0f75949588b7b561804b46332": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9cfabb8292254e1c8b14585c377f3879": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0d5e5bafd5004d35a44688d90c218a5f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fdaddc9124ed49dcbea3d5755e7d223e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ca616bbb149f441d92256a17ce617b3b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2401c5056fd247999edcd1b6176390a7",
       "IPY_MODEL_d1a36dba11be48d98ce88e4f2055dae1",
       "IPY_MODEL_a05853a983ed4bfb98659908e3f86b10"
      ],
      "layout": "IPY_MODEL_beec8bad40724459866bcaf255d283c4"
     }
    },
    "2401c5056fd247999edcd1b6176390a7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_334a14ca95d14b949f7608ec5bb9240c",
      "placeholder": "​",
      "style": "IPY_MODEL_a5dfafa758574a37863fc007ce51a101",
      "value": "100%"
     }
    },
    "d1a36dba11be48d98ce88e4f2055dae1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d1e1f9633bab406f9f8530a0fb6367f1",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1f414cdc46504d50adb70360e422606c",
      "value": 75
     }
    },
    "a05853a983ed4bfb98659908e3f86b10": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cbcfc49a593540dabf75d11a3f23329b",
      "placeholder": "​",
      "style": "IPY_MODEL_6bfae7df0b4049d9926714a2b979a8be",
      "value": " 75/75 [00:03&lt;00:00, 22.76it/s]"
     }
    },
    "beec8bad40724459866bcaf255d283c4": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "334a14ca95d14b949f7608ec5bb9240c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a5dfafa758574a37863fc007ce51a101": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d1e1f9633bab406f9f8530a0fb6367f1": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1f414cdc46504d50adb70360e422606c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "cbcfc49a593540dabf75d11a3f23329b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6bfae7df0b4049d9926714a2b979a8be": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "69f6c39f632444b69a0dd58056693399": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_150bc4489b554dc99330ba959dc6e41c",
       "IPY_MODEL_64b573aa115b40ca8e74036b8afb38c5",
       "IPY_MODEL_4af9756239b74b9382e2f34c11c48aec"
      ],
      "layout": "IPY_MODEL_2dace23764b644608443bb1e59ddba9d"
     }
    },
    "150bc4489b554dc99330ba959dc6e41c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_30780a4529474c8bb685919be6ea3953",
      "placeholder": "​",
      "style": "IPY_MODEL_a4c95e2fd15e40e7b14538760f6b7969",
      "value": "100%"
     }
    },
    "64b573aa115b40ca8e74036b8afb38c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f92e3e68b2ec4b2ab4568d80c189d844",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7e6a713e919b475aac4a952d741ac82a",
      "value": 75
     }
    },
    "4af9756239b74b9382e2f34c11c48aec": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9f63cf9941fd4495aea7862cfdbf2e68",
      "placeholder": "​",
      "style": "IPY_MODEL_fae247cd8826438697e6d6dba227f8b3",
      "value": " 75/75 [00:21&lt;00:00,  2.18it/s]"
     }
    },
    "2dace23764b644608443bb1e59ddba9d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "30780a4529474c8bb685919be6ea3953": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a4c95e2fd15e40e7b14538760f6b7969": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f92e3e68b2ec4b2ab4568d80c189d844": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7e6a713e919b475aac4a952d741ac82a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9f63cf9941fd4495aea7862cfdbf2e68": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fae247cd8826438697e6d6dba227f8b3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3b15da9197744637b728d07dd9596148": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_be5242e90a84427b9d62694b55a272ac",
       "IPY_MODEL_b9318fd216dc4689aa2cd24db76174b6",
       "IPY_MODEL_aedcb55caadd4a7dafbf9ed4bdd0cec0"
      ],
      "layout": "IPY_MODEL_eb0541cd87454eb1a45f4718e1efcd81"
     }
    },
    "be5242e90a84427b9d62694b55a272ac": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3c404b683cee41029950d94f13f32c36",
      "placeholder": "​",
      "style": "IPY_MODEL_f31cc652b1c54c6eb1207e8ecbba40e1",
      "value": "100%"
     }
    },
    "b9318fd216dc4689aa2cd24db76174b6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_263b2006a5314ae088127e1624c3a42c",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b682304c931f458eb2f1c2cf687ca042",
      "value": 75
     }
    },
    "aedcb55caadd4a7dafbf9ed4bdd0cec0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_63b7d1158f7a46d88fafc231391fbd11",
      "placeholder": "​",
      "style": "IPY_MODEL_0aca9c01b50241e6ad2cc35b58f77bad",
      "value": " 75/75 [00:08&lt;00:00, 11.54it/s]"
     }
    },
    "eb0541cd87454eb1a45f4718e1efcd81": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3c404b683cee41029950d94f13f32c36": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f31cc652b1c54c6eb1207e8ecbba40e1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "263b2006a5314ae088127e1624c3a42c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b682304c931f458eb2f1c2cf687ca042": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "63b7d1158f7a46d88fafc231391fbd11": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0aca9c01b50241e6ad2cc35b58f77bad": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "01a7ae757b9644bb8f51df6f3e040805": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_131764a259134322bda8e402faac2574",
       "IPY_MODEL_b66a2f3479ce4c02ba2f71953f8f1d21",
       "IPY_MODEL_3587668bd9f2495d9550226870e6d459"
      ],
      "layout": "IPY_MODEL_40a064a67338429dbc24c7e3d18a9a53"
     }
    },
    "131764a259134322bda8e402faac2574": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bf706e39dde5491d82f131b0af469290",
      "placeholder": "​",
      "style": "IPY_MODEL_db65392a4fb64586af0df7ea491bd94c",
      "value": "100%"
     }
    },
    "b66a2f3479ce4c02ba2f71953f8f1d21": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_36868620156c4089b252cfefec8d49dc",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8c3f6d77cc594ef8916300fb585c9c16",
      "value": 75
     }
    },
    "3587668bd9f2495d9550226870e6d459": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_af5cfbce278947d7905aeacdbdeba7bd",
      "placeholder": "​",
      "style": "IPY_MODEL_ef91a28b9be345eab3db6d3485c1bd11",
      "value": " 75/75 [00:17&lt;00:00,  3.88it/s]"
     }
    },
    "40a064a67338429dbc24c7e3d18a9a53": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bf706e39dde5491d82f131b0af469290": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db65392a4fb64586af0df7ea491bd94c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "36868620156c4089b252cfefec8d49dc": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8c3f6d77cc594ef8916300fb585c9c16": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "af5cfbce278947d7905aeacdbdeba7bd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ef91a28b9be345eab3db6d3485c1bd11": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b213a07145554c12b207dbd72c0d76c8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b60300798fbc45df806be401450d68a2",
       "IPY_MODEL_6c499589c869464cb975212ed6887358",
       "IPY_MODEL_35a1e99a57c642e2b85c35135e2ebfdf"
      ],
      "layout": "IPY_MODEL_0734b37b22174ec7b34b70deaf73f6e3"
     }
    },
    "b60300798fbc45df806be401450d68a2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_918cc795c2cb4bd6b77dc15d41fbbe45",
      "placeholder": "​",
      "style": "IPY_MODEL_de8526671a8646169200a16d11d71831",
      "value": "100%"
     }
    },
    "6c499589c869464cb975212ed6887358": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_751bad9743f547e791575bdad7d07932",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f82ec77de29b4576b8ac1bb94b092631",
      "value": 75
     }
    },
    "35a1e99a57c642e2b85c35135e2ebfdf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5e415b151b894bc18a44a6f858c5439d",
      "placeholder": "​",
      "style": "IPY_MODEL_3cdf780414ca481186f6fc195df7032f",
      "value": " 75/75 [00:08&lt;00:00, 10.83it/s]"
     }
    },
    "0734b37b22174ec7b34b70deaf73f6e3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "918cc795c2cb4bd6b77dc15d41fbbe45": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "de8526671a8646169200a16d11d71831": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "751bad9743f547e791575bdad7d07932": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f82ec77de29b4576b8ac1bb94b092631": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5e415b151b894bc18a44a6f858c5439d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3cdf780414ca481186f6fc195df7032f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "61bde2cf87764e98b7b6e4d2bef74635": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4b26ea2f2a8947d6989d687952ef90f9",
       "IPY_MODEL_c313f2d0b0e64b14b6ce52a89f3481fc",
       "IPY_MODEL_01814c150eab4edb941b02411e6fd904"
      ],
      "layout": "IPY_MODEL_aa8b54944432416486cbc27babffc3fb"
     }
    },
    "4b26ea2f2a8947d6989d687952ef90f9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c73633ddc1ab40c0a89cc36f4099f550",
      "placeholder": "​",
      "style": "IPY_MODEL_490036306fb14b5da2f0b74822d35e0a",
      "value": "config.json: 100%"
     }
    },
    "c313f2d0b0e64b14b6ce52a89f3481fc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cdf608c489e74aeab10bc384cee0ce43",
      "max": 571,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_dcfe120f20aa42a0853db39f9dccaf99",
      "value": 571
     }
    },
    "01814c150eab4edb941b02411e6fd904": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_11843f8af3174008ac15d7823330b1b6",
      "placeholder": "​",
      "style": "IPY_MODEL_3905358e2ee6402fb14f59e45d62535f",
      "value": " 571/571 [00:00&lt;00:00, 8.13kB/s]"
     }
    },
    "aa8b54944432416486cbc27babffc3fb": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c73633ddc1ab40c0a89cc36f4099f550": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "490036306fb14b5da2f0b74822d35e0a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cdf608c489e74aeab10bc384cee0ce43": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dcfe120f20aa42a0853db39f9dccaf99": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "11843f8af3174008ac15d7823330b1b6": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3905358e2ee6402fb14f59e45d62535f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "889644558f4545baafebe515da640fdc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8594b70918014ce18fe56b7d5869311b",
       "IPY_MODEL_e731cdfea27f4407bfb1964a26af7b2a",
       "IPY_MODEL_290253241b04490fb6a801336aca0053"
      ],
      "layout": "IPY_MODEL_4eaae0855a174b75807af821a530a50c"
     }
    },
    "8594b70918014ce18fe56b7d5869311b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9f21918403fe443bb07fe015e034a2eb",
      "placeholder": "​",
      "style": "IPY_MODEL_1491f89bcec1471c875c1b779ed2de33",
      "value": "model.safetensors.index.json: 100%"
     }
    },
    "e731cdfea27f4407bfb1964a26af7b2a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f2a31a3cc3d24b4aa7a14bc30b62b4bc",
      "max": 25125,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_597d99f7772f4e9a89d8130aeca45912",
      "value": 25125
     }
    },
    "290253241b04490fb6a801336aca0053": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ebb1be8d68284638abbeb8ba19e5e8bd",
      "placeholder": "​",
      "style": "IPY_MODEL_d24b30669c9a4b1bba753226f179d212",
      "value": " 25.1k/25.1k [00:00&lt;00:00, 364kB/s]"
     }
    },
    "4eaae0855a174b75807af821a530a50c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9f21918403fe443bb07fe015e034a2eb": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1491f89bcec1471c875c1b779ed2de33": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f2a31a3cc3d24b4aa7a14bc30b62b4bc": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "597d99f7772f4e9a89d8130aeca45912": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "ebb1be8d68284638abbeb8ba19e5e8bd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d24b30669c9a4b1bba753226f179d212": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fb36ca2e6f66461dbc1054ec1fb9ab0f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fc24cd8378944ba3881080fe9d225493",
       "IPY_MODEL_625c961f94f94be998d063272f90ffd5",
       "IPY_MODEL_337136d190a4499aa12c72de4dce0a69"
      ],
      "layout": "IPY_MODEL_e118f57460b24beea9279a3de53c9e8f"
     }
    },
    "fc24cd8378944ba3881080fe9d225493": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_77b1efde2db943bca74242090a3b66dc",
      "placeholder": "​",
      "style": "IPY_MODEL_3f70d6e135e044ceb51be6914ff42385",
      "value": "Downloading shards: 100%"
     }
    },
    "625c961f94f94be998d063272f90ffd5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bcf6012285b34feaa1b9144b591370b3",
      "max": 2,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0770d6701f95429e81db462502a001dc",
      "value": 2
     }
    },
    "337136d190a4499aa12c72de4dce0a69": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_706922f202ca4dffa4a4d6ffc123fc05",
      "placeholder": "​",
      "style": "IPY_MODEL_56beafbff551493d82570b507d6f8fbc",
      "value": " 2/2 [02:19&lt;00:00, 65.38s/it]"
     }
    },
    "e118f57460b24beea9279a3de53c9e8f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "77b1efde2db943bca74242090a3b66dc": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3f70d6e135e044ceb51be6914ff42385": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bcf6012285b34feaa1b9144b591370b3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0770d6701f95429e81db462502a001dc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "706922f202ca4dffa4a4d6ffc123fc05": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "56beafbff551493d82570b507d6f8fbc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "507fd096fb0e43b895ba4d5ad3566c33": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_43bb1c701401416e93ae65e069ef9143",
       "IPY_MODEL_a8fc6d6ebb3545ff8d472565d2b8d788",
       "IPY_MODEL_fe783695f6e441d3bd2788176be8b214"
      ],
      "layout": "IPY_MODEL_761d86bcd9d940f6b402bda5af974b1b"
     }
    },
    "43bb1c701401416e93ae65e069ef9143": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_db47ed472db44dd2a3688810c5941a9c",
      "placeholder": "​",
      "style": "IPY_MODEL_4f4e8575a81f469886f95d465fe56996",
      "value": "model-00001-of-00002.safetensors: 100%"
     }
    },
    "a8fc6d6ebb3545ff8d472565d2b8d788": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d3e5282742f74e30b0e83fcc4d24e8c0",
      "max": 9942981696,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ddce6b8c73d846e7ad8fa7d41e67bd9b",
      "value": 9942981696
     }
    },
    "fe783695f6e441d3bd2788176be8b214": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_24dddd7b7f954619b59d5c72107a315b",
      "placeholder": "​",
      "style": "IPY_MODEL_62b545e1a0714810a379ad3d9ada8530",
      "value": " 9.94G/9.94G [01:34&lt;00:00, 176MB/s]"
     }
    },
    "761d86bcd9d940f6b402bda5af974b1b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db47ed472db44dd2a3688810c5941a9c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4f4e8575a81f469886f95d465fe56996": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d3e5282742f74e30b0e83fcc4d24e8c0": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ddce6b8c73d846e7ad8fa7d41e67bd9b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "24dddd7b7f954619b59d5c72107a315b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "62b545e1a0714810a379ad3d9ada8530": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "476cae2e8fd040cbb69141d88dfe96d1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ed8e4e559ac6460d80adf9df95e115b0",
       "IPY_MODEL_559585f0be7a4ae2984bb007374fb83c",
       "IPY_MODEL_2d40f47ff6e7409882edfef7e2a46588"
      ],
      "layout": "IPY_MODEL_af37875552af48498f5440962bfc0135"
     }
    },
    "ed8e4e559ac6460d80adf9df95e115b0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_274d9d1d796a458e8059b7eff510b4a2",
      "placeholder": "​",
      "style": "IPY_MODEL_ad5416de1aaa4a329443c093787c1fb1",
      "value": "model-00002-of-00002.safetensors: 100%"
     }
    },
    "559585f0be7a4ae2984bb007374fb83c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_402307636b59421a82248dd433132617",
      "max": 4540516344,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a812d758e10e45ada68bca3f7659fe8d",
      "value": 4540516344
     }
    },
    "2d40f47ff6e7409882edfef7e2a46588": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_83e2cc7ee9aa4892bd6342c32bfabc9c",
      "placeholder": "​",
      "style": "IPY_MODEL_98ac1600cba14c26adda5f3d5e9599af",
      "value": " 4.54G/4.54G [00:44&lt;00:00, 137MB/s]"
     }
    },
    "af37875552af48498f5440962bfc0135": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "274d9d1d796a458e8059b7eff510b4a2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ad5416de1aaa4a329443c093787c1fb1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "402307636b59421a82248dd433132617": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a812d758e10e45ada68bca3f7659fe8d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "83e2cc7ee9aa4892bd6342c32bfabc9c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "98ac1600cba14c26adda5f3d5e9599af": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9b455df848f04c17ac5022dba1cde781": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_0a7d8e5a349441318167f9095268d1af",
       "IPY_MODEL_453f436dc1aa4686bb4b3e3a90db6c03",
       "IPY_MODEL_14e8d395b56c42f4b5628366db68b280"
      ],
      "layout": "IPY_MODEL_6292aa65d3c04dd7b0705a2748a6bcaa"
     }
    },
    "0a7d8e5a349441318167f9095268d1af": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ca14381c0fb547fcb840cfc64d77c18e",
      "placeholder": "​",
      "style": "IPY_MODEL_9f815762f4c34ddeacea219313b59417",
      "value": "Loading checkpoint shards: 100%"
     }
    },
    "453f436dc1aa4686bb4b3e3a90db6c03": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_741e5ba8e416488ab4537d403a1ff97c",
      "max": 2,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_529031d7699743b78566fc3c0da7416a",
      "value": 2
     }
    },
    "14e8d395b56c42f4b5628366db68b280": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_58a9b6c104aa40c1917a36ddccc6d408",
      "placeholder": "​",
      "style": "IPY_MODEL_fa585bdf39cc418996fc6317906f6ad7",
      "value": " 2/2 [01:23&lt;00:00, 38.67s/it]"
     }
    },
    "6292aa65d3c04dd7b0705a2748a6bcaa": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ca14381c0fb547fcb840cfc64d77c18e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9f815762f4c34ddeacea219313b59417": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "741e5ba8e416488ab4537d403a1ff97c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "529031d7699743b78566fc3c0da7416a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "58a9b6c104aa40c1917a36ddccc6d408": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fa585bdf39cc418996fc6317906f6ad7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5cd896948ecd4cf59f20ea2a6485f0fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_059e676192b84a6288d78df790767adb",
       "IPY_MODEL_143639dd1d10425dbac8aa3f619b76bc",
       "IPY_MODEL_3ed76de4a01c46618a3482d0f8c5eeed"
      ],
      "layout": "IPY_MODEL_ef68ffccdc9244f09eeb001a0c529ccc"
     }
    },
    "059e676192b84a6288d78df790767adb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d4ba6c8d120340c990d8b297e28cf3d6",
      "placeholder": "​",
      "style": "IPY_MODEL_752fe6fddd514490bd932724c3eda477",
      "value": "generation_config.json: 100%"
     }
    },
    "143639dd1d10425dbac8aa3f619b76bc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3e9e402c493a4176a360e0d5d1601818",
      "max": 116,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_69c1695056c64314b38b5716da58e684",
      "value": 116
     }
    },
    "3ed76de4a01c46618a3482d0f8c5eeed": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f2cc787744ec46d5bf85630f6f679da8",
      "placeholder": "​",
      "style": "IPY_MODEL_3d95a1cc89e343a6ac29f81ed4cb5945",
      "value": " 116/116 [00:00&lt;00:00, 4.11kB/s]"
     }
    },
    "ef68ffccdc9244f09eeb001a0c529ccc": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d4ba6c8d120340c990d8b297e28cf3d6": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "752fe6fddd514490bd932724c3eda477": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3e9e402c493a4176a360e0d5d1601818": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "69c1695056c64314b38b5716da58e684": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f2cc787744ec46d5bf85630f6f679da8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3d95a1cc89e343a6ac29f81ed4cb5945": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "35f3f7cb3a70420495e55ae6721e55a4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_612ce55e7cdc4d028a402bab0d0d22b8",
       "IPY_MODEL_67330f73c19e493ca8036a471646e093",
       "IPY_MODEL_c702a3fce244413695685ee042805e5e"
      ],
      "layout": "IPY_MODEL_ce192cdf6653461abed37aebe0509a55"
     }
    },
    "612ce55e7cdc4d028a402bab0d0d22b8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5f5617b2a18244f78974fa3771bcdb6d",
      "placeholder": "​",
      "style": "IPY_MODEL_c423ac30b643486a98b4eff882ef4396",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "67330f73c19e493ca8036a471646e093": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5b0cdce971bb4e9aa39a3c73c7e88f01",
      "max": 1467,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_773b4d9c7287424ebbbca4eabc1383a6",
      "value": 1467
     }
    },
    "c702a3fce244413695685ee042805e5e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4173ada84f9d4d6f91e65cdb2f0364a4",
      "placeholder": "​",
      "style": "IPY_MODEL_17451c57711e433daae16d309fe7e997",
      "value": " 1.47k/1.47k [00:00&lt;00:00, 49.8kB/s]"
     }
    },
    "ce192cdf6653461abed37aebe0509a55": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5f5617b2a18244f78974fa3771bcdb6d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c423ac30b643486a98b4eff882ef4396": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5b0cdce971bb4e9aa39a3c73c7e88f01": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "773b4d9c7287424ebbbca4eabc1383a6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "4173ada84f9d4d6f91e65cdb2f0364a4": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "17451c57711e433daae16d309fe7e997": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0da19d911f8c4e409798166ebca5f5ea": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_952fc87c2aba45569ab04c84b4ec36d4",
       "IPY_MODEL_9e750c7129794f1aa9b92094757a9e70",
       "IPY_MODEL_5e8e2b7c2e9945c8862fa9fbe5bcf754"
      ],
      "layout": "IPY_MODEL_21c6627657c74f4a9b1369470583b579"
     }
    },
    "952fc87c2aba45569ab04c84b4ec36d4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ca5cf69da5354d0b85170d0c4e3d7209",
      "placeholder": "​",
      "style": "IPY_MODEL_6a8f4ad9edb6434182c9988391032c68",
      "value": "tokenizer.model: 100%"
     }
    },
    "9e750c7129794f1aa9b92094757a9e70": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d49bead556784f66b195572c0103762d",
      "max": 493443,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8ae0a183e1d940a3857f5c95041e76df",
      "value": 493443
     }
    },
    "5e8e2b7c2e9945c8862fa9fbe5bcf754": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8c9fd541aded4ea4af997015b589fb06",
      "placeholder": "​",
      "style": "IPY_MODEL_4af3a0699a094fae9cca8458a5804e05",
      "value": " 493k/493k [00:00&lt;00:00, 25.3MB/s]"
     }
    },
    "21c6627657c74f4a9b1369470583b579": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ca5cf69da5354d0b85170d0c4e3d7209": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a8f4ad9edb6434182c9988391032c68": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d49bead556784f66b195572c0103762d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8ae0a183e1d940a3857f5c95041e76df": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "8c9fd541aded4ea4af997015b589fb06": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4af3a0699a094fae9cca8458a5804e05": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9020bbbca52541d4aae58df85fc6278d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3c051e183766421782510d3a5872952d",
       "IPY_MODEL_3aa456496426433ca499c47bc7f16353",
       "IPY_MODEL_b9091cb89166441a8d3cda2bbf6a1e70"
      ],
      "layout": "IPY_MODEL_f0d67326b4c14c5ba37d42b9e4d8a7da"
     }
    },
    "3c051e183766421782510d3a5872952d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_147ae93de43941b3be92053b96e83526",
      "placeholder": "​",
      "style": "IPY_MODEL_424c1786e02947ce82b74e15588cbbab",
      "value": "tokenizer.json: 100%"
     }
    },
    "3aa456496426433ca499c47bc7f16353": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_db03405d42de4065ad3fbbca8e3bfee2",
      "max": 1795303,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f4e129c465714709be9b1339d1b80ad2",
      "value": 1795303
     }
    },
    "b9091cb89166441a8d3cda2bbf6a1e70": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2bdbcc5258ee4684917dabb337a24905",
      "placeholder": "​",
      "style": "IPY_MODEL_404d6f4424ed45e38a0ab31d1db142e1",
      "value": " 1.80M/1.80M [00:00&lt;00:00, 4.24MB/s]"
     }
    },
    "f0d67326b4c14c5ba37d42b9e4d8a7da": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "147ae93de43941b3be92053b96e83526": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "424c1786e02947ce82b74e15588cbbab": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "db03405d42de4065ad3fbbca8e3bfee2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f4e129c465714709be9b1339d1b80ad2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2bdbcc5258ee4684917dabb337a24905": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "404d6f4424ed45e38a0ab31d1db142e1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e897f8bd26fa421585a0d2efe7515c09": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c69c9577417c4f58bd4c85d6a6a1b739",
       "IPY_MODEL_e28994ac1b464c4cace1416713c02cd6",
       "IPY_MODEL_71c51aca8e944a6b824ef8b6ead4628a"
      ],
      "layout": "IPY_MODEL_88123e66271d401dada14b92c3837ff4"
     }
    },
    "c69c9577417c4f58bd4c85d6a6a1b739": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0d53925aa50446ae85d09b6bdda148f1",
      "placeholder": "​",
      "style": "IPY_MODEL_24bf9fa741f047319547ccabef6472b1",
      "value": "special_tokens_map.json: 100%"
     }
    },
    "e28994ac1b464c4cace1416713c02cd6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_49027c6b8db34c0da48402dd36c26b4f",
      "max": 72,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_3cac471e2eb54ca8b9bfd7246f4f8236",
      "value": 72
     }
    },
    "71c51aca8e944a6b824ef8b6ead4628a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_06c3498fc6724dc284d6e15c4cf862f3",
      "placeholder": "​",
      "style": "IPY_MODEL_4604a5e0932c466e9af36debbece4960",
      "value": " 72.0/72.0 [00:00&lt;00:00, 3.11kB/s]"
     }
    },
    "88123e66271d401dada14b92c3837ff4": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0d53925aa50446ae85d09b6bdda148f1": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "24bf9fa741f047319547ccabef6472b1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "49027c6b8db34c0da48402dd36c26b4f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3cac471e2eb54ca8b9bfd7246f4f8236": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "06c3498fc6724dc284d6e15c4cf862f3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4604a5e0932c466e9af36debbece4960": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "68485f50f1fc4e528fa08beff8d03900": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a510a0d6b67849449452b2eecc165651",
       "IPY_MODEL_f47f8eedfda34b6e97e33cc040b61888",
       "IPY_MODEL_e0f20a0ec2be4736bfcd45b01eb458e9"
      ],
      "layout": "IPY_MODEL_dc19f17e15ba4b2bafcbcc2e32105104"
     }
    },
    "a510a0d6b67849449452b2eecc165651": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4ccc8312f5ec4b69a9b8218d9d65a3d1",
      "placeholder": "​",
      "style": "IPY_MODEL_5276348b709b4d72871b54d1c00468a1",
      "value": "100%"
     }
    },
    "f47f8eedfda34b6e97e33cc040b61888": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dd432188eb2a433ea7d43607b4f3aff0",
      "max": 20,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f59318cd060042daaca4b4fa26bf9925",
      "value": 20
     }
    },
    "e0f20a0ec2be4736bfcd45b01eb458e9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9e770af0c3f34486948b716c14727fc9",
      "placeholder": "​",
      "style": "IPY_MODEL_ddeee64de59c4a8faae835b0fc0f598b",
      "value": " 20/20 [00:10&lt;00:00,  2.93it/s]"
     }
    },
    "dc19f17e15ba4b2bafcbcc2e32105104": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4ccc8312f5ec4b69a9b8218d9d65a3d1": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5276348b709b4d72871b54d1c00468a1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dd432188eb2a433ea7d43607b4f3aff0": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f59318cd060042daaca4b4fa26bf9925": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9e770af0c3f34486948b716c14727fc9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ddeee64de59c4a8faae835b0fc0f598b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "417ac7d2c28f4d3b82d01349e5e6af34": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8eaf9375b40140be9e20a7fde5f99985",
       "IPY_MODEL_677a929849474e188db2ebecab975938",
       "IPY_MODEL_c84789a976a444bdb002aa6347721edd"
      ],
      "layout": "IPY_MODEL_86da482d7870400c8e8938214889043f"
     }
    },
    "8eaf9375b40140be9e20a7fde5f99985": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e454dfe23a4a426da3e03f19b957fde4",
      "placeholder": "​",
      "style": "IPY_MODEL_e3a56a80e2304a158e7e66388d3e4317",
      "value": "100%"
     }
    },
    "677a929849474e188db2ebecab975938": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0c753dd7107440bc8403c8fbe329cf7b",
      "max": 3,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c9af9c4c2f534f72ad066d28b1fb33e6",
      "value": 3
     }
    },
    "c84789a976a444bdb002aa6347721edd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d44b1932c69e4589bf6ae3067560d332",
      "placeholder": "​",
      "style": "IPY_MODEL_40c660d4ac0247928c153df91ae4eb78",
      "value": " 3/3 [00:13&lt;00:00,  4.47s/it]"
     }
    },
    "86da482d7870400c8e8938214889043f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e454dfe23a4a426da3e03f19b957fde4": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e3a56a80e2304a158e7e66388d3e4317": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0c753dd7107440bc8403c8fbe329cf7b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c9af9c4c2f534f72ad066d28b1fb33e6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d44b1932c69e4589bf6ae3067560d332": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "40c660d4ac0247928c153df91ae4eb78": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cf0eb0a3f0e449eebd522fdfc77431ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_051ae4f420cd47628c19f0ef96803995",
       "IPY_MODEL_e4d7a6143fca4847a827565566c305a2",
       "IPY_MODEL_554c85e291984153a4fda3be0062eb08"
      ],
      "layout": "IPY_MODEL_05e41fc637194280b6a2df80d5e07bce"
     }
    },
    "051ae4f420cd47628c19f0ef96803995": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_831c6e88707c41418d44a450fd168c8c",
      "placeholder": "​",
      "style": "IPY_MODEL_bdc135c82d8549cda8dc8266b6d28744",
      "value": "Downloading builder script: 100%"
     }
    },
    "e4d7a6143fca4847a827565566c305a2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_116ac1adfe6840e0b4b88b573e506351",
      "max": 5937,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1da39059b4c24b689e747a8cfdaf77e6",
      "value": 5937
     }
    },
    "554c85e291984153a4fda3be0062eb08": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d56ba311aab143f2af9ad64fcd1a8731",
      "placeholder": "​",
      "style": "IPY_MODEL_9d3c408f65054f40adf71aee594d0cd0",
      "value": " 5.94k/5.94k [00:00&lt;00:00, 355kB/s]"
     }
    },
    "05e41fc637194280b6a2df80d5e07bce": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "831c6e88707c41418d44a450fd168c8c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bdc135c82d8549cda8dc8266b6d28744": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "116ac1adfe6840e0b4b88b573e506351": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1da39059b4c24b689e747a8cfdaf77e6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d56ba311aab143f2af9ad64fcd1a8731": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9d3c408f65054f40adf71aee594d0cd0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d3aecc5dbdf943fc83a62b222ab3e701": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_18e325e59b614c9680bd072f68a5d8f3",
       "IPY_MODEL_a4779a6e714046a397c644c533854674",
       "IPY_MODEL_c0b8bda14a7747dfb003662d95c73dfc"
      ],
      "layout": "IPY_MODEL_7f32501b31334ddfae549be764dd0d71"
     }
    },
    "18e325e59b614c9680bd072f68a5d8f3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4c81895beec54f718ad80548bf1118d7",
      "placeholder": "​",
      "style": "IPY_MODEL_a2d06eba88cf4d11a71c258e5ed67ddf",
      "value": "Downloading extra modules: "
     }
    },
    "a4779a6e714046a397c644c533854674": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5c05b2f3aa7c4fa1ae636f701ca0892b",
      "max": 1554,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5a58e1c5ffc44de1af524826a28059cb",
      "value": 1554
     }
    },
    "c0b8bda14a7747dfb003662d95c73dfc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_88313eb879c742a290231489e8e317f9",
      "placeholder": "​",
      "style": "IPY_MODEL_c81441f912a64129927744af70b8369e",
      "value": " 4.07k/? [00:00&lt;00:00, 176kB/s]"
     }
    },
    "7f32501b31334ddfae549be764dd0d71": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4c81895beec54f718ad80548bf1118d7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a2d06eba88cf4d11a71c258e5ed67ddf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5c05b2f3aa7c4fa1ae636f701ca0892b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5a58e1c5ffc44de1af524826a28059cb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "88313eb879c742a290231489e8e317f9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c81441f912a64129927744af70b8369e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4a13fda5f49d497f8fca1ce8413ffd4b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_58e2c5cf163f4ee09e859bfb8c712512",
       "IPY_MODEL_a25c47accf9d446180914c9066dd4d03",
       "IPY_MODEL_99ee1547b9e34f9f88e77a4c8d688dbb"
      ],
      "layout": "IPY_MODEL_a8ae735e4abb4234b1ee06ba4e55b241"
     }
    },
    "58e2c5cf163f4ee09e859bfb8c712512": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_35a8cb8935c74298b49105539d71310e",
      "placeholder": "​",
      "style": "IPY_MODEL_8b91f07b56bf443aa5343ba9ab3d1236",
      "value": "Downloading extra modules: 100%"
     }
    },
    "a25c47accf9d446180914c9066dd4d03": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5fcf21c145bc4ebfac26089e1ddcf1c3",
      "max": 3344,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_70ebb6ec218f40f1a2e66b83c1e6dcdf",
      "value": 3344
     }
    },
    "99ee1547b9e34f9f88e77a4c8d688dbb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8d3bb7e843804a3cbdef41b1c93f2560",
      "placeholder": "​",
      "style": "IPY_MODEL_b88a2b516eae4dcfbbe6830850fde735",
      "value": " 3.34k/3.34k [00:00&lt;00:00, 180kB/s]"
     }
    },
    "a8ae735e4abb4234b1ee06ba4e55b241": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "35a8cb8935c74298b49105539d71310e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8b91f07b56bf443aa5343ba9ab3d1236": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5fcf21c145bc4ebfac26089e1ddcf1c3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "70ebb6ec218f40f1a2e66b83c1e6dcdf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "8d3bb7e843804a3cbdef41b1c93f2560": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b88a2b516eae4dcfbbe6830850fde735": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ab62a871485b4a50898595285cd6b162": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_9a7da06768574645a46d0aa9e9fa2722",
       "IPY_MODEL_d50bb1c95ec746aba820dcd9b1722401",
       "IPY_MODEL_564f318ec0ea4045b7439032ffe97b7d"
      ],
      "layout": "IPY_MODEL_3c93c5b03e964e339730fb0a8e3ccd3d"
     }
    },
    "9a7da06768574645a46d0aa9e9fa2722": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ea5abbd055024ca98f03f2daf332284f",
      "placeholder": "​",
      "style": "IPY_MODEL_13d0beb61d244723a9fe4e5fccfc7bcc",
      "value": "Downloading builder script: 100%"
     }
    },
    "d50bb1c95ec746aba820dcd9b1722401": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ac6e1d90a6ae4504b17c1ccadbfc4e6e",
      "max": 6270,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c9e94e2ab35f416c930cf5675e9a8c6b",
      "value": 6270
     }
    },
    "564f318ec0ea4045b7439032ffe97b7d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_242a040e8b434513880cdc33aae097c2",
      "placeholder": "​",
      "style": "IPY_MODEL_2d7f8772755a4e4b8508fda8c4bc7438",
      "value": " 6.27k/6.27k [00:00&lt;00:00, 205kB/s]"
     }
    },
    "3c93c5b03e964e339730fb0a8e3ccd3d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ea5abbd055024ca98f03f2daf332284f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "13d0beb61d244723a9fe4e5fccfc7bcc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ac6e1d90a6ae4504b17c1ccadbfc4e6e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c9e94e2ab35f416c930cf5675e9a8c6b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "242a040e8b434513880cdc33aae097c2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2d7f8772755a4e4b8508fda8c4bc7438": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ed7e1ef5655b4460ab7b5186f438c64f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c7adb46dadd74f249647fdcb22ea40e7",
       "IPY_MODEL_66ad787dbf5c49abad425f8eaadbb447",
       "IPY_MODEL_51735840ede543669d7ed21aef805428"
      ],
      "layout": "IPY_MODEL_ca4ef0b0550842b8b7770f0d337473b6"
     }
    },
    "c7adb46dadd74f249647fdcb22ea40e7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cccf7a0295e74c9b94aca66bf4313752",
      "placeholder": "​",
      "style": "IPY_MODEL_e55847dc6159494794395c7df07ffde7",
      "value": "Downloading builder script: 100%"
     }
    },
    "66ad787dbf5c49abad425f8eaadbb447": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e33b05b166384fcda9fa9620da3f2c38",
      "max": 7950,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e2c05f613fcc49f892cd384661a9935b",
      "value": 7950
     }
    },
    "51735840ede543669d7ed21aef805428": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_04d5aea12628438590b96ecde03e46b9",
      "placeholder": "​",
      "style": "IPY_MODEL_2f046d82889f474bac793b91809eb387",
      "value": " 7.95k/7.95k [00:00&lt;00:00, 523kB/s]"
     }
    },
    "ca4ef0b0550842b8b7770f0d337473b6": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cccf7a0295e74c9b94aca66bf4313752": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e55847dc6159494794395c7df07ffde7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e33b05b166384fcda9fa9620da3f2c38": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e2c05f613fcc49f892cd384661a9935b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "04d5aea12628438590b96ecde03e46b9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2f046d82889f474bac793b91809eb387": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "834c03b008544b39a8c41f92d91b9017": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a2c04366af704af580dff65644578e93",
       "IPY_MODEL_f1b58582c1a74d5cbe567bcbadc92a0e",
       "IPY_MODEL_08cdcc4236ab462d9ec860e7e83e6e45"
      ],
      "layout": "IPY_MODEL_ccc477343005421ebfcf4eecc5c68f0f"
     }
    },
    "a2c04366af704af580dff65644578e93": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f3f70256597e4c6ba5441fe35f5ddef4",
      "placeholder": "​",
      "style": "IPY_MODEL_e6aae83d83c44a7fa1cdd20925a839c2",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "f1b58582c1a74d5cbe567bcbadc92a0e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0d64578a4f11450991e36c892d7ff785",
      "max": 25,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_eabb559f76914d709fae4ac97616813c",
      "value": 25
     }
    },
    "08cdcc4236ab462d9ec860e7e83e6e45": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f0a8bcad4f324495aa00891b85339d8d",
      "placeholder": "​",
      "style": "IPY_MODEL_75b99a13350041e29e54fbdb47572f5a",
      "value": " 25.0/25.0 [00:00&lt;00:00, 991B/s]"
     }
    },
    "ccc477343005421ebfcf4eecc5c68f0f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f3f70256597e4c6ba5441fe35f5ddef4": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e6aae83d83c44a7fa1cdd20925a839c2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0d64578a4f11450991e36c892d7ff785": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eabb559f76914d709fae4ac97616813c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f0a8bcad4f324495aa00891b85339d8d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "75b99a13350041e29e54fbdb47572f5a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "391da2f76d0f4be1bc5758b2a3bc3118": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_dd2839a63681428c9264ffc628045f42",
       "IPY_MODEL_c728d143846849ee97b25e992ea6b913",
       "IPY_MODEL_a278f2e97d7a4d37ac0243ab55063e04"
      ],
      "layout": "IPY_MODEL_ca294e7b1eb745999dee3ed877e4c099"
     }
    },
    "dd2839a63681428c9264ffc628045f42": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e7a1baf86a86474890b0de827a85c0af",
      "placeholder": "​",
      "style": "IPY_MODEL_c42e8b6969684ae79d04764f2c6dead2",
      "value": "config.json: 100%"
     }
    },
    "c728d143846849ee97b25e992ea6b913": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e83e90890c29406489109c650b125a1d",
      "max": 482,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_38026d179cbf4ad29e4f287192410a91",
      "value": 482
     }
    },
    "a278f2e97d7a4d37ac0243ab55063e04": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c26bea6bf970423c8a48ff6267aae71f",
      "placeholder": "​",
      "style": "IPY_MODEL_04c55e8b021949d59142203c390662de",
      "value": " 482/482 [00:00&lt;00:00, 21.8kB/s]"
     }
    },
    "ca294e7b1eb745999dee3ed877e4c099": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e7a1baf86a86474890b0de827a85c0af": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c42e8b6969684ae79d04764f2c6dead2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e83e90890c29406489109c650b125a1d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "38026d179cbf4ad29e4f287192410a91": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c26bea6bf970423c8a48ff6267aae71f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "04c55e8b021949d59142203c390662de": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9133bd0248ef4f07a083b7c756ecac58": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f7e178beb7ad4c6cb3e1a9c728fa5c50",
       "IPY_MODEL_124237f6ac514b5794a6ca967adc96b0",
       "IPY_MODEL_ca0543c972394bcab6947d27e1b33b47"
      ],
      "layout": "IPY_MODEL_fd3e86b41c024921bda45970c4449748"
     }
    },
    "f7e178beb7ad4c6cb3e1a9c728fa5c50": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_31a761ae907245eeab75207e3576f821",
      "placeholder": "​",
      "style": "IPY_MODEL_c3408de3d9ca42c9bba2572754f7d9a7",
      "value": "vocab.json: 100%"
     }
    },
    "124237f6ac514b5794a6ca967adc96b0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0e55089cf2f648a8b76e99f1198f3851",
      "max": 898823,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_bf4c5734ec7d478487d32c4d7721b750",
      "value": 898823
     }
    },
    "ca0543c972394bcab6947d27e1b33b47": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_87ffb501c4624d3bbc6bcbd0c234a23d",
      "placeholder": "​",
      "style": "IPY_MODEL_9fe03839860c4bb3a684c3142f74a6a5",
      "value": " 899k/899k [00:00&lt;00:00, 11.0MB/s]"
     }
    },
    "fd3e86b41c024921bda45970c4449748": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "31a761ae907245eeab75207e3576f821": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c3408de3d9ca42c9bba2572754f7d9a7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0e55089cf2f648a8b76e99f1198f3851": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bf4c5734ec7d478487d32c4d7721b750": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "87ffb501c4624d3bbc6bcbd0c234a23d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9fe03839860c4bb3a684c3142f74a6a5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d490c04d6f3e47328bf442eaeb5db7c1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_6604e705c356441fa3bcf44f749db803",
       "IPY_MODEL_fdb4b65e48c14d6b814661a19022c952",
       "IPY_MODEL_b6c032e0311f406c8d7d8f303950b21f"
      ],
      "layout": "IPY_MODEL_2240383532fc4a0d86d530c297041a72"
     }
    },
    "6604e705c356441fa3bcf44f749db803": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7bcd95d4afbf41f7b6b7b46ff0f5d298",
      "placeholder": "​",
      "style": "IPY_MODEL_6204a4d12ba24491bcab3f46b1ae837f",
      "value": "merges.txt: 100%"
     }
    },
    "fdb4b65e48c14d6b814661a19022c952": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_34a578b037d444b299184b23e951d53c",
      "max": 456318,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9d05c6a1515b4cff86819914c22db465",
      "value": 456318
     }
    },
    "b6c032e0311f406c8d7d8f303950b21f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e1009e5016c34ae78b3116b316938e93",
      "placeholder": "​",
      "style": "IPY_MODEL_18c7ce83ed534293a9f2f2d2ab0df23a",
      "value": " 456k/456k [00:00&lt;00:00, 8.69MB/s]"
     }
    },
    "2240383532fc4a0d86d530c297041a72": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7bcd95d4afbf41f7b6b7b46ff0f5d298": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6204a4d12ba24491bcab3f46b1ae837f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "34a578b037d444b299184b23e951d53c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9d05c6a1515b4cff86819914c22db465": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e1009e5016c34ae78b3116b316938e93": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "18c7ce83ed534293a9f2f2d2ab0df23a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6a36b773606f47148c2e3f427956eda2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_89c9e05b87c44c689d4dfb5da7a1e9df",
       "IPY_MODEL_cf1e3b1a708b4df1b2c368b563ae3d48",
       "IPY_MODEL_0b41255ca2db4ff0b4f11d91589513d0"
      ],
      "layout": "IPY_MODEL_54fa8a1f67734ecbbb34b717295db173"
     }
    },
    "89c9e05b87c44c689d4dfb5da7a1e9df": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b5aaa32e99464bc59cbf3c7f1fab0e36",
      "placeholder": "​",
      "style": "IPY_MODEL_489209144d7c42ff88961adf48d22e20",
      "value": "tokenizer.json: 100%"
     }
    },
    "cf1e3b1a708b4df1b2c368b563ae3d48": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d5ca8f29ea1d4f229d413950c365938e",
      "max": 1355863,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_cc3b808cd68041658fefb195ee47f117",
      "value": 1355863
     }
    },
    "0b41255ca2db4ff0b4f11d91589513d0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e4d9e01c6c87413281fe0220a00f07f8",
      "placeholder": "​",
      "style": "IPY_MODEL_28dca91697ae4a489f72408b586bc930",
      "value": " 1.36M/1.36M [00:00&lt;00:00, 15.2MB/s]"
     }
    },
    "54fa8a1f67734ecbbb34b717295db173": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b5aaa32e99464bc59cbf3c7f1fab0e36": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "489209144d7c42ff88961adf48d22e20": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d5ca8f29ea1d4f229d413950c365938e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cc3b808cd68041658fefb195ee47f117": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e4d9e01c6c87413281fe0220a00f07f8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "28dca91697ae4a489f72408b586bc930": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8f42fb4d6c82470e9f8499236d9b771f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7773f7cdac984ce08e7fa77d4acc167c",
       "IPY_MODEL_15a949c80d01436b8f5e05ffa6889cbf",
       "IPY_MODEL_b15d7eeabf724b76a72e2ade1a00b6c5"
      ],
      "layout": "IPY_MODEL_fd33f022f9504e308e193a7323f4b38c"
     }
    },
    "7773f7cdac984ce08e7fa77d4acc167c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_33f0a478237d445ebb84e17f3a86a502",
      "placeholder": "​",
      "style": "IPY_MODEL_90f434d040f94167bad1ac152a45138f",
      "value": "model.safetensors: 100%"
     }
    },
    "15a949c80d01436b8f5e05ffa6889cbf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3ecaecd0b28749c8acccf74e1cb55be3",
      "max": 1421700479,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_3a6793b7c618449baa32b3b622a96c88",
      "value": 1421700479
     }
    },
    "b15d7eeabf724b76a72e2ade1a00b6c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d6f913d8dafd4603a93b582fe994e006",
      "placeholder": "​",
      "style": "IPY_MODEL_0eeb8ae9055341f6b20d9a6a0899e2f1",
      "value": " 1.42G/1.42G [00:09&lt;00:00, 79.8MB/s]"
     }
    },
    "fd33f022f9504e308e193a7323f4b38c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "33f0a478237d445ebb84e17f3a86a502": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "90f434d040f94167bad1ac152a45138f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3ecaecd0b28749c8acccf74e1cb55be3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3a6793b7c618449baa32b3b622a96c88": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d6f913d8dafd4603a93b582fe994e006": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0eeb8ae9055341f6b20d9a6a0899e2f1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c7fa5aebed914a7a8119446ffe3eace1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_6e5cee424c9744eca134522502ffbd50",
       "IPY_MODEL_8dc9b127022a435ca36a980cc2cbf480",
       "IPY_MODEL_af2f82fcabf240b29c24837f49634669"
      ],
      "layout": "IPY_MODEL_0b9c60753dea46c19a63f3f22056a52d"
     }
    },
    "6e5cee424c9744eca134522502ffbd50": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f3c09a3133ab432fb977dc88e7468bab",
      "placeholder": "​",
      "style": "IPY_MODEL_51e65deb6872469da0bfe033daf43489",
      "value": "100%"
     }
    },
    "8dc9b127022a435ca36a980cc2cbf480": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6167e1e7720845fc9c209acb01d7a31c",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5ffe56cc26254574a40aeb55c1f06250",
      "value": 75
     }
    },
    "af2f82fcabf240b29c24837f49634669": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_20296844674746d0b28ae8695ecb7f3b",
      "placeholder": "​",
      "style": "IPY_MODEL_4812fba6cc2f44e08e34c41b2acfea81",
      "value": " 75/75 [07:45&lt;00:00,  5.82s/it]"
     }
    },
    "0b9c60753dea46c19a63f3f22056a52d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f3c09a3133ab432fb977dc88e7468bab": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "51e65deb6872469da0bfe033daf43489": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6167e1e7720845fc9c209acb01d7a31c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5ffe56cc26254574a40aeb55c1f06250": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "20296844674746d0b28ae8695ecb7f3b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4812fba6cc2f44e08e34c41b2acfea81": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "976e1a4b685b4233b472e568705e7775": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_825132085c2a4f8992bf2f301109d606",
       "IPY_MODEL_c102d7d2b15a41808ee22b1da57d6665",
       "IPY_MODEL_0403fdffcf2c4846a14ad4ff062bbe7a"
      ],
      "layout": "IPY_MODEL_bcdf98dda3084383bd6856ae3f57e62c"
     }
    },
    "825132085c2a4f8992bf2f301109d606": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3e59d8afc712401590ed2bd915df3a00",
      "placeholder": "​",
      "style": "IPY_MODEL_81f7c1411dc94120a0e5fa088e3e1517",
      "value": "100%"
     }
    },
    "c102d7d2b15a41808ee22b1da57d6665": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_157b31bc02734196b24dfa7bb69638f5",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b57015603c684ec3bb96bd5bf180f60a",
      "value": 75
     }
    },
    "0403fdffcf2c4846a14ad4ff062bbe7a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a0c78826c3b646e29cb68e52eff1cb31",
      "placeholder": "​",
      "style": "IPY_MODEL_1293a890dc224affa97865081453938f",
      "value": " 75/75 [06:12&lt;00:00,  4.82s/it]"
     }
    },
    "bcdf98dda3084383bd6856ae3f57e62c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3e59d8afc712401590ed2bd915df3a00": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "81f7c1411dc94120a0e5fa088e3e1517": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "157b31bc02734196b24dfa7bb69638f5": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b57015603c684ec3bb96bd5bf180f60a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a0c78826c3b646e29cb68e52eff1cb31": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1293a890dc224affa97865081453938f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5778db39bc114992a8f6c31b302bb44b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_712d7cffcc5f4495b88190ffa245c587",
       "IPY_MODEL_923575169bcb4b7d8612028b1738a98a",
       "IPY_MODEL_2aa6158ab8cf4663b2fb2e2a76d75774"
      ],
      "layout": "IPY_MODEL_c58a08489d26472e84a2b237ff99d2ef"
     }
    },
    "712d7cffcc5f4495b88190ffa245c587": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_77f990018462443c8f1afe29e7122c41",
      "placeholder": "​",
      "style": "IPY_MODEL_86c0789f0d954fdcbceb9346a0fd50e1",
      "value": "100%"
     }
    },
    "923575169bcb4b7d8612028b1738a98a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_25baed0450454523aaef42522d2c25c2",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_83d86f4a3e524feaaf967327ae91c133",
      "value": 75
     }
    },
    "2aa6158ab8cf4663b2fb2e2a76d75774": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ace18903d734403a9e2b2979a3bb6ed3",
      "placeholder": "​",
      "style": "IPY_MODEL_5afa6c6bc56a4b0aa4467c47c72b56fa",
      "value": " 75/75 [00:13&lt;00:00,  6.34it/s]"
     }
    },
    "c58a08489d26472e84a2b237ff99d2ef": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "77f990018462443c8f1afe29e7122c41": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "86c0789f0d954fdcbceb9346a0fd50e1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "25baed0450454523aaef42522d2c25c2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "83d86f4a3e524feaaf967327ae91c133": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "ace18903d734403a9e2b2979a3bb6ed3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5afa6c6bc56a4b0aa4467c47c72b56fa": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "72d2621aa2cc46128fb80e642870613e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_52e0d474b14e411d94d44471adc51d3e",
       "IPY_MODEL_a3f0fdef77484329a0b85fd8b72a002f",
       "IPY_MODEL_e0e08b2a73464e1fb770195ccbf63705"
      ],
      "layout": "IPY_MODEL_adca83dd502744f0b2c3cb6c32d62c95"
     }
    },
    "52e0d474b14e411d94d44471adc51d3e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_902ae631c3504613ac48a03ce49cb3cd",
      "placeholder": "​",
      "style": "IPY_MODEL_864454ad08b0462b8a1177ff976e097d",
      "value": "100%"
     }
    },
    "a3f0fdef77484329a0b85fd8b72a002f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5f57760bc4d84024a81fee52009586a9",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_afdc394622a4403aa50be734a69c35d3",
      "value": 75
     }
    },
    "e0e08b2a73464e1fb770195ccbf63705": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f1576ba2ec794d55a59d7472691f9339",
      "placeholder": "​",
      "style": "IPY_MODEL_75e1177de3db44eab0d0ac2ce216d4c4",
      "value": " 75/75 [00:03&lt;00:00, 23.77it/s]"
     }
    },
    "adca83dd502744f0b2c3cb6c32d62c95": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "902ae631c3504613ac48a03ce49cb3cd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "864454ad08b0462b8a1177ff976e097d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5f57760bc4d84024a81fee52009586a9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "afdc394622a4403aa50be734a69c35d3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f1576ba2ec794d55a59d7472691f9339": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "75e1177de3db44eab0d0ac2ce216d4c4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c3be67afe07d47e38ea9c45569939469": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5cf7acd34f1c4fadab40efc2d46cfae5",
       "IPY_MODEL_f6562ce628ca4e14a0edbee346ba14ee",
       "IPY_MODEL_2003ecee31914f43a81048562c25fbb9"
      ],
      "layout": "IPY_MODEL_cd46cb9433c94d60ae560b279878c74c"
     }
    },
    "5cf7acd34f1c4fadab40efc2d46cfae5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6a7714dce29740b593e6663a664b038a",
      "placeholder": "​",
      "style": "IPY_MODEL_5c79112b538d44738ef1c60dc42b9c9b",
      "value": "100%"
     }
    },
    "f6562ce628ca4e14a0edbee346ba14ee": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3679dcf8a39c49a8863ba16674bf64ca",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8e762196f7d94891a63b2a64800e59fc",
      "value": 75
     }
    },
    "2003ecee31914f43a81048562c25fbb9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e52485bbbab44e0ab6f916bbc02e83c1",
      "placeholder": "​",
      "style": "IPY_MODEL_fcdf13af13994b05a091a8026a1c0938",
      "value": " 75/75 [00:13&lt;00:00,  6.28it/s]"
     }
    },
    "cd46cb9433c94d60ae560b279878c74c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a7714dce29740b593e6663a664b038a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5c79112b538d44738ef1c60dc42b9c9b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3679dcf8a39c49a8863ba16674bf64ca": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8e762196f7d94891a63b2a64800e59fc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e52485bbbab44e0ab6f916bbc02e83c1": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fcdf13af13994b05a091a8026a1c0938": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8df40ee934674cad9d61f11ce3e0db21": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_416ff5fc14f141e9801a7e62d71aca98",
       "IPY_MODEL_13f885c14b4a4317a59f5c79ffedad9c",
       "IPY_MODEL_38ccf2ce2bb7472f92d33949abec18d7"
      ],
      "layout": "IPY_MODEL_3b646c078cc848419cbafd591a4081b1"
     }
    },
    "416ff5fc14f141e9801a7e62d71aca98": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a0b603b86643417b923e91c6bd33a7ee",
      "placeholder": "​",
      "style": "IPY_MODEL_75fa22125ffd48d58bde40b6d8ac4990",
      "value": "100%"
     }
    },
    "13f885c14b4a4317a59f5c79ffedad9c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d7580f74a21b40fdbaff386ca0bcb888",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_85ab9557cb5d4f6aacb5c36f4186d9aa",
      "value": 75
     }
    },
    "38ccf2ce2bb7472f92d33949abec18d7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c391d01b0e1a43c7840175b51c0fbfbc",
      "placeholder": "​",
      "style": "IPY_MODEL_cf72beb3ea604d38ab4da728c9419e5b",
      "value": " 75/75 [00:02&lt;00:00, 27.95it/s]"
     }
    },
    "3b646c078cc848419cbafd591a4081b1": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a0b603b86643417b923e91c6bd33a7ee": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "75fa22125ffd48d58bde40b6d8ac4990": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d7580f74a21b40fdbaff386ca0bcb888": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "85ab9557cb5d4f6aacb5c36f4186d9aa": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c391d01b0e1a43c7840175b51c0fbfbc": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cf72beb3ea604d38ab4da728c9419e5b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e85c9670afe7406b8c94a1b9607371d0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b132f55973fd4a42a8a563895c26b0dd",
       "IPY_MODEL_3e36367514f0494c964834deebc0dd55",
       "IPY_MODEL_f78e6da9b092440f9fb482d55ea091b8"
      ],
      "layout": "IPY_MODEL_5b0f88ad2d1045df85ad4c9b36243c85"
     }
    },
    "b132f55973fd4a42a8a563895c26b0dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d4d67fd96ac0426abe408d39ec73f162",
      "placeholder": "​",
      "style": "IPY_MODEL_bee89ff017bc4dd8b928f33e0ea7eae9",
      "value": "100%"
     }
    },
    "3e36367514f0494c964834deebc0dd55": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_50089f650cc54427951644274bff6ea1",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1189d28a99534bd78ea42785c0406306",
      "value": 75
     }
    },
    "f78e6da9b092440f9fb482d55ea091b8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_132d799e7d984c16a8fe742aeba6e022",
      "placeholder": "​",
      "style": "IPY_MODEL_cf280191ef2d44caa80ca561ba6b266b",
      "value": " 75/75 [00:13&lt;00:00,  4.33it/s]"
     }
    },
    "5b0f88ad2d1045df85ad4c9b36243c85": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d4d67fd96ac0426abe408d39ec73f162": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bee89ff017bc4dd8b928f33e0ea7eae9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "50089f650cc54427951644274bff6ea1": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1189d28a99534bd78ea42785c0406306": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "132d799e7d984c16a8fe742aeba6e022": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cf280191ef2d44caa80ca561ba6b266b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fce574ee4bd7427eb6e4e8f2afd587fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2b05f0a04cec459884f620fdd34ac7e4",
       "IPY_MODEL_8e48d65637784406b250bf2b7f495fcb",
       "IPY_MODEL_313422453c784d039a249f372ee2ca51"
      ],
      "layout": "IPY_MODEL_91e8247ada37499481adf8674491855f"
     }
    },
    "2b05f0a04cec459884f620fdd34ac7e4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_db32d583fb8144b789910e6df68ca397",
      "placeholder": "​",
      "style": "IPY_MODEL_a56df808bf7f4c30b1639906b285cea1",
      "value": "100%"
     }
    },
    "8e48d65637784406b250bf2b7f495fcb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c373d163413540fa885826f6d1367372",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_11203e680c3740b182f09910410c979b",
      "value": 75
     }
    },
    "313422453c784d039a249f372ee2ca51": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9c33dc99b6524ec487075ef75511791e",
      "placeholder": "​",
      "style": "IPY_MODEL_d972c0cf99144713a18a2c4afeacd7e8",
      "value": " 75/75 [00:03&lt;00:00, 23.11it/s]"
     }
    },
    "91e8247ada37499481adf8674491855f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db32d583fb8144b789910e6df68ca397": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a56df808bf7f4c30b1639906b285cea1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c373d163413540fa885826f6d1367372": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "11203e680c3740b182f09910410c979b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9c33dc99b6524ec487075ef75511791e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d972c0cf99144713a18a2c4afeacd7e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ed0a34cdf8cd4e5c9c3cfce43f9b25ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f1ec6e5a37954e02a70f4002979778c7",
       "IPY_MODEL_96050a1be9db4178a24fb0a5da3cb9b8",
       "IPY_MODEL_3c7e805965eb4922bda7b9d723d8bbbb"
      ],
      "layout": "IPY_MODEL_b5a9251e0fae44168ba11bc78427e467"
     }
    },
    "f1ec6e5a37954e02a70f4002979778c7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f4ebe2b010744a3db4999b6d222d1224",
      "placeholder": "​",
      "style": "IPY_MODEL_39cfffdd84b84131a3449f6aa58ad0c8",
      "value": "100%"
     }
    },
    "96050a1be9db4178a24fb0a5da3cb9b8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b5a2cf6897324660b2e10b24d21cdabb",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ede15688e7b243fea67c88024c495ed2",
      "value": 75
     }
    },
    "3c7e805965eb4922bda7b9d723d8bbbb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3d308556fa5a4ef196fe084981d59089",
      "placeholder": "​",
      "style": "IPY_MODEL_6cdf6a5fad4041baa7028230d3485fb1",
      "value": " 75/75 [00:13&lt;00:00,  5.48it/s]"
     }
    },
    "b5a9251e0fae44168ba11bc78427e467": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f4ebe2b010744a3db4999b6d222d1224": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "39cfffdd84b84131a3449f6aa58ad0c8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b5a2cf6897324660b2e10b24d21cdabb": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ede15688e7b243fea67c88024c495ed2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "3d308556fa5a4ef196fe084981d59089": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6cdf6a5fad4041baa7028230d3485fb1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b1f670c6832f45a8b5e3ad62d9005569": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_6b517ad5f23643579e0b9e76dc00c588",
       "IPY_MODEL_f42dc020ee814f39a33370a6780d82e8",
       "IPY_MODEL_e4b463ba77af4fa38091da61ba9234b7"
      ],
      "layout": "IPY_MODEL_a61471851fcf404eaf9e90d692034068"
     }
    },
    "6b517ad5f23643579e0b9e76dc00c588": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e23cee15d88b45bcb22edf56b7066903",
      "placeholder": "​",
      "style": "IPY_MODEL_0bca12f762344c039cf5850eefe2787e",
      "value": "100%"
     }
    },
    "f42dc020ee814f39a33370a6780d82e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b2476e55bed24a8b8b18913ce29e2ff9",
      "max": 75,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a3033e42410945e189d3c70e15a5fde8",
      "value": 75
     }
    },
    "e4b463ba77af4fa38091da61ba9234b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "model_module_version": "1.5.0",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1595a07cdc524855b60b8e20c78dba82",
      "placeholder": "​",
      "style": "IPY_MODEL_7835ddde624c4dbdb02a016d19829dd9",
      "value": " 75/75 [00:02&lt;00:00, 35.96it/s]"
     }
    },
    "a61471851fcf404eaf9e90d692034068": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e23cee15d88b45bcb22edf56b7066903": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0bca12f762344c039cf5850eefe2787e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b2476e55bed24a8b8b18913ce29e2ff9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a3033e42410945e189d3c70e15a5fde8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "1595a07cdc524855b60b8e20c78dba82": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "model_module_version": "1.2.0",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7835ddde624c4dbdb02a016d19829dd9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "model_module_version": "1.5.0",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}