minotaur_15B_GPTQ.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Troyanovsky/Local-LLM-comparison/blob/main/minotaur_15B_GPTQ.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XNkpBtLvuTOp"
      },
      "source": [
        "## minotaur-15B-GPTQ WebUI\n",
        "1. Run the following cell, takes ~5 min\n",
        "2. Click the gradio link at the bottom\n",
        "3. In Chat settings - Instruction Template:\n",
        "```\n",
        "System: Below is a dialogue between a USER and an ASSISTANT. The USER may ask questions, request information, or provide instructions for a task, often supplementing with additional context. The ASSISTANT responds accurately and effectively, offering insights, answering questions, or executing tasks to the best of its ability based on the given information.\n",
        "\n",
        "USER: <prompt>\n",
        "ASSISTANT:\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "D-MiHp_bveP6",
        "outputId": "8861a49d-7c06-4036-d2ad-4c5c1b4f2fdd"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "/content\n",
            "Selecting previously unselected package libc-ares2:amd64.\n",
            "(Reading database ... 123069 files and directories currently installed.)\n",
            "Preparing to unpack .../libc-ares2_1.15.0-1ubuntu0.3_amd64.deb ...\n",
            "Unpacking libc-ares2:amd64 (1.15.0-1ubuntu0.3) ...\n",
            "Selecting previously unselected package libaria2-0:amd64.\n",
            "Preparing to unpack .../libaria2-0_1.35.0-1build1_amd64.deb ...\n",
            "Unpacking libaria2-0:amd64 (1.35.0-1build1) ...\n",
            "Selecting previously unselected package aria2.\n",
            "Preparing to unpack .../aria2_1.35.0-1build1_amd64.deb ...\n",
            "Unpacking aria2 (1.35.0-1build1) ...\n",
            "Setting up libc-ares2:amd64 (1.15.0-1ubuntu0.3) ...\n",
            "Setting up libaria2-0:amd64 (1.35.0-1build1) ...\n",
            "Setting up aria2 (1.35.0-1build1) ...\n",
            "Processing triggers for man-db (2.9.1-1) ...\n",
            "Processing triggers for libc-bin (2.31-0ubuntu9.9) ...\n",
            "Cloning into 'text-generation-webui'...\n",
            "remote: Enumerating objects: 8763, done.\u001b[K\n",
            "remote: Counting objects: 100% (192/192), done.\u001b[K\n",
            "remote: Compressing objects: 100% (132/132), done.\u001b[K\n",
            "remote: Total 8763 (delta 94), reused 125 (delta 56), pack-reused 8571\u001b[K\n",
            "Receiving objects: 100% (8763/8763), 3.02 MiB | 26.02 MiB/s, done.\n",
            "Resolving deltas: 100% (5840/5840), done.\n",
            "Note: switching to 'e52b43c934f412fa2ddf9d90db6cb088e7295689'.\n",
            "\n",
            "You are in 'detached HEAD' state. You can look around, make experimental\n",
            "changes and commit them, and you can discard any commits you make in this\n",
            "state without impacting any branches by switching back to a branch.\n",
            "\n",
            "If you want to create a new branch to retain commits you create, you may\n",
            "do so (now or later) by using -c with the switch command. Example:\n",
            "\n",
            "  git switch -c <new-branch-name>\n",
            "\n",
            "Or undo this operation with:\n",
            "\n",
            "  git switch -\n",
            "\n",
            "Turn off this advice by setting config variable advice.detachedHead to false\n",
            "\n",
            "/content/text-generation-webui\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting git+https://github.com/huggingface/peft@3714aa2fff158fdfa637b2b65952580801d890b2 (from -r requirements.txt (line 17))\n",
            "  Cloning https://github.com/huggingface/peft (to revision 3714aa2fff158fdfa637b2b65952580801d890b2) to /tmp/pip-req-build-t9dsh8g8\n",
            "  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft /tmp/pip-req-build-t9dsh8g8\n",
            "  Running command git rev-parse -q --verify 'sha^3714aa2fff158fdfa637b2b65952580801d890b2'\n",
            "  Running command git fetch -q https://github.com/huggingface/peft 3714aa2fff158fdfa637b2b65952580801d890b2\n",
            "  Running command git checkout -q 3714aa2fff158fdfa637b2b65952580801d890b2\n",
            "  Resolved https://github.com/huggingface/peft to commit 3714aa2fff158fdfa637b2b65952580801d890b2\n",
            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting git+https://github.com/huggingface/transformers@e45e756d22206ca8fa9fb057c8c3d8fa79bf81c6 (from -r requirements.txt (line 18))\n",
            "  Cloning https://github.com/huggingface/transformers (to revision e45e756d22206ca8fa9fb057c8c3d8fa79bf81c6) to /tmp/pip-req-build-btfep758\n",
            "  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers /tmp/pip-req-build-btfep758\n",
            "  Running command git rev-parse -q --verify 'sha^e45e756d22206ca8fa9fb057c8c3d8fa79bf81c6'\n",
            "  Running command git fetch -q https://github.com/huggingface/transformers e45e756d22206ca8fa9fb057c8c3d8fa79bf81c6\n",
            "  Running command git checkout -q e45e756d22206ca8fa9fb057c8c3d8fa79bf81c6\n",
            "  Resolved https://github.com/huggingface/transformers to commit e45e756d22206ca8fa9fb057c8c3d8fa79bf81c6\n",
            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting git+https://github.com/huggingface/accelerate@0226f750257b3bf2cadc4f189f9eef0c764a0467 (from -r requirements.txt (line 19))\n",
            "  Cloning https://github.com/huggingface/accelerate (to revision 0226f750257b3bf2cadc4f189f9eef0c764a0467) to /tmp/pip-req-build-fgcrn04q\n",
            "  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/accelerate /tmp/pip-req-build-fgcrn04q\n",
            "  Running command git rev-parse -q --verify 'sha^0226f750257b3bf2cadc4f189f9eef0c764a0467'\n",
            "  Running command git fetch -q https://github.com/huggingface/accelerate 0226f750257b3bf2cadc4f189f9eef0c764a0467\n",
            "  Running command git checkout -q 0226f750257b3bf2cadc4f189f9eef0c764a0467\n",
            "  Resolved https://github.com/huggingface/accelerate to commit 0226f750257b3bf2cadc4f189f9eef0c764a0467\n",
            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Ignoring bitsandbytes: markers 'platform_system == \"Windows\"' don't match your environment\n",
            "Ignoring llama-cpp-python: markers 'platform_system == \"Windows\"' don't match your environment\n",
            "Collecting colorama (from -r requirements.txt (line 1))\n",
            "  Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n",
            "Collecting datasets (from -r requirements.txt (line 2))\n",
            "  Downloading datasets-2.13.1-py3-none-any.whl (486 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m486.2/486.2 kB\u001b[0m \u001b[31m27.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting einops (from -r requirements.txt (line 3))\n",
            "  Downloading einops-0.6.1-py3-none-any.whl (42 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.2/42.2 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting flexgen==0.1.7 (from -r requirements.txt (line 4))\n",
            "  Downloading flexgen-0.1.7-py3-none-any.whl (50 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.8/50.8 kB\u001b[0m \u001b[31m5.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting gradio_client==0.2.5 (from -r requirements.txt (line 5))\n",
            "  Downloading gradio_client-0.2.5-py3-none-any.whl (288 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.1/288.1 kB\u001b[0m \u001b[31m29.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting gradio==3.31.0 (from -r requirements.txt (line 6))\n",
            "  Downloading gradio-3.31.0-py3-none-any.whl (17.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.4/17.4 MB\u001b[0m \u001b[31m81.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: markdown in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 7)) (3.4.3)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 8)) (1.22.4)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (1.5.3)\n",
            "Collecting Pillow>=9.5.0 (from -r requirements.txt (line 10))\n",
            "  Downloading Pillow-9.5.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m115.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 11)) (6.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (2.27.1)\n",
            "Collecting safetensors==0.3.1 (from -r requirements.txt (line 13))\n",
            "  Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m95.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting sentencepiece (from -r requirements.txt (line 14))\n",
            "  Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m83.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 15)) (4.65.0)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 16)) (1.10.1)\n",
            "Collecting bitsandbytes==0.39.0 (from -r requirements.txt (line 20))\n",
            "  Downloading bitsandbytes-0.39.0-py3-none-any.whl (92.2 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.2/92.2 MB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting llama-cpp-python==0.1.56 (from -r requirements.txt (line 22))\n",
            "  Downloading llama_cpp_python-0.1.56.tar.gz (1.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.4/1.4 MB\u001b[0m \u001b[31m90.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: torch>=1.12 in /usr/local/lib/python3.10/dist-packages (from flexgen==0.1.7->-r requirements.txt (line 4)) (2.0.1+cu118)\n",
            "Collecting pulp (from flexgen==0.1.7->-r requirements.txt (line 4))\n",
            "  Downloading PuLP-2.7.0-py3-none-any.whl (14.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m14.3/14.3 MB\u001b[0m \u001b[31m22.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: attrs in /usr/local/lib/python3.10/dist-packages (from flexgen==0.1.7->-r requirements.txt (line 4)) (23.1.0)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio_client==0.2.5->-r requirements.txt (line 5)) (2023.6.0)\n",
            "Collecting httpx (from gradio_client==0.2.5->-r requirements.txt (line 5))\n",
            "  Downloading httpx-0.24.1-py3-none-any.whl (75 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.4/75.4 kB\u001b[0m \u001b[31m8.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting huggingface-hub>=0.13.0 (from gradio_client==0.2.5->-r requirements.txt (line 5))\n",
            "  Downloading huggingface_hub-0.15.1-py3-none-any.whl (236 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m236.8/236.8 kB\u001b[0m \u001b[31m21.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio_client==0.2.5->-r requirements.txt (line 5)) (23.1)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from gradio_client==0.2.5->-r requirements.txt (line 5)) (4.6.3)\n",
            "Collecting websockets (from gradio_client==0.2.5->-r requirements.txt (line 5))\n",
            "  Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m14.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting aiofiles (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading aiofiles-23.1.0-py3-none-any.whl (14 kB)\n",
            "Collecting aiohttp (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading aiohttp-3.8.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m77.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.31.0->-r requirements.txt (line 6)) (4.2.2)\n",
            "Collecting fastapi (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading fastapi-0.98.0-py3-none-any.whl (56 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.0/57.0 kB\u001b[0m \u001b[31m8.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting ffmpy (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading ffmpy-0.3.0.tar.gz (4.8 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from gradio==3.31.0->-r requirements.txt (line 6)) (3.1.2)\n",
            "Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.31.0->-r requirements.txt (line 6)) (3.0.0)\n",
            "Requirement already satisfied: markupsafe in /usr/local/lib/python3.10/dist-packages (from gradio==3.31.0->-r requirements.txt (line 6)) (2.1.3)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from gradio==3.31.0->-r requirements.txt (line 6)) (3.7.1)\n",
            "Collecting mdit-py-plugins<=0.3.3 (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.5/50.5 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting orjson (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading orjson-3.9.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (136 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m137.0/137.0 kB\u001b[0m \u001b[31m17.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pydantic in /usr/local/lib/python3.10/dist-packages (from gradio==3.31.0->-r requirements.txt (line 6)) (1.10.9)\n",
            "Collecting pydub (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n",
            "Requirement already satisfied: pygments>=2.12.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.31.0->-r requirements.txt (line 6)) (2.14.0)\n",
            "Collecting python-multipart (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting semantic-version (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n",
            "Collecting uvicorn>=0.14.0 (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading uvicorn-0.22.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets->-r requirements.txt (line 2)) (9.0.0)\n",
            "Collecting dill<0.3.7,>=0.3.0 (from datasets->-r requirements.txt (line 2))\n",
            "  Downloading dill-0.3.6-py3-none-any.whl (110 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m110.5/110.5 kB\u001b[0m \u001b[31m15.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting xxhash (from datasets->-r requirements.txt (line 2))\n",
            "  Downloading xxhash-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m212.5/212.5 kB\u001b[0m \u001b[31m30.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting multiprocess (from datasets->-r requirements.txt (line 2))\n",
            "  Downloading multiprocess-0.70.14-py310-none-any.whl (134 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.3/134.3 kB\u001b[0m \u001b[31m21.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 9)) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 9)) (2022.7.1)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 12)) (1.26.16)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 12)) (2023.5.7)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 12)) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 12)) (3.4)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from peft==0.4.0.dev0->-r requirements.txt (line 17)) (5.9.5)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.0.dev0->-r requirements.txt (line 18)) (3.12.2)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.0.dev0->-r requirements.txt (line 18)) (2022.10.31)\n",
            "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.30.0.dev0->-r requirements.txt (line 18))\n",
            "  Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m107.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.31.0->-r requirements.txt (line 6)) (0.4)\n",
            "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.31.0->-r requirements.txt (line 6)) (4.3.3)\n",
            "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.31.0->-r requirements.txt (line 6)) (0.12.0)\n",
            "Collecting multidict<7.0,>=4.5 (from aiohttp->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading multidict-6.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.5/114.5 kB\u001b[0m \u001b[31m16.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\n",
            "Collecting yarl<2.0,>=1.0 (from aiohttp->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading yarl-1.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (268 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m33.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting frozenlist>=1.1.1 (from aiohttp->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading frozenlist-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m149.6/149.6 kB\u001b[0m \u001b[31m22.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting aiosignal>=1.1.2 (from aiohttp->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\n",
            "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.31.0->-r requirements.txt (line 6)) (0.1.2)\n",
            "Collecting linkify-it-py<3,>=1 (from markdown-it-py[linkify]>=2.0.0->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading linkify_it_py-2.0.2-py3-none-any.whl (19 kB)\n",
            "INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n",
            "Collecting mdit-py-plugins<=0.3.3 (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading mdit_py_plugins-0.3.2-py3-none-any.whl (50 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.4/50.4 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.3.1-py3-none-any.whl (46 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.5/46.5 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.3.0-py3-none-any.whl (43 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.2.8-py3-none-any.whl (41 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.2.7-py3-none-any.whl (41 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.2.6-py3-none-any.whl (39 kB)\n",
            "  Downloading mdit_py_plugins-0.2.5-py3-none-any.whl (39 kB)\n",
            "INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n",
            "  Downloading mdit_py_plugins-0.2.4-py3-none-any.whl (39 kB)\n",
            "  Downloading mdit_py_plugins-0.2.3-py3-none-any.whl (39 kB)\n",
            "  Downloading mdit_py_plugins-0.2.2-py3-none-any.whl (39 kB)\n",
            "  Downloading mdit_py_plugins-0.2.1-py3-none-any.whl (38 kB)\n",
            "  Downloading mdit_py_plugins-0.2.0-py3-none-any.whl (38 kB)\n",
            "INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n",
            "  Downloading mdit_py_plugins-0.1.0-py3-none-any.whl (37 kB)\n",
            "Collecting markdown-it-py[linkify]>=2.0.0 (from gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m13.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading markdown_it_py-2.2.0-py3-none-any.whl (84 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.5/84.5 kB\u001b[0m \u001b[31m12.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->-r requirements.txt (line 9)) (1.16.0)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.12->flexgen==0.1.7->-r requirements.txt (line 4)) (1.11.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.12->flexgen==0.1.7->-r requirements.txt (line 4)) (3.1)\n",
            "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.12->flexgen==0.1.7->-r requirements.txt (line 4)) (2.0.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.12->flexgen==0.1.7->-r requirements.txt (line 4)) (3.25.2)\n",
            "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.12->flexgen==0.1.7->-r requirements.txt (line 4)) (16.0.6)\n",
            "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.31.0->-r requirements.txt (line 6)) (8.1.3)\n",
            "Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting starlette<0.28.0,>=0.27.0 (from fastapi->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m9.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting httpcore<0.18.0,>=0.15.0 (from httpx->gradio_client==0.2.5->-r requirements.txt (line 5))\n",
            "  Downloading httpcore-0.17.2-py3-none-any.whl (72 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m72.5/72.5 kB\u001b[0m \u001b[31m11.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio_client==0.2.5->-r requirements.txt (line 5)) (1.3.0)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.31.0->-r requirements.txt (line 6)) (1.1.0)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.31.0->-r requirements.txt (line 6)) (0.11.0)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.31.0->-r requirements.txt (line 6)) (4.40.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.31.0->-r requirements.txt (line 6)) (1.4.4)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.31.0->-r requirements.txt (line 6)) (3.1.0)\n",
            "Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.10/dist-packages (from httpcore<0.18.0,>=0.15.0->httpx->gradio_client==0.2.5->-r requirements.txt (line 5)) (3.7.0)\n",
            "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.31.0->-r requirements.txt (line 6)) (0.19.3)\n",
            "Collecting uc-micro-py (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio==3.31.0->-r requirements.txt (line 6))\n",
            "  Downloading uc_micro_py-1.0.2-py3-none-any.whl (6.2 kB)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.12->flexgen==0.1.7->-r requirements.txt (line 4)) (1.3.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5.0,>=3.0->httpcore<0.18.0,>=0.15.0->httpx->gradio_client==0.2.5->-r requirements.txt (line 5)) (1.1.1)\n",
            "Building wheels for collected packages: llama-cpp-python, peft, transformers, accelerate, ffmpy\n",
            "  Building wheel for llama-cpp-python (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.56-cp310-cp310-linux_x86_64.whl size=210595 sha256=3747c40507155a2175abf158942830c3ee443c7ab276c68080009ae7ca2c30cf\n",
            "  Stored in directory: /root/.cache/pip/wheels/81/ab/27/4fedca13e6164e94a70a061310cd011c894fc699559af26edf\n",
            "  Building wheel for peft (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for peft: filename=peft-0.4.0.dev0-py3-none-any.whl size=56971 sha256=7406ac39e6229b8a44da862c9756e8dd2ed4227f9f216917959dec56ffbf6610\n",
            "  Stored in directory: /root/.cache/pip/wheels/5f/d0/6d/4b23d09e9ae6a2be26784237ac5b1801ee19fb9c2b2597b374\n",
            "  Building wheel for transformers (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for transformers: filename=transformers-4.30.0.dev0-py3-none-any.whl size=7102790 sha256=166ff60bbec09533239f030edc6ad2aed17bdb5ae86cec1ccd747cd979230a9b\n",
            "  Stored in directory: /root/.cache/pip/wheels/31/a7/0d/74848287c66e147f41c6ae09e116236d00775d52a84c862021\n",
            "  Building wheel for accelerate (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for accelerate: filename=accelerate-0.20.0.dev0-py3-none-any.whl size=225769 sha256=3dd0cd9f4df30b6a2ff077b4b0b529656d84438626d3c51db4ae4512d59b5354\n",
            "  Stored in directory: /root/.cache/pip/wheels/1e/61/eb/5c95b515e16875f1653f1b8bc1fb7c9b35c6eee91cd97b829a\n",
            "  Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for ffmpy: filename=ffmpy-0.3.0-py3-none-any.whl size=4694 sha256=e0f75e0ce315879294b4a8a335a2e96a814198f613ebbdd8e2f8b725640cd9b9\n",
            "  Stored in directory: /root/.cache/pip/wheels/0c/c2/0e/3b9c6845c6a4e35beb90910cc70d9ac9ab5d47402bd62af0df\n",
            "Successfully built llama-cpp-python peft transformers accelerate ffmpy\n",
            "Installing collected packages: tokenizers, sentencepiece, safetensors, pydub, pulp, ffmpy, bitsandbytes, xxhash, websockets, uc-micro-py, semantic-version, python-multipart, Pillow, orjson, multidict, markdown-it-py, llama-cpp-python, h11, frozenlist, einops, dill, colorama, async-timeout, aiofiles, yarl, uvicorn, starlette, multiprocess, mdit-py-plugins, linkify-it-py, huggingface-hub, httpcore, aiosignal, transformers, httpx, fastapi, aiohttp, gradio_client, gradio, datasets, accelerate, peft, flexgen\n",
            "  Attempting uninstall: Pillow\n",
            "    Found existing installation: Pillow 8.4.0\n",
            "    Uninstalling Pillow-8.4.0:\n",
            "      Successfully uninstalled Pillow-8.4.0\n",
            "  Attempting uninstall: markdown-it-py\n",
            "    Found existing installation: markdown-it-py 3.0.0\n",
            "    Uninstalling markdown-it-py-3.0.0:\n",
            "      Successfully uninstalled markdown-it-py-3.0.0\n",
            "Successfully installed Pillow-9.5.0 accelerate-0.20.0.dev0 aiofiles-23.1.0 aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 bitsandbytes-0.39.0 colorama-0.4.6 datasets-2.13.1 dill-0.3.6 einops-0.6.1 fastapi-0.98.0 ffmpy-0.3.0 flexgen-0.1.7 frozenlist-1.3.3 gradio-3.31.0 gradio_client-0.2.5 h11-0.14.0 httpcore-0.17.2 httpx-0.24.1 huggingface-hub-0.15.1 linkify-it-py-2.0.2 llama-cpp-python-0.1.56 markdown-it-py-2.2.0 mdit-py-plugins-0.3.3 multidict-6.0.4 multiprocess-0.70.14 orjson-3.9.1 peft-0.4.0.dev0 pulp-2.7.0 pydub-0.25.1 python-multipart-0.0.6 safetensors-0.3.1 semantic-version-2.10.0 sentencepiece-0.1.99 starlette-0.27.0 tokenizers-0.13.3 transformers-4.30.0.dev0 uc-micro-py-1.0.2 uvicorn-0.22.0 websockets-11.0.3 xxhash-3.2.0 yarl-1.9.2\n"
          ]
        },
        {
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "PIL"
                ]
              }
            }
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting gradio==3.32.0\n",
            "  Downloading gradio-3.32.0-py3-none-any.whl (19.9 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.9/19.9 MB\u001b[0m \u001b[31m51.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: aiofiles in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (23.1.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (3.8.4)\n",
            "Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (4.2.2)\n",
            "Requirement already satisfied: fastapi in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.98.0)\n",
            "Requirement already satisfied: ffmpy in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.3.0)\n",
            "Requirement already satisfied: gradio-client>=0.2.4 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.2.5)\n",
            "Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.24.1)\n",
            "Requirement already satisfied: huggingface-hub>=0.13.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.15.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (3.1.2)\n",
            "Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.2.0)\n",
            "Requirement already satisfied: markupsafe in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.1.3)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (3.7.1)\n",
            "Requirement already satisfied: mdit-py-plugins<=0.3.3 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.3.3)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (1.22.4)\n",
            "Requirement already satisfied: orjson in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (3.9.1)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (1.5.3)\n",
            "Requirement already satisfied: pillow in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (9.5.0)\n",
            "Requirement already satisfied: pydantic in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (1.10.9)\n",
            "Requirement already satisfied: pydub in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.25.1)\n",
            "Requirement already satisfied: pygments>=2.12.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.14.0)\n",
            "Requirement already satisfied: python-multipart in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.0.6)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (6.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.27.1)\n",
            "Requirement already satisfied: semantic-version in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.10.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (4.6.3)\n",
            "Requirement already satisfied: uvicorn>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.22.0)\n",
            "Requirement already satisfied: websockets>=10.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (11.0.3)\n",
            "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.32.0) (0.4)\n",
            "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.32.0) (4.3.3)\n",
            "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.32.0) (0.12.0)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.2.4->gradio==3.32.0) (2023.6.0)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.2.4->gradio==3.32.0) (23.1)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.0->gradio==3.32.0) (3.12.2)\n",
            "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.0->gradio==3.32.0) (4.65.0)\n",
            "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.32.0) (0.1.2)\n",
            "Requirement already satisfied: linkify-it-py<3,>=1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.32.0) (2.0.2)\n",
            "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->gradio==3.32.0) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->gradio==3.32.0) (2022.7.1)\n",
            "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.32.0) (8.1.3)\n",
            "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.32.0) (0.14.0)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (23.1.0)\n",
            "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (2.0.12)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (6.0.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (4.0.2)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (1.3.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (1.3.1)\n",
            "Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /usr/local/lib/python3.10/dist-packages (from fastapi->gradio==3.32.0) (0.27.0)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (2023.5.7)\n",
            "Requirement already satisfied: httpcore<0.18.0,>=0.15.0 in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (0.17.2)\n",
            "Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (3.4)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (1.3.0)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (1.1.0)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (0.11.0)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (4.40.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (1.4.4)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (3.1.0)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->gradio==3.32.0) (1.26.16)\n",
            "Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.10/dist-packages (from httpcore<0.18.0,>=0.15.0->httpx->gradio==3.32.0) (3.7.0)\n",
            "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.32.0) (0.19.3)\n",
            "Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.10/dist-packages (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio==3.32.0) (1.0.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->gradio==3.32.0) (1.16.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5.0,>=3.0->httpcore<0.18.0,>=0.15.0->httpx->gradio==3.32.0) (1.1.1)\n",
            "Installing collected packages: gradio\n",
            "  Attempting uninstall: gradio\n",
            "    Found existing installation: gradio 3.31.0\n",
            "    Uninstalling gradio-3.31.0:\n",
            "      Successfully uninstalled gradio-3.31.0\n",
            "Successfully installed gradio-3.32.0\n",
            "/content/text-generation-webui/repositories\n",
            "Cloning into 'GPTQ-for-LLaMa'...\n",
            "remote: Enumerating objects: 818, done.\u001b[K\n",
            "remote: Counting objects: 100% (818/818), done.\u001b[K\n",
            "remote: Compressing objects: 100% (323/323), done.\u001b[K\n",
            "remote: Total 818 (delta 496), reused 798 (delta 490), pack-reused 0\u001b[K\n",
            "Receiving objects: 100% (818/818), 471.90 KiB | 20.52 MiB/s, done.\n",
            "Resolving deltas: 100% (496/496), done.\n",
            "Note: switching to '53a2aeac935b7eef0e1451eb7b465fea4f318e7a'.\n",
            "\n",
            "You are in 'detached HEAD' state. You can look around, make experimental\n",
            "changes and commit them, and you can discard any commits you make in this\n",
            "state without impacting any branches by switching back to a branch.\n",
            "\n",
            "If you want to create a new branch to retain commits you create, you may\n",
            "do so (now or later) by using -c with the switch command. Example:\n",
            "\n",
            "  git switch -c <new-branch-name>\n",
            "\n",
            "Or undo this operation with:\n",
            "\n",
            "  git switch -\n",
            "\n",
            "Turn off this advice by setting config variable advice.detachedHead to false\n",
            "\n",
            "/content/text-generation-webui/repositories/GPTQ-for-LLaMa\n",
            "running install\n",
            "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.\n",
            "!!\n",
            "\n",
            "        ********************************************************************************\n",
            "        Please avoid running ``setup.py`` directly.\n",
            "        Instead, use pypa/build, pypa/installer, pypa/build or\n",
            "        other standards-based tools.\n",
            "\n",
            "        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.\n",
            "        ********************************************************************************\n",
            "\n",
            "!!\n",
            "  self.initialize_options()\n",
            "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.\n",
            "!!\n",
            "\n",
            "        ********************************************************************************\n",
            "        Please avoid running ``setup.py`` and ``easy_install``.\n",
            "        Instead, use pypa/build, pypa/installer, pypa/build or\n",
            "        other standards-based tools.\n",
            "\n",
            "        See https://github.com/pypa/setuptools/issues/917 for details.\n",
            "        ********************************************************************************\n",
            "\n",
            "!!\n",
            "  self.initialize_options()\n",
            "running bdist_egg\n",
            "running egg_info\n",
            "creating quant_cuda.egg-info\n",
            "writing quant_cuda.egg-info/PKG-INFO\n",
            "writing dependency_links to quant_cuda.egg-info/dependency_links.txt\n",
            "writing top-level names to quant_cuda.egg-info/top_level.txt\n",
            "writing manifest file 'quant_cuda.egg-info/SOURCES.txt'\n",
            "/usr/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.\n",
            "  warnings.warn(msg.format('we could not find ninja.'))\n",
            "reading manifest file 'quant_cuda.egg-info/SOURCES.txt'\n",
            "writing manifest file 'quant_cuda.egg-info/SOURCES.txt'\n",
            "installing library code to build/bdist.linux-x86_64/egg\n",
            "running install_lib\n",
            "running build_ext\n",
            "/usr/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py:398: UserWarning: There are no x86_64-linux-gnu-g++ version bounds defined for CUDA version 11.8\n",
            "  warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')\n",
            "building 'quant_cuda' extension\n",
            "creating build\n",
            "creating build/temp.linux-x86_64-cpython-310\n",
            "x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/local/lib/python3.10/dist-packages/torch/include -I/usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.10/dist-packages/torch/include/TH -I/usr/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c quant_cuda.cpp -o build/temp.linux-x86_64-cpython-310/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\\\"_gcc\\\" -DPYBIND11_STDLIB=\\\"_libstdcpp\\\" -DPYBIND11_BUILD_ABI=\\\"_cxxabi1011\\\" -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17\n",
            "/usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.10/dist-packages/torch/include -I/usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.10/dist-packages/torch/include/TH -I/usr/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c quant_cuda_kernel.cu -o build/temp.linux-x86_64-cpython-310/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\\\"_gcc\\\" -DPYBIND11_STDLIB=\\\"_libstdcpp\\\" -DPYBIND11_BUILD_ABI=\\\"_cxxabi1011\\\" -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17\n",
            "\u001b[01m\u001b[0m\u001b[01m/usr/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54)\u001b[0m: \u001b[01;35mwarning\u001b[0m #186-D: pointless comparison of unsigned integer with zero\n",
            "          detected during:\n",
            "            instantiation of \u001b[01m\"__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]\"\u001b[0m \u001b[32m\n",
            "(61): here\u001b[0m\n",
            "            instantiation of \u001b[01m\"__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]\"\u001b[0m \u001b[32m\n",
            "/usr/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here\u001b[0m\n",
            "\n",
            "\u001b[01m\u001b[0m\u001b[01m/usr/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54)\u001b[0m: \u001b[01;35mwarning\u001b[0m #186-D: pointless comparison of unsigned integer with zero\n",
            "          detected during:\n",
            "            instantiation of \u001b[01m\"__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]\"\u001b[0m \u001b[32m\n",
            "(61): here\u001b[0m\n",
            "            instantiation of \u001b[01m\"__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]\"\u001b[0m \u001b[32m\n",
            "/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here\u001b[0m\n",
            "\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:41:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kat::DeprecatedTypeProperties& at::Tensor::type() const\u001b[m\u001b[K’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                         \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:222:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  222 | \u001b[01;36m\u001b[K  De\u001b[m\u001b[KprecatedTypeProperties & type() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:163:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kc10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)\u001b[m\u001b[K’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/Dispatch.h:122:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  122 | \u001b[01;36m\u001b[Kinline at::\u001b[m\u001b[KScalarType scalar_type(const at::DeprecatedTypeProperties& t) {\n",
            "      | \u001b[01;36m\u001b[K^~~~~~~~~~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:163:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kc10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)\u001b[m\u001b[K’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/Dispatch.h:122:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  122 | \u001b[01;36m\u001b[Kinline at::\u001b[m\u001b[KScalarType scalar_type(const at::DeprecatedTypeProperties& t) {\n",
            "      | \u001b[01;36m\u001b[K^~~~~~~~~~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:1011:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:1032:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:1056:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:1083:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:1106:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:1999:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:2020:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:2043:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:2069:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:166:2092:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  166 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:41:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kat::DeprecatedTypeProperties& at::Tensor::type() const\u001b[m\u001b[K’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                         \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:222:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  222 | \u001b[01;36m\u001b[K  De\u001b[m\u001b[KprecatedTypeProperties & type() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:163:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kc10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)\u001b[m\u001b[K’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/Dispatch.h:122:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  122 | \u001b[01;36m\u001b[Kinline at::\u001b[m\u001b[KScalarType scalar_type(const at::DeprecatedTypeProperties& t) {\n",
            "      | \u001b[01;36m\u001b[K^~~~~~~~~~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:163:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kc10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)\u001b[m\u001b[K’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/Dispatch.h:122:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  122 | \u001b[01;36m\u001b[Kinline at::\u001b[m\u001b[KScalarType scalar_type(const at::DeprecatedTypeProperties& t) {\n",
            "      | \u001b[01;36m\u001b[K^~~~~~~~~~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:1011:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:1032:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:1056:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:1083:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:1106:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:1999:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:2020:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:2043:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:2069:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:261:2092:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  261 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:41:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kat::DeprecatedTypeProperties& at::Tensor::type() const\u001b[m\u001b[K’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                         \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:222:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  222 | \u001b[01;36m\u001b[K  De\u001b[m\u001b[KprecatedTypeProperties & type() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:163:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kc10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)\u001b[m\u001b[K’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/Dispatch.h:122:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  122 | \u001b[01;36m\u001b[Kinline at::\u001b[m\u001b[KScalarType scalar_type(const at::DeprecatedTypeProperties& t) {\n",
            "      | \u001b[01;36m\u001b[K^~~~~~~~~~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:163:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kc10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)\u001b[m\u001b[K’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/Dispatch.h:122:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  122 | \u001b[01;36m\u001b[Kinline at::\u001b[m\u001b[KScalarType scalar_type(const at::DeprecatedTypeProperties& t) {\n",
            "      | \u001b[01;36m\u001b[K^~~~~~~~~~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:1011:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:1032:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:1056:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:1083:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:1106:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:1999:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:2020:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:2043:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:2069:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:420:2092:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  420 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:41:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kat::DeprecatedTypeProperties& at::Tensor::type() const\u001b[m\u001b[K’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                         \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:222:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  222 | \u001b[01;36m\u001b[K  De\u001b[m\u001b[KprecatedTypeProperties & type() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:163:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kc10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)\u001b[m\u001b[K’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/Dispatch.h:122:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  122 | \u001b[01;36m\u001b[Kinline at::\u001b[m\u001b[KScalarType scalar_type(const at::DeprecatedTypeProperties& t) {\n",
            "      | \u001b[01;36m\u001b[K^~~~~~~~~~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:163:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[Kc10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)\u001b[m\u001b[K’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/Dispatch.h:122:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  122 | \u001b[01;36m\u001b[Kinline at::\u001b[m\u001b[KScalarType scalar_type(const at::DeprecatedTypeProperties& t) {\n",
            "      | \u001b[01;36m\u001b[K^~~~~~~~~~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:1011:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:1032:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:1056:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:1083:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = double]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:1106:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:\u001b[m\u001b[K In lambda function:\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:1999:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:2020:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:2043:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:2069:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = float]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[Kquant_cuda_kernel.cu:507:2092:\u001b[m\u001b[K \u001b[01;35m\u001b[Kwarning: \u001b[m\u001b[K‘\u001b[01m\u001b[KT* at::Tensor::data() const [with T = int]\u001b[m\u001b[K’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [\u001b[01;35m\u001b[K-Wdeprecated-declarations\u001b[m\u001b[K]\n",
            "  507 |   AT_DISPATCH_FLOATING_TYPES(\n",
            "      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            \u001b[01;35m\u001b[K^\u001b[m\u001b[K\n",
            "\u001b[01m\u001b[K/usr/local/lib/python3.10/dist-packages/torch/include/ATen/core/TensorBody.h:244:1:\u001b[m\u001b[K \u001b[01;36m\u001b[Knote: \u001b[m\u001b[Kdeclared here\n",
            "  244 | \u001b[01;36m\u001b[K  T \u001b[m\u001b[K* data() const {\n",
            "      | \u001b[01;36m\u001b[K^\u001b[m\u001b[K \u001b[01;36m\u001b[K~~\u001b[m\u001b[K\n",
            "creating build/lib.linux-x86_64-cpython-310\n",
            "x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 build/temp.linux-x86_64-cpython-310/quant_cuda.o build/temp.linux-x86_64-cpython-310/quant_cuda_kernel.o -L/usr/local/lib/python3.10/dist-packages/torch/lib -L/usr/local/cuda/lib64 -L/usr/lib/x86_64-linux-gnu -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-310/quant_cuda.cpython-310-x86_64-linux-gnu.so\n",
            "creating build/bdist.linux-x86_64\n",
            "creating build/bdist.linux-x86_64/egg\n",
            "copying build/lib.linux-x86_64-cpython-310/quant_cuda.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg\n",
            "creating stub loader for quant_cuda.cpython-310-x86_64-linux-gnu.so\n",
            "byte-compiling build/bdist.linux-x86_64/egg/quant_cuda.py to quant_cuda.cpython-310.pyc\n",
            "creating build/bdist.linux-x86_64/egg/EGG-INFO\n",
            "copying quant_cuda.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO\n",
            "copying quant_cuda.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\n",
            "copying quant_cuda.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\n",
            "copying quant_cuda.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\n",
            "writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt\n",
            "zip_safe flag not set; analyzing archive contents...\n",
            "__pycache__.quant_cuda.cpython-310: module references __file__\n",
            "creating dist\n",
            "creating 'dist/quant_cuda-0.0.0-py3.10-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it\n",
            "removing 'build/bdist.linux-x86_64/egg' (and everything under it)\n",
            "Processing quant_cuda-0.0.0-py3.10-linux-x86_64.egg\n",
            "creating /usr/local/lib/python3.10/dist-packages/quant_cuda-0.0.0-py3.10-linux-x86_64.egg\n",
            "Extracting quant_cuda-0.0.0-py3.10-linux-x86_64.egg to /usr/local/lib/python3.10/dist-packages\n",
            "Adding quant-cuda 0.0.0 to easy-install.pth file\n",
            "\n",
            "Installed /usr/local/lib/python3.10/dist-packages/quant_cuda-0.0.0-py3.10-linux-x86_64.egg\n",
            "Processing dependencies for quant-cuda==0.0.0\n",
            "Finished processing dependencies for quant-cuda==0.0.0\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "26164b|\u001b[1;32mOK\u001b[0m  |   219KiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/config.json\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "8ae5b8|\u001b[1;32mOK\u001b[0m  |    28KiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/generation_config.json\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "b6b0e6|\u001b[1;32mOK\u001b[0m  |   129KiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/special_tokens_map.json\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "72c388|\u001b[1;32mOK\u001b[0m  |   103MiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/tokenizer.json\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "632e84|\u001b[1;32mOK\u001b[0m  |   165KiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/tokenizer_config.json\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\u001b[0m\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "29b1ab|\u001b[1;32mOK\u001b[0m  |   187MiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/gptq_model-4bit-128g.safetensors\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "9aaaf3|\u001b[1;32mOK\u001b[0m  |     226B/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/quantize_config.json\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "d75c3d|\u001b[1;32mOK\u001b[0m  |    61MiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/vocab.json\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "394eef|\u001b[1;32mOK\u001b[0m  |    30MiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/merges.txt\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "\n",
            "Download Results:\n",
            "gid   |stat|avg speed  |path/URI\n",
            "======+====+===========+=======================================================\n",
            "599fcc|\u001b[1;32mOK\u001b[0m  |    11MiB/s|/content/text-generation-webui/models/minotaur-15B-GPTQ/orig.pytorch_model.bin.index.json\n",
            "\n",
            "Status Legend:\n",
            "(OK):download completed.\n",
            "/content/text-generation-webui\n",
            "WARNING:\u001b[33mThe gradio \"share link\" feature uses a proprietary executable to create a reverse tunnel. Use it with care.\u001b[0m\n",
            "2023-06-26 09:55:25.619808: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
            "bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so\n",
            "INFO:\u001b[32mLoading minotaur-15B-GPTQ...\u001b[0m\n",
            "INFO:\u001b[32mFound the following quantized model: models/minotaur-15B-GPTQ/gptq_model-4bit-128g.safetensors\u001b[0m\n",
            "INFO:\u001b[32mLoaded the model in 55.20 seconds.\n",
            "\u001b[0m\n",
            "INFO:\u001b[32mLoading the extension \"gallery\"...\u001b[0m\n",
            "Running on local URL:  http://127.0.0.1:7860\n",
            "Running on public URL: https://de78c02a71f097f30f.gradio.live\n",
            "\n",
            "This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces\n",
            "Output generated in 15.98 seconds (1.38 tokens/s, 22 tokens, context 28, seed 1116291238)\n",
            "Output generated in 38.33 seconds (3.29 tokens/s, 126 tokens, context 77, seed 164990070)\n",
            "Output generated in 101.62 seconds (5.03 tokens/s, 511 tokens, context 56, seed 67805345)\n",
            "Traceback (most recent call last):\n",
            "  File \"/content/text-generation-webui/modules/callbacks.py\", line 73, in gentask\n",
            "    ret = self.mfunc(callback=_callback, **self.kwargs)\n",
            "  File \"/content/text-generation-webui/modules/text_generation.py\", line 286, in generate_with_callback\n",
            "    shared.model.generate(**kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py\", line 115, in decorate_context\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\", line 1568, in generate\n",
            "    return self.sample(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\", line 2615, in sample\n",
            "    outputs = self(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 808, in forward\n",
            "    transformer_outputs = self.transformer(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 673, in forward\n",
            "    outputs = block(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 352, in forward\n",
            "    hidden_states = self.ln_2(hidden_states)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/normalization.py\", line 190, in forward\n",
            "    return F.layer_norm(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py\", line 2515, in layer_norm\n",
            "    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)\n",
            "RuntimeError: expected scalar type Float but found Half\n",
            "Output generated in 1.09 seconds (0.00 tokens/s, 0 tokens, context 567, seed 1918367810)\n",
            "Traceback (most recent call last):\n",
            "  File \"/content/text-generation-webui/modules/callbacks.py\", line 73, in gentask\n",
            "    ret = self.mfunc(callback=_callback, **self.kwargs)\n",
            "  File \"/content/text-generation-webui/modules/text_generation.py\", line 286, in generate_with_callback\n",
            "    shared.model.generate(**kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py\", line 115, in decorate_context\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\", line 1568, in generate\n",
            "    return self.sample(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\", line 2615, in sample\n",
            "    outputs = self(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 808, in forward\n",
            "    transformer_outputs = self.transformer(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 673, in forward\n",
            "    outputs = block(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 352, in forward\n",
            "    hidden_states = self.ln_2(hidden_states)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/normalization.py\", line 190, in forward\n",
            "    return F.layer_norm(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py\", line 2515, in layer_norm\n",
            "    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)\n",
            "RuntimeError: expected scalar type Float but found Half\n",
            "Output generated in 0.78 seconds (0.00 tokens/s, 0 tokens, context 567, seed 978710917)\n",
            "Traceback (most recent call last):\n",
            "  File \"/content/text-generation-webui/modules/callbacks.py\", line 73, in gentask\n",
            "    ret = self.mfunc(callback=_callback, **self.kwargs)\n",
            "  File \"/content/text-generation-webui/modules/text_generation.py\", line 286, in generate_with_callback\n",
            "    shared.model.generate(**kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py\", line 115, in decorate_context\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\", line 1568, in generate\n",
            "    return self.sample(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\", line 2615, in sample\n",
            "    outputs = self(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 808, in forward\n",
            "    transformer_outputs = self.transformer(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 673, in forward\n",
            "    outputs = block(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py\", line 352, in forward\n",
            "    hidden_states = self.ln_2(hidden_states)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\", line 1501, in _call_impl\n",
            "    return forward_call(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/modules/normalization.py\", line 190, in forward\n",
            "    return F.layer_norm(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py\", line 2515, in layer_norm\n",
            "    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)\n",
            "RuntimeError: expected scalar type Float but found Half\n",
            "Output generated in 0.79 seconds (0.00 tokens/s, 0 tokens, context 567, seed 2084806834)\n",
            "Output generated in 36.14 seconds (10.82 tokens/s, 391 tokens, context 48, seed 2067062755)\n",
            "Output generated in 11.73 seconds (7.76 tokens/s, 91 tokens, context 48, seed 1980913369)\n",
            "Output generated in 16.69 seconds (9.59 tokens/s, 160 tokens, context 48, seed 2141649440)\n",
            "Output generated in 8.19 seconds (6.59 tokens/s, 54 tokens, context 39, seed 923472076)\n",
            "Output generated in 11.24 seconds (8.10 tokens/s, 91 tokens, context 48, seed 495600492)\n",
            "Output generated in 4.32 seconds (0.92 tokens/s, 4 tokens, context 48, seed 309890525)\n",
            "Output generated in 3.94 seconds (0.76 tokens/s, 3 tokens, context 48, seed 1872802022)\n",
            "Output generated in 6.11 seconds (2.95 tokens/s, 18 tokens, context 47, seed 619958206)\n",
            "Output generated in 5.08 seconds (2.76 tokens/s, 14 tokens, context 48, seed 457015875)\n",
            "Output generated in 4.79 seconds (2.09 tokens/s, 10 tokens, context 48, seed 530163302)\n",
            "Output generated in 9.57 seconds (7.52 tokens/s, 72 tokens, context 48, seed 147503791)\n",
            "Output generated in 7.55 seconds (6.23 tokens/s, 47 tokens, context 48, seed 884159242)\n",
            "Output generated in 4.83 seconds (0.83 tokens/s, 4 tokens, context 46, seed 1836938857)\n",
            "Output generated in 5.35 seconds (3.74 tokens/s, 20 tokens, context 48, seed 1337594978)\n",
            "Output generated in 6.96 seconds (6.61 tokens/s, 46 tokens, context 48, seed 161825219)\n"
          ]
        }
      ],
      "source": [
        "%cd /content\n",
        "!apt-get -y install -qq aria2\n",
        "\n",
        "!git clone -b V1.1 https://github.com/Troyanovsky/text-generation-webui\n",
        "%cd /content/text-generation-webui\n",
        "!pip install -r requirements.txt\n",
        "!pip install -U gradio==3.32.0\n",
        "\n",
        "!mkdir /content/text-generation-webui/repositories\n",
        "%cd /content/text-generation-webui/repositories\n",
        "!git clone -b V1.0 https://github.com/Troyanovsky/GPTQ-for-LLaMa\n",
        "%cd GPTQ-for-LLaMa\n",
        "!python setup_cuda.py install\n",
        "\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/config.json -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o config.json\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/generation_config.json -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o generation_config.json\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/special_tokens_map.json -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o special_tokens_map.json\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/tokenizer.json -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o tokenizer.json\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/tokenizer_config.json -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o tokenizer_config.json\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/gptq_model-4bit-128g.safetensors -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o gptq_model-4bit-128g.safetensors\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/quantize_config.json -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o quantize_config.json\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/vocab.json -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o vocab.json\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/merges.txt -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o merges.txt\n",
        "!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/TheBloke/minotaur-15B-GPTQ/resolve/main/orig.pytorch_model.bin.index.json -d /content/text-generation-webui/models/minotaur-15B-GPTQ -o orig.pytorch_model.bin.index.json\n",
        "\n",
        "%cd /content/text-generation-webui\n",
        "!python server.py --share --chat --wbits 4 --groupsize 128 --model_type llama\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "authorship_tag": "ABX9TyNA8q09R4S2bFHYCs5Tdp+4",
      "include_colab_link": true,
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}