Bielik_2_(AWQ)_structured_output.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "83f31ef9",
      "metadata": {
        "id": "83f31ef9"
      },
      "source": [
        "<p align=\"center\">\n",
        "  <img src=\"https://huggingface.co/speakleash/Bielik-7B-Instruct-v0.1/raw/main/speakleash_cyfronet.png\">\n",
        "</p>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aced9a46",
      "metadata": {
        "id": "aced9a46"
      },
      "source": [
        "# Bielik Structured output vLLM + Outlines\n",
        "### Po co structured output:\n",
        "\n",
        "Pozwala na uzyskanie ustrukturyzowanych danych z modelu LLM, co ułatwia ich przetwarzanie i integrację z innymi systemami.\n",
        "\n",
        "Jest kilka rozwiązań które pozwalają to uzyskać jak np:\n",
        "- [llama.cpp](https://github.com/ggerganov/llama.cpp) i [grammar](https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md)\n",
        "- [Instructor](https://python.useinstructor.com/).\n",
        "\n",
        "Dojrzałe produkcyjnie rozwiązanie to [vLLM](https://github.com/vllm-project/vllm) + [Outlines](https://dottxt-ai.github.io/outlines/).\n",
        "\n",
        "### Jak działa Outlines w skrócie\n",
        "Outlines wykorzystuje [automaty](https://dottxt-ai.github.io/outlines/reference/generation/structured_generation_explanation/) do generowania tekstu w oparciu o zdefiniowane wzorce.\n",
        "\n",
        "Proces polega na tym, że model językowy generuje tekst token po tokenie, ale tylko legalne tokeny (zgodne ze wzorcem) są brane pod uwagę na każdym kroku.\n",
        "\n",
        "Przykładowo, jeśli wzorzec ma opisywać liczby całkowite i dziesiętne, automaty przeprowadzają analizę, jakie tokeny (cyfry, kropki, itp.) są dozwolone w danym momencie.\n",
        "\n",
        "Outlines modyfikuje prawdopodobieństwa tokenów, eliminując te niezgodne z wzorcem, co zapewnia precyzyjne i kontrolowane generowanie tekstu przez model.\n",
        "\n",
        "### Wymagania do uruchomienia tego notebooka:\n",
        "GPU min 24 GB vRam - np L4 na Google Colab Pro albo [lightning.ai](https://lightning.ai/)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b5af344a-aed1-4003-b8e9-4055b5e814fb",
      "metadata": {
        "scrolled": true,
        "id": "b5af344a-aed1-4003-b8e9-4055b5e814fb"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "!pip install vllm outlines"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e4cee777-145e-425a-9af4-f4721b8990b1",
      "metadata": {
        "id": "e4cee777-145e-425a-9af4-f4721b8990b1"
      },
      "outputs": [],
      "source": [
        "from enum import Enum\n",
        "from typing import List, Optional, Union\n",
        "\n",
        "from IPython.display import display, Markdown\n",
        "from huggingface_hub import notebook_login\n",
        "from outlines import generate, models\n",
        "from pydantic import BaseModel, Field, constr\n",
        "from transformers import AutoTokenizer"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "72b171e1",
      "metadata": {
        "id": "72b171e1"
      },
      "source": [
        "### Do pobrania modelu z Hugging Face potrzebny jest [token](https://huggingface.co/docs/hub/en/security-tokens)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bce37fe7-36cd-4515-8212-227ca9319670",
      "metadata": {
        "colab": {
          "referenced_widgets": [
            "cda91c6f43d44ec988a7fbf693515022"
          ]
        },
        "id": "bce37fe7-36cd-4515-8212-227ca9319670",
        "outputId": "acc05637-8b69-47ac-adbb-646c52a2f51d"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "cda91c6f43d44ec988a7fbf693515022",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "notebook_login()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7a39445c",
      "metadata": {
        "id": "7a39445c"
      },
      "outputs": [],
      "source": [
        "tokenizer = AutoTokenizer.from_pretrained(\"speakleash/Bielik-11B-v2.2-Instruct\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8042dbbb",
      "metadata": {
        "id": "8042dbbb"
      },
      "source": [
        "### Na potrzeby demo użyjemy kwantyfikowany model AWQ"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "52bcef9c-3d1d-40cb-a916-c5fbcc3bfd82",
      "metadata": {
        "colab": {
          "referenced_widgets": [
            "2aed80e2939c48fab80fa385032a314b"
          ]
        },
        "id": "52bcef9c-3d1d-40cb-a916-c5fbcc3bfd82",
        "outputId": "53dcb6c6-be2c-45f8-df0f-ca456ea4044c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "INFO 09-26 09:07:42 awq_marlin.py:93] Detected that the model can run with awq_marlin, however you specified quantization=awq explicitly, so forcing awq. Use quantization=awq_marlin for faster inference\n",
            "WARNING 09-26 09:07:42 config.py:330] awq quantization is not fully optimized yet. The speed can be slower than non-quantized models.\n",
            "INFO 09-26 09:07:42 llm_engine.py:213] Initializing an LLM engine (v0.6.0) with config: model='speakleash/Bielik-11B-v2.2-Instruct-AWQ', speculative_config=None, tokenizer='speakleash/Bielik-11B-v2.2-Instruct-AWQ', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=awq, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=speakleash/Bielik-11B-v2.2-Instruct-AWQ, use_v2_block_manager=False, num_scheduler_steps=1, enable_prefix_caching=False, use_async_output_proc=True)\n",
            "INFO 09-26 09:07:43 model_runner.py:915] Starting to load model speakleash/Bielik-11B-v2.2-Instruct-AWQ...\n",
            "INFO 09-26 09:07:44 weight_utils.py:236] Using model weights format ['*.safetensors']\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "2aed80e2939c48fab80fa385032a314b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Loading safetensors checkpoint shards:   0% Completed | 0/2 [00:00<?, ?it/s]\n"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "INFO 09-26 09:09:20 model_runner.py:926] Loading model weights took 5.7895 GB\n",
            "INFO 09-26 09:09:34 gpu_executor.py:122] # GPU blocks: 3292, # CPU blocks: 1310\n",
            "INFO 09-26 09:09:41 model_runner.py:1217] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.\n",
            "INFO 09-26 09:09:41 model_runner.py:1221] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing `gpu_memory_utilization` or enforcing eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.\n",
            "INFO 09-26 09:10:17 model_runner.py:1335] Graph capturing finished in 36 secs.\n"
          ]
        }
      ],
      "source": [
        "model = models.vllm(\"speakleash/Bielik-11B-v2.2-Instruct-AWQ\", quantization=\"awq\")\n",
        "# model = models.vllm(\"speakleash/Bielik-11B-v2.2-Instruct\")  # use A100 GPU at least to load full model"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f2ed9ec7",
      "metadata": {
        "id": "f2ed9ec7"
      },
      "source": [
        "### Tekst na którym będziemy testować ekstrakcję treści do formatu JSON"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c627d649",
      "metadata": {
        "id": "c627d649"
      },
      "outputs": [],
      "source": [
        "text = \"\"\"\n",
        "Każdy z hobbitów miał plecak, a w nim najpotrzebniejsze rzeczy.\n",
        "Frodo niósł trzy zapasowe koszule, dwa płaszcze i parę koców.\n",
        "Sam miał cztery bochenki chleba, sześć kawałków suszonego mięsa i kilka ciastek,\n",
        "a Merry wziął bukłak z wodą, dwie latarnie i dziesięć świec.\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c449bfb6",
      "metadata": {
        "id": "c449bfb6"
      },
      "source": [
        "### Pydantic i definicja struktury którą użyje model do odpowiedzi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "93ecaef3-7d5f-4c6b-a950-af22fb09c4db",
      "metadata": {
        "id": "93ecaef3-7d5f-4c6b-a950-af22fb09c4db"
      },
      "outputs": [],
      "source": [
        "class CharactersNames(Enum):\n",
        "    Frodo = \"Frodo\"\n",
        "    Sam = \"Sam\"\n",
        "    Merry = \"Merry\"\n",
        "\n",
        "\n",
        "class Item(BaseModel):\n",
        "    name: str\n",
        "    quantity: Optional[Union[int, str]]\n",
        "\n",
        "\n",
        "class Character(BaseModel):\n",
        "    name: CharactersNames\n",
        "    items: List[Item]\n",
        "\n",
        "\n",
        "class Company(BaseModel):\n",
        "    characters: List[Character]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5fd5a27b",
      "metadata": {
        "id": "5fd5a27b"
      },
      "outputs": [],
      "source": [
        "schema = Company.model_json_schema()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "692b32a2",
      "metadata": {
        "id": "692b32a2"
      },
      "source": [
        "### Generator korzystający ze schema z Pydantic"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "414f5a4c",
      "metadata": {
        "id": "414f5a4c",
        "outputId": "53598695-c389-4f4a-bbff-abb083e4f36f"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Compiling FSM index for all state transitions: 100%|██████████| 119/119 [00:00<00:00, 197.12it/s]\n"
          ]
        }
      ],
      "source": [
        "# generator with structured output\n",
        "generator = generate.json(model, Company)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "93723ff1",
      "metadata": {
        "id": "93723ff1"
      },
      "source": [
        "### Dla porównania spróbujemy zmusić model do odpowiadania w formacie JSON także samym promptem"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "69d93092",
      "metadata": {
        "id": "69d93092"
      },
      "outputs": [],
      "source": [
        "# generator controiled with prompting only\n",
        "generator_unstructured = generate.text(model)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6f52f9f9",
      "metadata": {
        "id": "6f52f9f9"
      },
      "source": [
        "### Prompt + JSON schema"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dd94752b",
      "metadata": {
        "id": "dd94752b"
      },
      "outputs": [],
      "source": [
        "chat = [\n",
        "    {\"role\": \"system\", \"content\": f\"\"\"\n",
        "Jesteś modelem AI odpowiającym na pytania używając tylko formatu JSON\n",
        "\n",
        "Zastosuj się do podanego JSON schema:\\n{Company.schema_json()}\"\"\"},\n",
        "    {\"role\": \"user\", \"content\": f\"{text}\"},\n",
        "]\n",
        "\n",
        "prompt = f\"{tokenizer.apply_chat_template(chat, tokenize=False)}<|im_start|>assistant\\n<schema>\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e4773931",
      "metadata": {
        "id": "e4773931",
        "outputId": "31aade35-81fe-4d28-9c40-138fd79a1e3c"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "### Cały prompt po użyciu tokenizera i templatki chatu wygląda tak:\n",
              "---\n",
              " ```text\n",
              "<s><|im_start|>system\n",
              "\n",
              "Jesteś modelem AI odpowiającym na pytania używając tylko formatu JSON\n",
              "\n",
              "Zastosuj się do podanego JSON schema:\n",
              "{\"$defs\": {\"Character\": {\"properties\": {\"name\": {\"$ref\": \"#/$defs/CharactersNames\"}, \"items\": {\"items\": {\"$ref\": \"#/$defs/Item\"}, \"title\": \"Items\", \"type\": \"array\"}}, \"required\": [\"name\", \"items\"], \"title\": \"Character\", \"type\": \"object\"}, \"CharactersNames\": {\"enum\": [\"Frodo\", \"Sam\", \"Merry\"], \"title\": \"CharactersNames\", \"type\": \"string\"}, \"Item\": {\"properties\": {\"name\": {\"title\": \"Name\", \"type\": \"string\"}, \"quantity\": {\"anyOf\": [{\"type\": \"integer\"}, {\"type\": \"string\"}, {\"type\": \"null\"}], \"title\": \"Quantity\"}}, \"required\": [\"name\", \"quantity\"], \"title\": \"Item\", \"type\": \"object\"}}, \"properties\": {\"characters\": {\"items\": {\"$ref\": \"#/$defs/Character\"}, \"title\": \"Characters\", \"type\": \"array\"}}, \"required\": [\"characters\"], \"title\": \"Company\", \"type\": \"object\"}<|im_end|>\n",
              "<|im_start|>user\n",
              "\n",
              "Każdy z hobbitów miał plecak, a w nim najpotrzebniejsze rzeczy. \n",
              "Frodo niósł trzy zapasowe koszule, dwa płaszcze i parę koców. \n",
              "Sam miał cztery bochenki chleba, sześć kawałków suszonego mięsa i kilka ciastek, \n",
              "a Merry wziął bukłak z wodą, dwie latarnie i dziesięć świec. \n",
              "<|im_end|>\n",
              "<|im_start|>assistant\n",
              "<schema>\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "markdown_content = f'### Cały prompt po użyciu tokenizera i templatki chatu wygląda tak:\\n---\\n ```text\\n{prompt}\\n```'\n",
        "\n",
        "# Display the Markdown content\n",
        "display(Markdown(markdown_content))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e57af238",
      "metadata": {
        "id": "e57af238"
      },
      "source": [
        "### Do próby zmuszenia modelu do odpowiedzenia w takim formacie jak chcemy bez Outlines użyjemy inny prompt, z bardziej szczegółową instrukcją"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c58d0be0",
      "metadata": {
        "id": "c58d0be0"
      },
      "outputs": [],
      "source": [
        "chat_unstructured = [\n",
        "  {\"role\": \"system\", \"content\": \"\"\"\n",
        "# ZADANIE:\n",
        "Z danego tekstu wybierz postacie i ich ekwipunek.\n",
        "## Przykład:\n",
        "### Tekst\n",
        "Bilbo miał dwa jabłka a Frodo miecz.\n",
        "### Odpowiedź\n",
        "```\n",
        "{\n",
        "    \"characters\": [\n",
        "        {\n",
        "            \"name\": \"Bilbo\",\n",
        "            \"items\": [\n",
        "                {\n",
        "                    \"name\": \"jabłko\",\n",
        "                    \"quantity\": 2\n",
        "                }\n",
        "            ]\n",
        "        },\n",
        "        {\n",
        "            \"name\": \"Frodo\",\n",
        "            \"items\": [\n",
        "                {\n",
        "                    \"name\": \"miecz\",\n",
        "                    \"quantity\": 1\n",
        "                }\n",
        "            ]\n",
        "        },\n",
        "    ]\n",
        "}\n",
        "```\n",
        "\n",
        "Odpowiadaj tylko w formacie JSON, nie dodawaj nic więcej.\"\"\"},\n",
        "  {\"role\": \"user\", \"content\": f\"# TEKST: \\n{ text }\"},\n",
        "]\n",
        "\n",
        "prompt_unstructured = f\"{tokenizer.apply_chat_template(chat_unstructured, tokenize=False)}<|im_start|>assistant\\n# JSON:\""
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f9082c9f",
      "metadata": {
        "id": "f9082c9f"
      },
      "source": [
        "### Dla porównania wygenerujemy rezultat bez narzucania struktury outputu"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8c5f72b9",
      "metadata": {
        "id": "8c5f72b9",
        "outputId": "104f7d37-cd8a-4b4c-801c-acd286d198a9"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Processed prompts: 100%|██████████| 1/1 [00:11<00:00, 11.85s/it, est. speed input: 31.73 toks/s, output: 32.48 toks/s]\n"
          ]
        }
      ],
      "source": [
        "response_unstructured = generator_unstructured(prompt_unstructured, max_tokens=1024)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bef63d83",
      "metadata": {
        "id": "bef63d83",
        "outputId": "5d048f36-ce65-4bd4-d94b-5d72fba1faf6"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "### Response bez wymuszenia struktury:\n",
              "---\n",
              "\n",
              "\n",
              "```json\n",
              "{\n",
              "  \"characters\": [\n",
              "    {\n",
              "      \"name\": \"Każdy z hobbitów\",\n",
              "      \"items\": [\n",
              "        {\n",
              "          \"name\": \"plecak\",\n",
              "          \"quantity\": 1\n",
              "        }\n",
              "      ]\n",
              "    },\n",
              "    {\n",
              "      \"name\": \"Frodo\",\n",
              "      \"items\": [\n",
              "        {\n",
              "          \"name\": \"zapasowe koszule\",\n",
              "          \"quantity\": 3\n",
              "        },\n",
              "        {\n",
              "          \"name\": \"płaszcze\",\n",
              "          \"quantity\": 2\n",
              "        },\n",
              "        {\n",
              "          \"name\": \"koce\",\n",
              "          \"quantity\": 1\n",
              "        }\n",
              "      ]\n",
              "    },\n",
              "    {\n",
              "      \"name\": \"Sam\",\n",
              "      \"items\": [\n",
              "        {\n",
              "          \"name\": \"bochenki chleba\",\n",
              "          \"quantity\": 4\n",
              "        },\n",
              "        {\n",
              "          \"name\": \"suszonego mięsa\",\n",
              "          \"quantity\": 6\n",
              "        },\n",
              "        {\n",
              "          \"name\": \"ciastka\",\n",
              "          \"quantity\": 1\n",
              "        }\n",
              "      ]\n",
              "    },\n",
              "    {\n",
              "      \"name\": \"Merry\",\n",
              "      \"items\": [\n",
              "        {\n",
              "          \"name\": \"bukłak z wodą\",\n",
              "          \"quantity\": 1\n",
              "        },\n",
              "        {\n",
              "          \"name\": \"latarnie\",\n",
              "          \"quantity\": 2\n",
              "        },\n",
              "        {\n",
              "          \"name\": \"świece\",\n",
              "          \"quantity\": 10\n",
              "        }\n",
              "      ]\n",
              "    }\n",
              "  ]\n",
              "}\n",
              "```\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "markdown_content = f'### Response bez wymuszenia struktury:\\n---\\n{response_unstructured}\\n'\n",
        "\n",
        "display(Markdown(markdown_content))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b6077477",
      "metadata": {
        "id": "b6077477"
      },
      "source": [
        "O ile da się zmusić model do odpowiedzi, nie jest to deterministyczny mechanizm i output nie jest gwarantowany.  \n",
        "\n",
        "Trudno go wykorzystać w produkcyjnych warunkach.\n",
        "\n",
        "Model dodaje tez czasem tekst poza odpowiedzią w formacie JSON"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5d5a20fa",
      "metadata": {
        "id": "5d5a20fa"
      },
      "source": [
        "### I kontrolując strukturę przez Outlines i Pydantic"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9927d931",
      "metadata": {
        "id": "9927d931",
        "outputId": "e9c39de8-32c2-4028-d443-22570d7f762e"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Processed prompts: 100%|██████████| 1/1 [00:06<00:00,  6.15s/it, est. speed input: 71.66 toks/s, output: 31.36 toks/s]\n"
          ]
        }
      ],
      "source": [
        "response = generator(prompt, max_tokens=1024)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a878eb64",
      "metadata": {
        "id": "a878eb64",
        "outputId": "d3cdbdd3-4082-4403-b049-c3489d622ecc"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "### Response gdy output ma wymuszoną strukturę, obiekt Pydantic po konwersji do JSON:\n",
              "---\n",
              "```json\n",
              "{\n",
              "    \"characters\": [\n",
              "        {\n",
              "            \"name\": \"Frodo\",\n",
              "            \"items\": [\n",
              "                {\n",
              "                    \"name\": \"koszula\",\n",
              "                    \"quantity\": 3\n",
              "                },\n",
              "                {\n",
              "                    \"name\": \"płaszcz\",\n",
              "                    \"quantity\": 2\n",
              "                },\n",
              "                {\n",
              "                    \"name\": \"koce\",\n",
              "                    \"quantity\": 1\n",
              "                }\n",
              "            ]\n",
              "        },\n",
              "        {\n",
              "            \"name\": \"Sam\",\n",
              "            \"items\": [\n",
              "                {\n",
              "                    \"name\": \"bochenki chleba\",\n",
              "                    \"quantity\": 4\n",
              "                },\n",
              "                {\n",
              "                    \"name\": \"suszone mięso\",\n",
              "                    \"quantity\": 6\n",
              "                },\n",
              "                {\n",
              "                    \"name\": \"ciastka\",\n",
              "                    \"quantity\": \"kilka\"\n",
              "                }\n",
              "            ]\n",
              "        },\n",
              "        {\n",
              "            \"name\": \"Merry\",\n",
              "            \"items\": [\n",
              "                {\n",
              "                    \"name\": \"bukłak z wodą\",\n",
              "                    \"quantity\": 1\n",
              "                },\n",
              "                {\n",
              "                    \"name\": \"latarnie\",\n",
              "                    \"quantity\": 2\n",
              "                },\n",
              "                {\n",
              "                    \"name\": \"świece\",\n",
              "                    \"quantity\": 10\n",
              "                }\n",
              "            ]\n",
              "        }\n",
              "    ]\n",
              "}\n",
              "```"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "json_str = response.model_dump_json(indent=4)\n",
        "\n",
        "markdown_content = f'### Response gdy output ma wymuszoną strukturę, obiekt Pydantic po konwersji do JSON:\\n---\\n```json\\n{json_str}\\n```'\n",
        "\n",
        "display(Markdown(markdown_content))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "57694ef1",
      "metadata": {
        "id": "57694ef1"
      },
      "source": [
        "### Z Outlines rezultat ma gwarantowaną strukturę"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.10"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}