python/notebooks/10-multiple-results-per-prompt.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "68e1c158",
   "metadata": {},
   "source": [
    "# Multiple Results"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fb81bacd",
   "metadata": {},
   "source": [
    "In this notebook we show how you can in a single request, have the LLM model return multiple results per prompt. This is useful for running experiments where you want to evaluate the robustness of your prompt and the parameters of your config against a particular large language model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a77bdf89",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -m pip install semantic-kernel==0.3.10.dev0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "508ad44f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import semantic_kernel as sk\n",
    "from semantic_kernel.connectors.ai import ChatRequestSettings, CompleteRequestSettings\n",
    "from semantic_kernel.connectors.ai.open_ai import AzureTextCompletion, AzureChatCompletion, OpenAITextCompletion, OpenAIChatCompletion\n",
    "from semantic_kernel.connectors.ai.hugging_face import HuggingFaceTextCompletion"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d8ddffc1",
   "metadata": {},
   "source": [
    "First, we will set up the text and chat services we will be submitting prompts to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f8dcbc6",
   "metadata": {},
   "outputs": [],
   "source": [
    "kernel = sk.Kernel()\n",
    "\n",
    "# Configure Azure LLM service\n",
    "deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
    "azure_text_service = AzureTextCompletion(deployment, endpoint, api_key)\n",
    "azure_chat_service = AzureChatCompletion(deployment, endpoint, api_key)\n",
    "\n",
    "# Configure OpenAI service\n",
    "api_key, org_id = sk.openai_settings_from_dot_env()\n",
    "oai_text_service = OpenAITextCompletion(\"text-davinci-003\", api_key, org_id)\n",
    "oai_chat_service = OpenAIChatCompletion(\"gpt-3.5-turbo\", api_key, org_id)\n",
    "\n",
    "# Configure Hugging Face service\n",
    "hf_text_service = HuggingFaceTextCompletion(\"distilgpt2\", task=\"text-generation\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "50561d82",
   "metadata": {},
   "source": [
    "Next, we'll set up the completion request settings for text completion services."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "628c843e",
   "metadata": {},
   "outputs": [],
   "source": [
    "request_settings = CompleteRequestSettings(\n",
    "    max_tokens=80,\n",
    "    temperature=0.7,\n",
    "    top_p=1,\n",
    "    frequency_penalty=0.5,\n",
    "    presence_penalty=0.5,\n",
    "    number_of_responses=3\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "857a9c89",
   "metadata": {},
   "source": [
    "## Multiple Open AI Text Completions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2979db8",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"what is the purpose of a rubber duck?\"\n",
    "results = await oai_text_service.complete_async(prompt, request_settings)\n",
    "i = 1\n",
    "for result in results:\n",
    "    print(f\"Result {i}: {result}\")\n",
    "    i += 1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4288d09f",
   "metadata": {},
   "source": [
    "## Multiple Azure Open AI Text Completions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5319f14d",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"provide me a list of possible meanings for the acronym 'ORLD'\"\n",
    "results = await azure_text_service.complete_async(prompt, request_settings)\n",
    "i = 1\n",
    "for result in results:\n",
    "    print(f\"Result {i}: {result}\")\n",
    "    i += 1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "eb548f9c",
   "metadata": {},
   "source": [
    "## Multiple Hugging Face Text Completions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9525e4f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"The purpose of a rubber duck is\"\n",
    "results = await hf_text_service.complete_async(prompt, request_settings)\n",
    "i = 1\n",
    "for result in results:\n",
    "    print(f\"Result {i}: {result}\")\n",
    "    i += 1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "da632e12",
   "metadata": {},
   "source": [
    "Here, we're setting up the settings for Chat completions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5f11e46",
   "metadata": {},
   "outputs": [],
   "source": [
    "chat_request_settings = ChatRequestSettings(\n",
    "    max_tokens=80,\n",
    "    temperature=0.7,\n",
    "    top_p=1,\n",
    "    frequency_penalty=0.5,\n",
    "    presence_penalty=0.5,\n",
    "    number_of_responses=3\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d6bf238e",
   "metadata": {},
   "source": [
    "## Multiple OpenAI Chat Completions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dabc6a4c",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"It's a beautiful day outside, birds are singing, flowers are blooming. On days like these, kids like you...\"\n",
    "results = await oai_chat_service.complete_chat_async([(\"user\", prompt)], chat_request_settings)\n",
    "i = 0\n",
    "for result in results:\n",
    "    print(f\"Result {i}: {result}\")\n",
    "    i += 1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "cdb8f740",
   "metadata": {},
   "source": [
    "## Multiple Azure OpenAI Chat Completions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b74a64a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"Tomorow is going to be a great day, I can feel it. I'm going to wake up early, go for a run, and then...\"\n",
    "results = await azure_chat_service.complete_chat_async([(\"user\", prompt)], chat_request_settings)\n",
    "i = 0\n",
    "for result in results:\n",
    "    print(f\"Result {i}: {result}\")\n",
    "    i += 1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "98c8191d",
   "metadata": {},
   "source": [
    "## Streaming Multiple Results\n",
    "\n",
    "Here is an example pattern if you want to stream your multiple results. Note that this is not supported for Hugging Face text completions at this time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "26a37702",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result 1: \n",
      "\n",
      "A rubber duck is a toy shaped like a bath duck, often yellow with an orange bill and white eyes. It is often used as a bath time companion for children to play games with. Rubber ducks can also be used as decorations in the bathroom or around the house, and for other various games and activities.\n",
      "Result 2: \n",
      "\n",
      "A rubber duck is a type of toy shaped like a duck, often used in bath time play. It can also be used as a decorative item or to relieve stress.\n",
      "Result 3: \n",
      "\n",
      "A rubber duck is a toy shaped like a duck, often yellow and made of rubber. Its purpose is to provide children with entertainment and promote imaginative play.\n",
      "----------------------------------------\n"
     ]
    }
   ],
   "source": [
    "from IPython.display import clear_output\n",
    "\n",
    "if os.name == \"nt\":\n",
    "    clear = \"cls\"\n",
    "else:\n",
    "    clear = \"clear\"\n",
    "\n",
    "prompt = \"what is the purpose of a rubber duck?\"\n",
    "stream = oai_text_service.complete_stream_async(prompt, request_settings)\n",
    "texts = [\"\"] * request_settings.number_of_responses\n",
    "async for results in stream:\n",
    "    i = 1\n",
    "    clear_output(wait=True)\n",
    "    for result in results:\n",
    "        texts[i - 1] += result\n",
    "        print(f\"Result {i}: {texts[i - 1]}\")\n",
    "        i += 1\n",
    "\n",
    "print(\"----------------------------------------\")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}