L3_model_selection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f501ffce-f4f6-4701-8dd3-3ed32a5134ba",
   "metadata": {},
   "source": [
    "# Model Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd845946-a8c7-4e06-9ffe-dd893ee5e137",
   "metadata": {},
   "source": [
    "### Get API Key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4896b6d5-9e73-4fd1-9fc0-8a7cb93bb750",
   "metadata": {
    "height": 59
   },
   "outputs": [],
   "source": [
    "from helper import load_mistral_api_key\n",
    "api_key, dlai_endpoint = load_mistral_api_key(ret_key=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5800f02-c8a0-4e82-a4bd-ce6ddf7a26b7",
   "metadata": {},
   "source": [
    "- Note: in the classroom, if you print out this `api_key` variable, it is not a real API key (for security reasons).\n",
    "- If you wish to run this code on your own machine, outside of the classroom, you can still reuse the code that you see in `helper.py`.\n",
    "- It uses [python-dotenv](https://pypi.org/project/python-dotenv/) library to securely save and load sensitive information such as API keys."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "51403fae-91e9-4e24-9353-02495cb2babc",
   "metadata": {
    "height": 297
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from mistralai.client import MistralClient\n",
    "from mistralai.models.chat_completion import ChatMessage\n",
    "\n",
    "def mistral(user_message, model=\"mistral-small-latest\", is_json=False):\n",
    "    client = MistralClient(api_key=api_key, endpoint=dlai_endpoint)\n",
    "    messages = [ChatMessage(role=\"user\", content=user_message)]\n",
    "\n",
    "    if is_json:\n",
    "        chat_response = client.chat(\n",
    "            model=model, messages=messages, response_format={\"type\": \"json_object\"}\n",
    "        )\n",
    "    else:\n",
    "        chat_response = client.chat(model=model, messages=messages)\n",
    "\n",
    "    return chat_response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03db3133-63d2-4cc7-b562-efce0991a143",
   "metadata": {},
   "source": [
    "## Mistral Small\n",
    "\n",
    "Good for simple tasks, fast inference, lower cost.\n",
    "- classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "18eaa6a5-6653-4587-8077-e409191c790b",
   "metadata": {
    "height": 178
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\"\n",
    "Classify the following email to determine if it is spam or not.\n",
    "Only respond with the exact text \"Spam\" or \"Not Spam\". \n",
    "\n",
    "# Email:\n",
    "🎉 Urgent! You've Won a $1,000,000 Cash Prize! \n",
    "💰 To claim your prize, please click on the link below: \n",
    "https://bit.ly/claim-your-prize\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "11cef6cd-bb24-46c5-9e83-2d81168057eb",
   "metadata": {
    "height": 30,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Spam'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mistral(prompt, model=\"mistral-small-latest\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ac31f9e-8446-417c-aec8-598e141686bc",
   "metadata": {},
   "source": [
    "## Mistral Medium\n",
    "\n",
    "Good for intermediate tasks such as language transformation.\n",
    "- Composing text based on provided context (e.g. writing a customer service email based on purchase information)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "87ce9677-a0a2-40ef-903d-571561c0fc65",
   "metadata": {
    "height": 263
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\"\n",
    "Compose a welcome email for new customers who have just made \n",
    "their first purchase with your product. \n",
    "Start by expressing your gratitude for their business, \n",
    "and then convey your excitement for having them as a customer. \n",
    "Include relevant details about their recent order. \n",
    "Sign the email with \"The Fun Shop Team\".\n",
    "\n",
    "Order details:\n",
    "- Customer name: Anna\n",
    "- Product: hat \n",
    "- Estimate date of delivery: Feb. 25, 2024\n",
    "- Return policy: 30 days\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1e0bb8a6-5be9-4a83-9413-14e154c845de",
   "metadata": {
    "height": 42
   },
   "outputs": [],
   "source": [
    "response_medium = mistral(prompt, model=\"mistral-medium-latest\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6de3dada-f0e1-4dee-9a41-31cac83256b0",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Subject: Welcome to The Fun Shop, Anna! Your New Hat is On Its Way!\n",
      "\n",
      "Dear Anna,\n",
      "\n",
      "We're so excited to have you as a customer at The Fun Shop! We want to express our heartfelt gratitude for your recent purchase with us. We're confident that you're going to love your new hat.\n",
      "\n",
      "Your order details are as follows:\n",
      "\n",
      "Product: Hat\n",
      "Estimated Delivery Date: February 25, 2024\n",
      "Return Policy: 30 days\n",
      "\n",
      "We're committed to providing you with the best possible shopping experience, and we're confident that you're going to love your new hat. Our team is dedicated to ensuring that your order arrives on time and in perfect condition.\n",
      "\n",
      "If you have any questions or concerns about your order, please don't hesitate to reach out to us at support@thefunshop.com. Our customer service team is available 24/7 to help you with any issues you may encounter.\n",
      "\n",
      "We're also pleased to let you know that we have a 30-day return policy, so if you're not completely satisfied with your purchase, you can return it for a full refund or exchange.\n",
      "\n",
      "Thank you again for choosing The Fun Shop for your shopping needs. We're thrilled to have you as a customer and can't wait to see you rocking your new hat!\n",
      "\n",
      "Best regards,\n",
      "The Fun Shop Team\n"
     ]
    }
   ],
   "source": [
    "print(response_medium)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d3b2c99-c01a-4beb-8685-355f6c21ce55",
   "metadata": {},
   "source": [
    "## Mistral Large: \n",
    "\n",
    "Good for complex tasks that require advanced reasoning.\n",
    "- Math and reasoning with numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "fc4c9957-48dc-46b8-bb6f-f506af1f768f",
   "metadata": {
    "height": 263
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\"\n",
    "Calculate the difference in payment dates between the two \\\n",
    "customers whose payment amounts are closest to each other \\\n",
    "in the following dataset. Do not write code.\n",
    "\n",
    "# dataset: \n",
    "'{\n",
    "  \"transaction_id\":{\"0\":\"T1001\",\"1\":\"T1002\",\"2\":\"T1003\",\"3\":\"T1004\",\"4\":\"T1005\"},\n",
    "    \"customer_id\":{\"0\":\"C001\",\"1\":\"C002\",\"2\":\"C003\",\"3\":\"C002\",\"4\":\"C001\"},\n",
    "    \"payment_amount\":{\"0\":125.5,\"1\":89.99,\"2\":120.0,\"3\":54.3,\"4\":210.2},\n",
    "\"payment_date\":{\"0\":\"2021-10-05\",\"1\":\"2021-10-06\",\"2\":\"2021-10-07\",\"3\":\"2021-10-05\",\"4\":\"2021-10-08\"},\n",
    "    \"payment_status\":{\"0\":\"Paid\",\"1\":\"Unpaid\",\"2\":\"Paid\",\"3\":\"Paid\",\"4\":\"Pending\"}\n",
    "}'\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ba02ea8b-8872-4a7d-8195-6285e4b422a3",
   "metadata": {
    "height": 42
   },
   "outputs": [],
   "source": [
    "response_small = mistral(prompt, model=\"mistral-small-latest\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "74554e58-25a0-402d-95ad-8ada5c0cc743",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To solve this, we first need to find the two customers with the closest payment amounts. From the dataset, we can see that the closest payment amounts are 125.5 and 120.0, belonging to customers C001 and C003.\n",
      "\n",
      "Next, we need to find the payment dates for these customers. Customer C001 made a payment on 2021-10-05, and customer C003 made a payment on 2021-10-07.\n",
      "\n",
      "Finally, we calculate the difference between these dates. In this case, the difference is 2 days (from 2021-10-05 to 2021-10-07).\n"
     ]
    }
   ],
   "source": [
    "print(response_small)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "987a0aef-67dc-4ada-82ac-4b0dc1b6b5ae",
   "metadata": {
    "height": 42
   },
   "outputs": [],
   "source": [
    "response_large = mistral(prompt, model=\"mistral-large-latest\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f21a55c1-c700-4bf3-be22-5ad432dc3f10",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To solve this problem without writing code, we first need to identify the two customers whose payment amounts are closest to each other. Looking at the dataset, the payment amounts are:\n",
      "\n",
      "- C001: 125.5, 210.2\n",
      "- C002: 89.99, 54.3\n",
      "- C003: 120.0\n",
      "\n",
      "The closest pair of payments is 125.5 and 120.0, which are from customers C001 and C003.\n",
      "\n",
      "Next, we need to calculate the difference in payment dates. The payment dates are:\n",
      "\n",
      "- C001: 2021-10-05, 2021-10-08\n",
      "- C002: 2021-10-06, 2021-10-05\n",
      "- C003: 2021-10-07\n",
      "\n",
      "The payment date for the 125.5 transaction from C001 is 2021-10-05, and the payment date for the 120.0 transaction from C003 is 2021-10-07.\n",
      "\n",
      "So, the difference in payment dates is 2 days (2021-10-07 minus 2021-10-05).\n"
     ]
    }
   ],
   "source": [
    "print(response_large)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a15be44d-ecb4-47f6-bed5-5b08c6bc391a",
   "metadata": {},
   "source": [
    "## Expense reporting task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f7c1deb9-37f4-4182-b43c-3bad2637598c",
   "metadata": {
    "height": 348
   },
   "outputs": [],
   "source": [
    "transactions = \"\"\"\n",
    "McDonald's: 8.40\n",
    "Safeway: 10.30\n",
    "Carrefour: 15.00\n",
    "Toys R Us: 20.50\n",
    "Panda Express: 10.20\n",
    "Beanie Baby Outlet: 25.60\n",
    "World Food Wraps: 22.70\n",
    "Stuffed Animals Shop: 45.10\n",
    "Sanrio Store: 85.70\n",
    "\"\"\"\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Given the purchase details, how much did I spend on each category:\n",
    "1) restaurants\n",
    "2) groceries\n",
    "3) stuffed animals and props\n",
    "{transactions}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c534c567-cd20-4abb-adbc-cc9fe919c2f8",
   "metadata": {
    "height": 59
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To categorize your purchases, I've made the following assumptions:\n",
      "\n",
      "1) restaurants: McDonald's and Panda Express\n",
      "2) groceries: Safeway and Carrefour\n",
      "3) stuffed animals and props: Toys R Us, Beanie Baby Outlet, Stuffed Animals Shop, and Sanrio Store\n",
      "\n",
      "Here's the breakdown of your spending by category:\n",
      "\n",
      "1) restaurants:\n",
      "   - McDonald's: 8.40\n",
      "   - Panda Express: 10.20\n",
      "   Total: 18.60\n",
      "\n",
      "2) groceries:\n",
      "   - Safeway: 10.30\n",
      "   - Carrefour: 15.00\n",
      "   Total: 25.30\n",
      "\n",
      "3) stuffed animals and props:\n",
      "   - Toys R Us: 20.50\n",
      "   - Beanie Baby Outlet: 25.60\n",
      "   - Stuffed Animals Shop: 45.10\n",
      "   - Sanrio Store: 85.70\n",
      "   Total: 176.90\n"
     ]
    }
   ],
   "source": [
    "response_small = mistral(prompt, model=\"mistral-small-latest\")\n",
    "print(response_small)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "47960a78-7689-47ee-adee-6c8412d5477b",
   "metadata": {
    "height": 59
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sure, I can help you categorize your expenses. Here's the breakdown:\n",
      "\n",
      "1) Restaurants:\n",
      "   - McDonald's: 8.40\n",
      "   - Panda Express: 10.20\n",
      "   - World Food Wraps: 22.70\n",
      "   Total spent on restaurants: 41.30\n",
      "\n",
      "2) Groceries:\n",
      "   - Safeway: 10.30\n",
      "   - Carrefour: 15.00\n",
      "   Total spent on groceries: 25.30\n",
      "\n",
      "3) Stuffed animals and props:\n",
      "   - Toys R Us: 20.50\n",
      "   - Beanie Baby Outlet: 25.60\n",
      "   - Stuffed Animals Shop: 45.10\n",
      "   - Sanrio Store: 85.70\n",
      "   Total spent on stuffed animals and props: 176.90\n"
     ]
    }
   ],
   "source": [
    "response_large = mistral(prompt, model=\"mistral-large-latest\")\n",
    "print(response_large)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f90d6068-61b3-4490-aeb4-0be67ba9fd1b",
   "metadata": {},
   "source": [
    "## Writing and checking code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "27945975-d2bc-40b9-9a59-48b10fe4da4b",
   "metadata": {
    "height": 246
   },
   "outputs": [],
   "source": [
    "user_message = \"\"\"\n",
    "Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\n",
    "\n",
    "You may assume that each input would have exactly one solution, and you may not use the same element twice.\n",
    "\n",
    "You can return the answer in any order.\n",
    "\n",
    "Your code should pass these tests:\n",
    "\n",
    "assert twoSum([2,7,11,15], 9) == [0,1]\n",
    "assert twoSum([3,2,4], 6) == [1,2]\n",
    "assert twoSum([3,3], 6) == [0,1]\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "8e00a7bc-069a-43f9-bb02-e33a5d3dcc2f",
   "metadata": {
    "height": 42
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sure, I can help you with that. Here's a Python function that should do the trick:\n",
      "\n",
      "```python\n",
      "def twoSum(nums, target):\n",
      "    if len(nums) <= 1:\n",
      "        return \"The list should have at least two elements\"\n",
      "\n",
      "    num_dict = {}\n",
      "    for i, num in enumerate(nums):\n",
      "        if num in num_dict:\n",
      "            return [num_dict[num], i]\n",
      "        else:\n",
      "            num_dict[target - num] = i\n",
      "\n",
      "    return \"No two sum solution\"\n",
      "```\n",
      "\n",
      "This function uses a dictionary to store the indices of the numbers in the list. It then iterates through the list and for each number, it checks if the number is in the dictionary. If it is, it means that we have found two numbers that add up to the target and we return their indices. If we iterate through the entire list and do not find any two numbers that add up to the target, we return \"No two sum solution\".\n",
      "\n",
      "Please note that the function will return an error message if the input list has less than two elements.\n",
      "\n",
      "You can test the function with your test cases like this:\n",
      "\n",
      "```python\n",
      "assert twoSum([2,7,11,15], 9) == [0,1]\n",
      "assert twoSum([3,2,4], 6) == [1,2]\n",
      "assert twoSum([3,3], 6) == [0,1]\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "print(mistral(user_message, model=\"mistral-large-latest\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8e007ef-17fb-450a-85d5-f456d06352e1",
   "metadata": {},
   "source": [
    "### Try out the code that the model provided\n",
    "- Copy the code that the model provided and try running it!\n",
    "\n",
    "Here is the code that was output at the time of filming:\n",
    "```Python\n",
    "def twoSum(nums, target):\n",
    "    seen = {}\n",
    "    for i, num in enumerate(nums):\n",
    "        complement = target - num\n",
    "        if complement in seen:\n",
    "            return [seen[complement], i]\n",
    "        seen[num] = i\n",
    "```\n",
    "- Also try running the assert statements in the original prompt\n",
    "```Python\n",
    "assert twoSum([2,7,11,15], 9) == [0,1]\n",
    "assert twoSum([3,2,4], 6) == [1,2]\n",
    "assert twoSum([3,3], 6) == [0,1]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fcd58acf-fa92-431d-a08a-ce05c1e21003",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49638589-88c6-439c-8192-65c511522fc1",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "3db8aa2a-aa1f-45f1-b4a4-d8eedc927677",
   "metadata": {},
   "source": [
    "## Natively Fluent in English, French, Spanish, German, and Italian\n",
    "- This means that you can use Mistral models for more than translating from one language to another.\n",
    "- If you are a native Spanish speaker, for instance, you can communicate with Mistral models in Spanish for any of your tasks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "3cbad716-8da8-4c00-8eb1-889b69567986",
   "metadata": {
    "height": 76
   },
   "outputs": [],
   "source": [
    "user_message = \"\"\"\n",
    "Lequel est le plus lourd une livre de fer ou un kilogramme de plume\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "72722d49-b12b-4334-bc1c-318874c57959",
   "metadata": {
    "height": 42,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Une livre de fer et un kilogramme de plumes ont des poids différents.\n",
      "\n",
      "Une livre (lb) est une unité de poids couramment utilisée aux États-Unis et dans d'autres systèmes de mesure impériaux, tandis qu'un kilogramme (kg) est l'unité de base de poids dans le système métrique international, utilisé dans la plupart des autres pays.\n",
      "\n",
      "Pour comparer ces deux quantités, nous devons les convertir dans la même unité.\n",
      "\n",
      "1 kilogramme est égal à environ 2,20462 livres. Donc, un kilogramme de plumes pèse plus qu'une livre de fer.\n",
      "\n",
      "Cependant, il est important de noter que cela ne signifie pas qu'une seule plume est plus lourde qu'un morceau de fer. C'est la somme de toutes les plumes dans un kilogramme qui est plus lourde que la somme de tout le fer dans une livre.\n"
     ]
    }
   ],
   "source": [
    "print(mistral(user_message, model=\"mistral-large-latest\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd4f1e52-49d5-4d02-b85c-5bce23896a20",
   "metadata": {},
   "source": [
    "### Try it out for yourself\n",
    "- Try communicating with the Mistral Large model in Spanish\n",
    "  - (If you need help, you can first translate a prompt from English to Spanish, and then prompt the model in Spanish)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc301515-157c-450b-828e-554ee6760809",
   "metadata": {},
   "source": [
    "## List of Mistral models that you can call:\n",
    "\n",
    "You can also call the two open source mistral models via API calls.\n",
    "Here is the list of models that you can try:\n",
    "```\n",
    "open-mistral-7b\n",
    "open-mixtral-8x7b\n",
    "open-mixtral-8x22b\n",
    "mistral-small-latest\n",
    "mistral-medium-latest\n",
    "mistral-large-latest\n",
    "```\n",
    "\n",
    "For example:\n",
    "```Python\n",
    "mistral(prompt, model=\"open-mixtral-8x22b\")\n",
    "```\n",
    "\n",
    "Note that we just released the `open-mixtral-8x22b` model. Check out our [release blog](https://mistral.ai/news/mixtral-8x22b/) for details. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6e316a8f-57fc-4774-980e-118c01239636",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "043a12ee",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}