Snowpark_For_Python_ML.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5f02f3fc",
   "metadata": {},
   "source": [
    "# Project Overview\n",
    "Perform data analysis and data preparation tasks to train a Linear Regression model to predict future ROI (Return On Investment) of variable ad spend budgets across multiple channels including search, video, social media, and email using Snowpark for Python, Streamlit and scikit-learn. By the end of the session, you will have an interactive web application deployed visualizing the ROI of different allocated advertising spend budgets.\n",
    "\n",
    "***Prerequisite**: Before proceeding with this Notebook, you must first successfully run through [Snowpark_For_Python_DE.ipynb](Snowpark_For_Python_DE.ipynb).*\n",
    "\n",
    "### Machine Learning\n",
    "In ths Notebook, we will focus on Machine Learning in Snowflake using Snowpark for Python.\n",
    "\n",
    "* Establish secure connection to Snowflake\n",
    "* Load features and target from Snowflake table into Snowpark DataFrame\n",
    "* Prepare features for model training\n",
    "* Create a Python Stored Procedure to deploy model training code on Snowflake\n",
    "* Create Python Scalar and Vectorized User-Defined Functions (UDF) for inference on new data points\n",
    "  *  *Note: The Scalar UDF is called from the Streamlit Apps. See [Snowpark_Streamlit_Revenue_Prediction.py](Snowpark_Streamlit_Revenue_Prediction.py) and [Snowpark_Streamlit_Revenue_Prediction_SiS.py](Snowpark_Streamlit_Revenue_Prediction_SiS.py)*\n",
    "\n",
    "*For environment setup including loading data into Snowflake tables, and step-by-step instructions, please refer to the [QuickStart Guide](https://quickstarts.snowflake.com/guide/getting_started_with_dataengineering_ml_using_snowpark_python/index.html#0).*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e2e1c4a",
   "metadata": {},
   "source": [
    "### Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "persistent-reasoning",
   "metadata": {
    "papermill": {
     "duration": 0.872363,
     "end_time": "2021-05-15T09:33:41.413139",
     "exception": false,
     "start_time": "2021-05-15T09:33:40.540776",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Snowpark for Python\n",
    "from snowflake.snowpark.session import Session\n",
    "from snowflake.snowpark.types import Variant\n",
    "from snowflake.snowpark.functions import udf,sum,col,array_construct,month,year,call_udf,lit\n",
    "from snowflake.snowpark.version import VERSION\n",
    "# Misc\n",
    "import json\n",
    "import logging \n",
    "logger = logging.getLogger(\"snowflake.snowpark.session\")\n",
    "logger.setLevel(logging.ERROR)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b855e80e",
   "metadata": {},
   "source": [
    "### Establish Secure Connection to Snowflake\n",
    "\n",
    "Using the Snowpark Python API, it’s quick and easy to establish a secure connection between Snowflake and Notebook.\n",
    "\n",
    " *Note: Other connection options include Username/Password, MFA, OAuth, Okta, SSO*\n",
    "\n",
    "TIP: Learn more about [Session](https://docs.snowflake.com/developer-guide/snowpark/reference/python/session.html) object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "rotary-dairy",
   "metadata": {
    "papermill": {
     "duration": 0.092202,
     "end_time": "2021-05-15T09:34:12.813622",
     "exception": false,
     "start_time": "2021-05-15T09:34:12.721420",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "User                        : DASHDESAI\n",
      "Role                        : \"ACCOUNTADMIN\"\n",
      "Database                    : \"DASH_DB\"\n",
      "Schema                      : \"DASH_SCHEMA\"\n",
      "Warehouse                   : \"DASH_S\"\n",
      "Snowflake version           : 7.9.0\n",
      "Snowpark for Python version : 1.0.0\n"
     ]
    }
   ],
   "source": [
    "# Create Snowflake Session object\n",
    "connection_parameters = json.load(open('connection.json'))\n",
    "session = Session.builder.configs(connection_parameters).create()\n",
    "session.sql_simplifier_enabled = True\n",
    "\n",
    "snowflake_environment = session.sql('select current_user(), current_version()').collect()\n",
    "snowpark_version = VERSION\n",
    "\n",
    "# Current Environment Details\n",
    "print('User                        : {}'.format(snowflake_environment[0][0]))\n",
    "print('Role                        : {}'.format(session.get_current_role()))\n",
    "print('Database                    : {}'.format(session.get_current_database()))\n",
    "print('Schema                      : {}'.format(session.get_current_schema()))\n",
    "print('Warehouse                   : {}'.format(session.get_current_warehouse()))\n",
    "print('Snowflake version           : {}'.format(snowflake_environment[0][1]))\n",
    "print('Snowpark for Python version : {}.{}.{}'.format(snowpark_version[0],snowpark_version[1],snowpark_version[2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2817d70c",
   "metadata": {},
   "source": [
    "### Features and Target\n",
    "\n",
    "At this point we are ready to perform the following actions to save features and target for model training.\n",
    "\n",
    "* Delete rows with missing values\n",
    "* Exclude columns we don't need for modeling\n",
    "* Save features into a Snowflake table called MARKETING_BUDGETS_FEATURES"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "bbd880a0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------\n",
      "|\"SEARCH_ENGINE\"  |\"SOCIAL_MEDIA\"  |\"VIDEO\"  |\"EMAIL\"  |\"REVENUE\"   |\n",
      "---------------------------------------------------------------------\n",
      "|522298           |520085          |521439   |518486   |3291585.64  |\n",
      "|502889           |503335          |504696   |504948   |3226349.23  |\n",
      "|521487           |523460          |518452   |520555   |3333880.29  |\n",
      "|520207           |520362          |521610   |520477   |3294351.31  |\n",
      "|502610           |505287          |504050   |503095   |3234601.13  |\n",
      "|521769           |519245          |523472   |520179   |3330604.0   |\n",
      "|505149           |502322          |504099   |501490   |3201100.65  |\n",
      "|523302           |520194          |521552   |524296   |3293194.93  |\n",
      "|521495           |521804          |521094   |521042   |3334369.19  |\n",
      "|470330           |469796          |472430   |467531   |2986760.74  |\n",
      "---------------------------------------------------------------------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Load data\n",
    "snow_df_spend_and_revenue_per_month = session.table('spend_and_revenue_per_month')\n",
    "\n",
    "# Delete rows with missing values\n",
    "snow_df_spend_and_revenue_per_month = snow_df_spend_and_revenue_per_month.dropna()\n",
    "\n",
    "# Exclude columns we don't need for modeling\n",
    "snow_df_spend_and_revenue_per_month = snow_df_spend_and_revenue_per_month.drop(['YEAR','MONTH'])\n",
    "\n",
    "# Save features into a Snowflake table call MARKETING_BUDGETS_FEATURES\n",
    "snow_df_spend_and_revenue_per_month.write.mode('overwrite').save_as_table('MARKETING_BUDGETS_FEATURES')\n",
    "snow_df_spend_and_revenue_per_month.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "social-enough",
   "metadata": {
    "papermill": {
     "duration": 0.07231,
     "end_time": "2021-05-15T09:33:52.171663",
     "exception": false,
     "start_time": "2021-05-15T09:33:52.099353",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Model Training in Snowflake \n",
    "\n",
    "#### Python function to train a Linear Regression model using scikit-learn\n",
    "\n",
    "Let's create a Python function that uses **scikit-learn and other packages which are already included in** [Snowflake Anaconda channel](https://repo.anaconda.com/pkgs/snowflake/) and therefore available on the server-side when executing the Python function as a Stored Procedure running in Snowflake.\n",
    "\n",
    "This function takes the following as parameters:\n",
    "\n",
    "* _session_: Snowflake Session object.\n",
    "* _features_table_: Name of the table that holds the features and target variable.\n",
    "* _number_of_folds_: Number of cross validation folds used in GridSearchCV.\n",
    "* _polynomial_features_degress_: PolynomialFeatures as a preprocessing step.\n",
    "* _train_accuracy_threshold_: Accuracy thresholds for train dataset. This values is used to determine if the model should be saved.\n",
    "* _test_accuracy_threshold_: Accuracy thresholds for test dataset. This values is used to determine if the model should be saved.\n",
    "* _save_model_: Boolean that determines if the model should be saved provided the accuracy thresholds are met.\n",
    "\n",
    "TIP: Learn more about [Snowpark-optimized Warehouses](https://docs.snowflake.com/en/user-guide/warehouses-snowpark-optimized.html).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "caring-weight",
   "metadata": {
    "papermill": {
     "duration": 0.079552,
     "end_time": "2021-05-15T09:33:52.631191",
     "exception": false,
     "start_time": "2021-05-15T09:33:52.551639",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def train_revenue_prediction_model(\n",
    "    session: Session, \n",
    "    features_table: str, \n",
    "    number_of_folds: int, \n",
    "    polynomial_features_degrees: int, \n",
    "    train_accuracy_threshold: float, \n",
    "    test_accuracy_threshold: float, \n",
    "    save_model: bool) -> Variant:\n",
    "    \n",
    "    from sklearn.compose import ColumnTransformer\n",
    "    from sklearn.pipeline import Pipeline\n",
    "    from sklearn.preprocessing import PolynomialFeatures\n",
    "    from sklearn.preprocessing import StandardScaler\n",
    "    from sklearn.linear_model import LinearRegression\n",
    "    from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "\n",
    "    import os\n",
    "    from joblib import dump\n",
    "\n",
    "    # Load features\n",
    "    df = session.table(features_table).to_pandas()\n",
    "\n",
    "    # Preprocess the Numeric columns\n",
    "    # We apply PolynomialFeatures and StandardScaler preprocessing steps to the numeric columns\n",
    "    # NOTE: High degrees can cause overfitting.\n",
    "    numeric_features = ['SEARCH_ENGINE','SOCIAL_MEDIA','VIDEO','EMAIL']\n",
    "    numeric_transformer = Pipeline(steps=[('poly',PolynomialFeatures(degree = polynomial_features_degrees)),('scaler', StandardScaler())])\n",
    "\n",
    "    # Combine the preprocessed step together using the Column Transformer module\n",
    "    preprocessor = ColumnTransformer(\n",
    "        transformers=[\n",
    "            ('num', numeric_transformer, numeric_features)])\n",
    "\n",
    "    # The next step is the integrate the features we just preprocessed with our Machine Learning algorithm to enable us to build a model\n",
    "    pipeline = Pipeline(steps=[('preprocessor', preprocessor),('classifier', LinearRegression())])\n",
    "    parameteres = {}\n",
    "\n",
    "    X = df.drop('REVENUE', axis = 1)\n",
    "    y = df['REVENUE']\n",
    "\n",
    "    # Split dataset into training and test\n",
    "    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state = 42)\n",
    "\n",
    "    # Use GridSearch to find the best fitting model based on number_of_folds folds\n",
    "    model = GridSearchCV(pipeline, param_grid=parameteres, cv=number_of_folds)\n",
    "\n",
    "    model.fit(X_train, y_train)\n",
    "    train_r2_score = model.score(X_train, y_train)\n",
    "    test_r2_score = model.score(X_test, y_test)\n",
    "\n",
    "    model_saved = False\n",
    "\n",
    "    if save_model:\n",
    "        if train_r2_score >= train_accuracy_threshold and test_r2_score >= test_accuracy_threshold:\n",
    "            # Upload trained model to a stage\n",
    "            model_output_dir = '/tmp'\n",
    "            model_file = os.path.join(model_output_dir, 'model.joblib')\n",
    "            dump(model, model_file)\n",
    "            session.file.put(model_file,\"@dash_models\",overwrite=True)\n",
    "            model_saved = True\n",
    "\n",
    "    # Return model R2 score on train and test data\n",
    "    return {\"R2 score on Train\": train_r2_score,\n",
    "            \"R2 threshold on Train\": train_accuracy_threshold,\n",
    "            \"R2 score on Test\": test_r2_score,\n",
    "            \"R2 threshold on Test\": test_accuracy_threshold,\n",
    "            \"Model saved\": model_saved}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26304617",
   "metadata": {},
   "source": [
    "#### Test Python function before deploying it as a Stored Procedure on Snowflake\n",
    "\n",
    "Since we're in test mode, we will set _save_model = False_ so that the model is not saved just yet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "924c4f5f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'R2 score on Train': 0.9951556820022875,\n",
       " 'R2 threshold on Train': 0.85,\n",
       " 'R2 score on Test': 0.9661098174787659,\n",
       " 'R2 threshold on Test': 0.85,\n",
       " 'Model saved': False}"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_validaton_folds = 10\n",
    "polynomial_features_degrees = 2\n",
    "train_accuracy_threshold = 0.85\n",
    "test_accuracy_threshold = 0.85\n",
    "save_model = False\n",
    "\n",
    "train_revenue_prediction_model(\n",
    "    session,\n",
    "    \"MARKETING_BUDGETS_FEATURES\",\n",
    "    cross_validaton_folds,\n",
    "    polynomial_features_degrees,\n",
    "    train_accuracy_threshold,\n",
    "    test_accuracy_threshold,\n",
    "    save_model)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "bbf90cdc",
   "metadata": {},
   "source": [
    "### Create Stored Procedure to deploy model training code on Snowflake\n",
    "\n",
    "Assuming the testing is complete and we're satisfied with the model, let's **register the model training Python function as a Snowpark Python Stored Procedure** by supplying the packages (_snowflake-snowpark-python,scikit-learn, and joblib_) it will need and use during execution.\n",
    "\n",
    "TIP: Learn more about [Snowpark Python Stored Procedures](https://docs.snowflake.com/en/sql-reference/stored-procedures-python.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "630975b1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<snowflake.snowpark.stored_procedure.StoredProcedure at 0x7fb1408d5e50>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session.sproc.register(\n",
    "    func=train_revenue_prediction_model,\n",
    "    name=\"train_revenue_prediction_model\",\n",
    "    packages=['snowflake-snowpark-python','scikit-learn','joblib'],\n",
    "    is_permanent=True,\n",
    "    stage_location=\"@dash_sprocs\",\n",
    "    replace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd5e6ddc",
   "metadata": {},
   "source": [
    "### >>>>>>>>>> *Examine Query History in Snowsight* <<<<<<<<<<"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3f1dcfa",
   "metadata": {},
   "source": [
    "### Execute Stored Procedure to train model and deploy it on Snowflake\n",
    "\n",
    "Now we're ready to train the model and save it onto a Snowflake stage so let's set _save_model = True_ and run/execute the Stored Procedure using _session.call()_ function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "a8846d95",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"Model saved\": true,\n",
      "  \"R2 score on Test\": 0.9661098174787681,\n",
      "  \"R2 score on Train\": 0.9951556820022874,\n",
      "  \"R2 threshold on Test\": 0.85,\n",
      "  \"R2 threshold on Train\": 0.85\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "cross_validaton_folds = 10\n",
    "polynomial_features_degrees = 2\n",
    "train_accuracy_threshold = 0.85\n",
    "test_accuracy_threshold = 0.85\n",
    "save_model = True\n",
    "\n",
    "print(session.call('train_revenue_prediction_model',\n",
    "                    'MARKETING_BUDGETS_FEATURES',\n",
    "                    cross_validaton_folds,\n",
    "                    polynomial_features_degrees,\n",
    "                    train_accuracy_threshold,\n",
    "                    test_accuracy_threshold,\n",
    "                    save_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd5e6ddc",
   "metadata": {},
   "source": [
    "### >>>>>>>>>> *Examine Query History in Snowsight* <<<<<<<<<<"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3d3e1dbf",
   "metadata": {},
   "source": [
    "### Create Scalar User-Defined Function (UDF) for inference\n",
    "\n",
    "Now to deploy this model for inference, let's **create and register a Snowpark Python UDF and add the trained model as a dependency**. Once registered, getting new predictions is as simple as calling the function by passing in data.\n",
    "\n",
    "*Note: Scalar UDFs operate on a single row / set of data points and are great for online inference in real-time. And this UDF is called from the Streamlit App. See [Snowpark_Streamlit_Revenue_Prediction.py](Snowpark_Streamlit_Revenue_Prediction.py)*\n",
    "\n",
    "TIP: Learn more about [Snowpark Python User-Defined Functions](https://docs.snowflake.com/en/developer-guide/snowpark/python/creating-udfs.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "4a2c9a07",
   "metadata": {},
   "outputs": [],
   "source": [
    "session.clear_imports()\n",
    "session.clear_packages()\n",
    "\n",
    "# Add trained model and Python packages from Snowflake Anaconda channel available on the server-side as UDF dependencies\n",
    "session.add_import('@dash_models/model.joblib.gz')\n",
    "session.add_packages('pandas','joblib','scikit-learn==1.1.1')\n",
    "\n",
    "@udf(name='predict_roi',session=session,replace=True,is_permanent=True,stage_location='@dash_udfs')\n",
    "def predict_roi(budget_allocations: list) -> float:\n",
    "    import sys\n",
    "    import pandas as pd\n",
    "    from joblib import load\n",
    "    import sklearn\n",
    "\n",
    "    IMPORT_DIRECTORY_NAME = \"snowflake_import_directory\"\n",
    "    import_dir = sys._xoptions[IMPORT_DIRECTORY_NAME]\n",
    "    \n",
    "    model_file = import_dir + 'model.joblib.gz'\n",
    "    model = load(model_file)\n",
    "            \n",
    "    features = ['SEARCH_ENGINE','SOCIAL_MEDIA','VIDEO','EMAIL']\n",
    "    df = pd.DataFrame([budget_allocations], columns=features)\n",
    "    roi = abs(model.predict(df)[0])\n",
    "    return roi"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8fc7202",
   "metadata": {},
   "source": [
    "### Call Scalar User-Defined Function (UDF) for inference on new data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d40e0fb7",
   "metadata": {},
   "source": [
    "Once the UDF is registered, getting new predictions is as simple as calling the _call_udf()_ Snowpark Python function and passing in new datapoints.\n",
    "\n",
    "Let's create a SnowPark DataFrame with some sample data and call the UDF to get new predictions.\n",
    "\n",
    "*Note: This UDF is also called from the Streamlit App. See [Snowpark_Streamlit_Revenue_Prediction.py](Snowpark_Streamlit_Revenue_Prediction.py)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "d922f065",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----------------------------------------------------------------------------\n",
      "|\"SEARCH_ENGINE\"  |\"SOCIAL_MEDIA\"  |\"VIDEO\"  |\"EMAIL\"  |\"PREDICTED_ROI\"     |\n",
      "-----------------------------------------------------------------------------\n",
      "|250000           |250000          |200000   |450000   |333166.0182359633   |\n",
      "|8500             |9500            |2000     |500      |233784.84401053237  |\n",
      "|500000           |500000          |500000   |500000   |3178590.4246200835  |\n",
      "-----------------------------------------------------------------------------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "test_df = session.create_dataframe([[250000,250000,200000,450000],[500000,500000,500000,500000],[8500,9500,2000,500]], \n",
    "                                    schema=['SEARCH_ENGINE','SOCIAL_MEDIA','VIDEO','EMAIL'])\n",
    "test_df.select(\n",
    "    'SEARCH_ENGINE','SOCIAL_MEDIA','VIDEO','EMAIL', \n",
    "    call_udf(\"predict_roi\", \n",
    "    array_construct(col(\"SEARCH_ENGINE\"), col(\"SOCIAL_MEDIA\"), col(\"VIDEO\"), col(\"EMAIL\"))).as_(\"PREDICTED_ROI\")).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c4825e30",
   "metadata": {},
   "source": [
    "### Create Vectorized User-Defined Function (UDF) using Batch API for inference\n",
    "\n",
    "Here we will leverage the Python UDF Batch API to create a **vectorized** UDF which takes a Pandas Dataframe as input. This means that each call to the UDF receives a set/batch of rows compared to a Scalar UDF which gets one row as input. \n",
    "\n",
    "First we will create a helper function _load_model()_ that uses **cachetools** to make sure we only load the model once followed by _batch_predict_roi()_ function that does the inference. \n",
    "\n",
    "*Note: Vectorized UDFs are great for offline inference in batch mode.*\n",
    "\n",
    "Advantages of using the Batch API over Scalar UDFs:\n",
    "\n",
    "* The potential for better performance if your Python code operates efficiently on batches of rows\n",
    "* Less transformation logic required if you are calling into libraries that operate on Pandas DataFrames or Pandas arrays\n",
    "\n",
    "TIP: Learn more about [Snowpark Python UDF Batch API](https://docs.snowflake.com/en/developer-guide/udf/python/udf-python-batch.html#getting-started-with-the-batch-api)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "9b938b6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "session.clear_imports()\n",
    "session.clear_packages()\n",
    "\n",
    "import cachetools\n",
    "from snowflake.snowpark.types import PandasSeries, PandasDataFrame\n",
    "\n",
    "# Add trained model and Python packages from Snowflake Anaconda channel available on the server-side as UDF dependencies\n",
    "session.add_import('@dash_models/model.joblib.gz')\n",
    "session.add_packages('pandas','joblib','scikit-learn','cachetools')\n",
    "\n",
    "@cachetools.cached(cache={})\n",
    "def load_model(filename):\n",
    "    import joblib\n",
    "    import sys\n",
    "    import os\n",
    "\n",
    "    IMPORT_DIRECTORY_NAME = \"snowflake_import_directory\"\n",
    "    import_dir = sys._xoptions[IMPORT_DIRECTORY_NAME]\n",
    "\n",
    "    if import_dir:\n",
    "        with open(os.path.join(import_dir, filename), 'rb') as file:\n",
    "            m = joblib.load(file)\n",
    "            return m\n",
    "\n",
    "@udf(name='batch_predict_roi',session=session,replace=True,is_permanent=True,stage_location='@dash_udfs')\n",
    "def batch_predict_roi(budget_allocations_df: PandasDataFrame[int, int, int, int]) -> PandasSeries[float]:\n",
    "    import sklearn\n",
    "    budget_allocations_df.columns = ['SEARCH_ENGINE','SOCIAL_MEDIA','VIDEO','EMAIL']\n",
    "    model = load_model('model.joblib.gz')\n",
    "    return abs(model.predict(budget_allocations_df))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "60a99309",
   "metadata": {},
   "source": [
    "### Call Vectorized User-Defined Function (UDF) using Batch API for inference on new data points\n",
    "\n",
    "*Note: You do not need to change how you write queries using Python UDFs. All batching is handled by the UDF framework rather than your own code*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "57045fca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----------------------------------------------------------------------------\n",
      "|\"SEARCH_ENGINE\"  |\"SOCIAL_MEDIA\"  |\"VIDEO\"  |\"EMAIL\"  |\"PREDICTED_ROI\"     |\n",
      "-----------------------------------------------------------------------------\n",
      "|8500             |9500            |2000     |500      |233784.84401053237  |\n",
      "|250000           |250000          |200000   |450000   |333166.0182359633   |\n",
      "|500000           |500000          |500000   |500000   |3178590.4246200835  |\n",
      "-----------------------------------------------------------------------------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "test_df.select(\n",
    "    'SEARCH_ENGINE','SOCIAL_MEDIA','VIDEO','EMAIL', \n",
    "    call_udf(\"batch_predict_roi\", \n",
    "    col(\"SEARCH_ENGINE\"), col(\"SOCIAL_MEDIA\"), col(\"VIDEO\"), col(\"EMAIL\")).as_(\"PREDICTED_ROI\")).show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "951e3902",
   "metadata": {},
   "source": [
    "_For comments and feedback, please reach out to dash.desai@snowflake.com | Follow on [Twitter](https://twitter.com/iamontheinet)_ "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.13 ('snowpark')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.15"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 41.012855,
   "end_time": "2021-05-15T09:34:14.646782",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2021-05-15T09:33:33.633927",
   "version": "2.3.3"
  },
  "vscode": {
   "interpreter": {
    "hash": "4fb0a37530c0004d75c43dbcefc0b8b6ea2fdc6f87c96f7fd6f8cf43b3f551c7"
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "03486abb248b48dcae43183c9ffb112a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "0539c58479b749dd99898d49f43ebcef": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_d097f3c35ec644bfa231be2d2fd50129",
       "max": 1,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_0be3e122c58347b79c67cbdfc6939065",
       "value": 1
      }
     },
     "0be3e122c58347b79c67cbdfc6939065": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "0debf43d9ebe494fa0871bc085432a31": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_7ea526b1e9bd43cdb270777b5f62a95e",
        "IPY_MODEL_0539c58479b749dd99898d49f43ebcef",
        "IPY_MODEL_5f80a955ced34cef86bc307a7a17ad50"
       ],
       "layout": "IPY_MODEL_4cea8ab8f6094c87a13a8b690c87c778"
      }
     },
     "106409d46bfe43f19cfedaa975ba2cc8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_79579de4686940fb87277980fb3b9033",
        "IPY_MODEL_1ee9a30ab6f14e9f890b0bc1aabad3db",
        "IPY_MODEL_60a533bb74804344b7e871fbf50d7630"
       ],
       "layout": "IPY_MODEL_b810367f103a4e4e9c5be322df67d4c9"
      }
     },
     "137e5d876fda44a481e04a46ce3d52c5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1ee9a30ab6f14e9f890b0bc1aabad3db": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_896278c20baf49bc9483c86dbe28e6db",
       "max": 1,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_214ce6435877469e8aec9b089f0a5b2b",
       "value": 1
      }
     },
     "214ce6435877469e8aec9b089f0a5b2b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "22828e38a6f84e388c66ed37fd49e1a3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "2476936f36354c0582cf209811858769": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "3047235624bd4df4b992d459afbad807": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3c0e879486d5463db0fa09977f4f5e7b": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "421538af4b344eb285a1049283fceb43": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4cea8ab8f6094c87a13a8b690c87c778": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "59f46be901ef49cea5e361a23bc258f0": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "5f80a955ced34cef86bc307a7a17ad50": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_c800f4d86d7f4cb595f97bdd36d14c8f",
       "placeholder": "​",
       "style": "IPY_MODEL_ebd7ac3b623a410a88be5c4d72e48986",
       "value": " 1/1 [00:00&lt;00:00,  1.66it/s]"
      }
     },
     "5faebb75f07148fca515e5e4d72a9e26": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "60a533bb74804344b7e871fbf50d7630": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3c0e879486d5463db0fa09977f4f5e7b",
       "placeholder": "​",
       "style": "IPY_MODEL_98f6f531ccd643d09a69140d13b76379",
       "value": " 1/1 [00:02&lt;00:00,  2.40s/it]"
      }
     },
     "6379beb1c16740269e13f8731b3b5f8b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_837225552aef4717b0ab001ff292f72d",
        "IPY_MODEL_997750a67ac645548adf9d6cc5a2c1dd",
        "IPY_MODEL_bd972d756dfe46978be7bb973f690392"
       ],
       "layout": "IPY_MODEL_ab6133b8ef4f45cfa84344f070d4447f"
      }
     },
     "731d09ee1159493095283eac20e86c4d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_5faebb75f07148fca515e5e4d72a9e26",
       "max": 17,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_2476936f36354c0582cf209811858769",
       "value": 17
      }
     },
     "74d2f123669f4151bf9fd59456f355ba": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_a06bfab18f1d49a6a4c0a87c360992c8",
        "IPY_MODEL_731d09ee1159493095283eac20e86c4d",
        "IPY_MODEL_cbd80b044afc4e96b1667ab76a3a1f74"
       ],
       "layout": "IPY_MODEL_7a11aa9b2ecb445a8fc14cc89f06ea13"
      }
     },
     "79579de4686940fb87277980fb3b9033": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_b0c4c0c11c45494db725902cd5bfb08c",
       "placeholder": "​",
       "style": "IPY_MODEL_f53f890fe98e493a96efcc400345b6b2",
       "value": "Generate report structure: 100%"
      }
     },
     "7a11aa9b2ecb445a8fc14cc89f06ea13": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7ea526b1e9bd43cdb270777b5f62a95e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ca8a709aac3f4afe9aac06b72112076f",
       "placeholder": "​",
       "style": "IPY_MODEL_c33355f13dfd4f4599031ef8f8db5f53",
       "value": "Render HTML: 100%"
      }
     },
     "837225552aef4717b0ab001ff292f72d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_59f46be901ef49cea5e361a23bc258f0",
       "placeholder": "​",
       "style": "IPY_MODEL_a4c35dfafabb40199a5c4c0ce5a14b0c",
       "value": "Export report to file: 100%"
      }
     },
     "896278c20baf49bc9483c86dbe28e6db": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "98f6f531ccd643d09a69140d13b76379": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "997750a67ac645548adf9d6cc5a2c1dd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3047235624bd4df4b992d459afbad807",
       "max": 1,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_f71a3435837f4fdd8eee21023f928dc2",
       "value": 1
      }
     },
     "a06bfab18f1d49a6a4c0a87c360992c8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_137e5d876fda44a481e04a46ce3d52c5",
       "placeholder": "​",
       "style": "IPY_MODEL_a757f29997924f8d97c5b33d53a151cd",
       "value": "Summarize dataset: 100%"
      }
     },
     "a4c35dfafabb40199a5c4c0ce5a14b0c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "a757f29997924f8d97c5b33d53a151cd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "ab6133b8ef4f45cfa84344f070d4447f": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b0c4c0c11c45494db725902cd5bfb08c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b810367f103a4e4e9c5be322df67d4c9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "bd972d756dfe46978be7bb973f690392": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_22828e38a6f84e388c66ed37fd49e1a3",
       "placeholder": "​",
       "style": "IPY_MODEL_ca9b1619f3f348178c14a36e88ae3d9b",
       "value": " 1/1 [00:00&lt;00:00, 33.29it/s]"
      }
     },
     "c33355f13dfd4f4599031ef8f8db5f53": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "c800f4d86d7f4cb595f97bdd36d14c8f": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ca8a709aac3f4afe9aac06b72112076f": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ca9b1619f3f348178c14a36e88ae3d9b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "cbd80b044afc4e96b1667ab76a3a1f74": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_421538af4b344eb285a1049283fceb43",
       "placeholder": "​",
       "style": "IPY_MODEL_03486abb248b48dcae43183c9ffb112a",
       "value": " 17/17 [00:05&lt;00:00,  2.09it/s, Completed]"
      }
     },
     "d097f3c35ec644bfa231be2d2fd50129": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ebd7ac3b623a410a88be5c4d72e48986": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "f53f890fe98e493a96efcc400345b6b2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "f71a3435837f4fdd8eee21023f928dc2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}