VPP_simulator-efan-20221229.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kTp62tn8pK5R"
   },
   "source": [
    "# Reinforcement Learning control strategies for Electric Vehicles fleet Virtual Power Plants\n",
    "Thesis based on the development of a RL agent that manages a VPP through EVs charging stations in an household environment. Main optimization objectives of the VPP are: Valley filling, peak shaving and zero resulting load over time. Main action performed to reach objectives are: storage of Renewable energy resources and power push in the grid at high demand times. The development of the Virtual Power Plant environment is based on the ELVIS (Electric Vehicles Infrastructure Simulator) open library from DAI-Labor: https://github.com/dailab/elvis The thesis code is currently available at: (https://github.com/francescomaldonato/RL_VPP_Thesis)\n",
    "\n",
    "Author: Francesco Maldonato\n",
    "\n",
    "## VPP simulator Notebook based on EVs arrival, with StableBaselines3 trained model\n",
    "(Recurrent PPO) [default: 25 EVs per week simulation]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g7NT2ScfCgZp"
   },
   "source": [
    "Installing required packages and dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 93710,
     "status": "ok",
     "timestamp": 1666456934135,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "E_SYbpdxyUWs"
   },
   "outputs": [],
   "source": [
    "# %%capture\n",
    "# !pip install py-elvis==0.2.1\n",
    "# !pip install pyyaml==5.4\n",
    "# !pip install plotly==5.9.0\n",
    "# !pip install -U kaleido==0.2.1\n",
    "\n",
    "# !pip install stable-baselines3[extra]==1.6.1\n",
    "# !pip install stable-baselines==1.6.1\n",
    "# !pip install sb3-contrib==1.6.1\n",
    "# !pip install gym==0.20.0\n",
    "# !pip install -q wandb==0.13.4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 458059,
     "status": "ok",
     "timestamp": 1666457392152,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "eMPWLVOVhlmL",
    "outputId": "2b439915-b3d3-4f18-e2bf-2a4d09c4fab3",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# from google.colab import drive\n",
    "# drive.mount('/content/drive')\n",
    "\n",
    "# %cd /content/drive/MyDrive/DAI-Labor/RL_VPP_Thesis/\n",
    "# %ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1666457392153,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "N1jW0iIkMXps"
   },
   "outputs": [],
   "source": [
    "# #Cloning repository and changing directory\n",
    "# !git clone https://github.com/francescomaldonato/RL_VPP_Thesis.git\n",
    "# %cd RL_VPP_Thesis/\n",
    "# %ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 6706,
     "status": "ok",
     "timestamp": 1666457398855,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "peQMgy_DMFSQ",
    "outputId": "13b6c680-2605-4b47-fcf7-fd2af730f87b"
   },
   "outputs": [],
   "source": [
    "import yaml\n",
    "import numpy as np\n",
    "from gym import Env\n",
    "from VPP_environment import VPPEnv, VPP_Scenario_config\n",
    "from elvis.config import ScenarioConfig\n",
    "import os\n",
    "import torch\n",
    "import random\n",
    "import wandb\n",
    "from sb3_contrib import RecurrentPPO #The available algoritmhs in sb3-contrib for the custom environment with MultiInputPolicy\n",
    "from sb3_contrib.common.maskable.utils import get_action_masks\n",
    "import stable_baselines3 as sb3\n",
    "from stable_baselines3.common.env_checker import check_env\n",
    "\n",
    "import plotly.io as pio  # 我加的，不知道为啥画不出来图了\n",
    "pio.renderers.default = \"notebook_connected\"  # 我加的，不知道为啥画不出来图了\n",
    "\n",
    "\n",
    "#Check if cuda device is available for training\n",
    "print(\"Torch-Cuda available device:\", torch.cuda.is_available())\n",
    "print(sb3.get_system_info())\n",
    "#!wandb --version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 14,
     "status": "ok",
     "timestamp": 1666457398857,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "bnJqlGkhe-A5"
   },
   "outputs": [],
   "source": [
    "# 确定性结果可复现  Ensure deterministic behavior\n",
    "torch.backends.cudnn.deterministic = True\n",
    "random.seed(0)\n",
    "torch.manual_seed(0)\n",
    "torch.cuda.manual_seed_all(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "URGrR_15MGNP"
   },
   "source": [
    "## Load ELVIS YAML config file\n",
    "Section where the EVs arrival simulation parameters are loaded through the Yaml config file from the 'data/config_builder/' folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 455,
     "status": "ok",
     "timestamp": 1666457399301,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "gRb71nAZ-LK1",
    "outputId": "93288a99-0fea-48e9-a8be-3c3f4bd2de03"
   },
   "outputs": [],
   "source": [
    "#Loading paths for input data\n",
    "current_folder = ''  # 如果是.py调用这里需要改成'/home/ef/Documents/code/ubuntu/Car/RL_VPP_Thesis-main/'\n",
    "VPP_training_data_input_path = current_folder + 'data/data_training/environment_table/' + 'Environment_data_2019.csv'\n",
    "VPP_testing_data_input_path = current_folder + 'data/data_testing/environment_table/' + 'Environment_data_2020.csv'\n",
    "VPP_validating_data_input_path = current_folder + 'data/data_validating/environment_table/' + 'Environment_data_2018.csv'\n",
    "elvis_input_folder = current_folder + 'data/config_builder/'\n",
    "\n",
    "#case = 'wohnblock_household_simulation_adaptive.yaml' #(loaded by default, 20 EVs arrivals per week with 50% average battery)\n",
    "\n",
    "#Try different simulation parameters, uncomment below\n",
    "#case = 'wohnblock_household_simulation_adaptive_10.yaml' #(10 EVs arrivals per week with 50% average battery) \n",
    "#case = 'wohnblock_household_simulation_adaptive_15.yaml' #(15 EVs arrivals per week with 50% average battery)\n",
    "case = 'wohnblock_household_simulation_adaptive_25.yaml' #(25 EVs arrivals per week with 50% average battery) \n",
    "#case = 'wohnblock_household_simulation_adaptive_30.yaml' #(30 EVs arrivals per week with 50% average battery) \n",
    "#case = 'wohnblock_household_simulation_adaptive_35.yaml' #(35 EVs arrivals per week with 50% average battery) \n",
    "\n",
    "with open(elvis_input_folder + case, 'r') as file:\n",
    "    yaml_str = yaml.full_load(file)\n",
    "\n",
    "elvis_config_file = ScenarioConfig.from_yaml(yaml_str)  # 要生成的场景信息-EV\n",
    "VPP_config_file = VPP_Scenario_config(yaml_str)  # VPP应该指的是整个优化场景\n",
    "\n",
    "# print(elvis_config_file)  # 占用时间\n",
    "# print(VPP_config_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TESTING Environment initialization\n",
    "env = VPPEnv(VPP_testing_data_input_path, elvis_config_file, VPP_config_file)  # elvis_config_file输入EV参数，调用Elvis返回的EV分布\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.plot_Dataset_autarky()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 717,
     "output_embedded_package_id": "18Kk3rBku6I40rDEoDT92C1HViOrTJ8Or"
    },
    "executionInfo": {
     "elapsed": 7360,
     "status": "ok",
     "timestamp": 1666457408638,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "s1xM-TTBjiod",
    "outputId": "6cd5ba99-61be-4290-8297-cc3c8587d739"
   },
   "outputs": [],
   "source": [
    "# env.plot_VPP_input_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 567
    },
    "executionInfo": {
     "elapsed": 10,
     "status": "ok",
     "timestamp": 1666457408639,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "OJLc0Zcue-A7",
    "outputId": "0885ead0-b302-4e13-92cd-fe24f9ce4cc9"
   },
   "outputs": [],
   "source": [
    "# env.plot_ELVIS_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 806
    },
    "executionInfo": {
     "elapsed": 862,
     "status": "ok",
     "timestamp": 1666457409492,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "rI_79t9_8gXw",
    "outputId": "992cc232-2a66-40ae-80ab-3ca30c507200"
   },
   "outputs": [],
   "source": [
    "#Function to check custom environment and output additional warnings if needed\n",
    "# check_env(env)  # baseline带的  检查环境是否遵循 Gym API\n",
    "# env.plot_reward_functions()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xyeqcbRcdvDL"
   },
   "source": [
    "### VPP Simulation test with random actions [no model loaded]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 761,
     "output_embedded_package_id": "1uratSXq6WAWHXXxeT9q7SYp7wpaI4oEU"
    },
    "executionInfo": {
     "elapsed": 18455,
     "status": "ok",
     "timestamp": 1666457427939,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "L7YJpamuceMR",
    "outputId": "f012ccb6-5681-4438-c2f5-55d35a5e8390"
   },
   "outputs": [],
   "source": [
    "episodes = 1\n",
    "for episode in range(1, episodes+1):\n",
    "    state = env.reset()\n",
    "    done = False  # 此done非self.done\n",
    "    score = 0\n",
    "    while not done:  # 执行35041次\n",
    "        action_masks = get_action_masks(env)  # [闲置*4, 充*4, ,放*4]值为True或False\n",
    "        action = env.action_space.sample()  # [3,3,3,3]\n",
    "        \n",
    "        n_state, reward, done, info = env.step(action)\n",
    "        score+=reward\n",
    "    print('Episode:{} Score:{}'.format(episode, score))\n",
    "\n",
    "VPP_table = env.VPP_table\n",
    "# env.plot_VPP_energies()  # EV电池变化曲线"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 12,
     "status": "ok",
     "timestamp": 1666457427939,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "P_Rvmi1_dvDM"
   },
   "outputs": [],
   "source": [
    "# env.plot_Elvis_results()  # 买电消费、EV功率、住宅还需负载、总负载曲线, 与下面的对比.   住宅+EV负载=总负载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 12,
     "status": "ok",
     "timestamp": 1666457427940,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "HswyajC0dvDN"
   },
   "outputs": [],
   "source": [
    "# env.plot_VPP_results()  # 由RL agent控制的充电行动的VPP仿真结果（35041步结束后记录,由houseRW_load, ev_power and overcost叠加的输入数据集）动作的比较乱。RL模型控制的就不乱"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 817
    },
    "executionInfo": {
     "elapsed": 11,
     "status": "ok",
     "timestamp": 1666457427940,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "LEoDS1KLj0to",
    "outputId": "47bdb08b-0bcd-4d50-a709-76c827874adb"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_autarky()  # 自消纳和自给自足对比:Elvis非控制性充电仿真和有控制性充电行动的VPP仿真(这里是随机动作)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 767,
     "output_embedded_package_id": "1ADG0eI8G7ySWSC_9i78_dWJDhYQhiQDM"
    },
    "executionInfo": {
     "elapsed": 23255,
     "status": "ok",
     "timestamp": 1666457451187,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "ZrhN2PopXLWn",
    "outputId": "0511002c-a535-4639-827b-3ff5801d288f"
   },
   "outputs": [],
   "source": [
    "# env.plot_VPP_supply_demand()  # 时间序列下的数据  正:#家庭负荷来源 ?2house #EV充电来源 ?2EV #消费实体 CO_?     # 负:#使用新能源的功率 RE2? #使用EV放电的功率 EV2? #产生的能源 PRO_?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 517
    },
    "executionInfo": {
     "elapsed": 22,
     "status": "ok",
     "timestamp": 1666457451189,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "YxDAjhREe-A-"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_Elvis_comparison()  # Elvis和VPP对比:EV离开时的平均能量、买电量、新能源给EV未使用的部分、买电费用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 617,
     "output_embedded_package_id": "13S6yAbMCfpfSpCnTWkirWSGMdHvXR38o"
    },
    "executionInfo": {
     "elapsed": 3737,
     "status": "ok",
     "timestamp": 1666457454908,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "WQFkpSnEe-A_",
    "outputId": "b698e3f2-bb02-4d01-998b-6f59c323f1d6"
   },
   "outputs": [],
   "source": [
    "# env.plot_rewards_results()  # VPP仿真中，EV奖励(每一步的total load，离开时电动车剩余电量, 可用EV数)随时间推移的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "executionInfo": {
     "elapsed": 53,
     "status": "ok",
     "timestamp": 1666457454909,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "DwMlfFwRe-A_"
   },
   "outputs": [],
   "source": [
    "env.plot_rewards_stats()  # 绘制奖励统计 见代码解释,cumulative = final_reward + step_total, step_total = step_EV_en + step_load"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 717
    },
    "executionInfo": {
     "elapsed": 54,
     "status": "ok",
     "timestamp": 1666457454910,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "SXcSxj0ye-BA"
   },
   "outputs": [],
   "source": [
    "env.plot_EVs_kpi()  # 在VPP仿真中，EV离开时能量区间计数图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 517
    },
    "executionInfo": {
     "elapsed": 52,
     "status": "ok",
     "timestamp": 1666457454910,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "mJkgLa0ve-BA"
   },
   "outputs": [],
   "source": [
    "env.plot_actions_kpi()  # 在VPP仿真中，agent所采取行动是否有效的热图。action_truth_table[T,T,T,F]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 817
    },
    "executionInfo": {
     "elapsed": 46,
     "status": "ok",
     "timestamp": 1666457454911,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "8sBOOL28e-BA"
   },
   "outputs": [],
   "source": [
    "env.plot_load_kpi()  # 取一周2022-01-01~2022-01-08、一月2022-06-01~2022-07-01、一年的负载数据柱状图分析, ELVIS和VPP对比"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 717,
     "output_embedded_package_id": "1kWGbKjzW3LInrzpAAeT67TapXKCf_Wx8"
    },
    "executionInfo": {
     "elapsed": 38,
     "status": "ok",
     "timestamp": 1666457454911,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "5aWnnCVve-BA",
    "outputId": "22553b4c-67fe-44ef-8aee-67b757ab8990"
   },
   "outputs": [],
   "source": [
    "env.plot_yearly_load_log()  # 按负载功率0.2kW为区间长度对step计数(年,叠加),画\"对数\"+直方图,对比ELVIS和VPP. 计算负载在-0.1~0.1区间的step百分比"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "executionInfo": {
     "elapsed": 25,
     "status": "ok",
     "timestamp": 1666457454912,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "UIpvXHb748le"
   },
   "outputs": [],
   "source": [
    "## Wandb login to load models\n",
    "#In Colab, uncomment below:\n",
    "# %env \"WANDB_DISABLE_CODE\" True\n",
    "# %env \"WANDB_NOTEBOOK_NAME\" \"VPP_simulator.ipynb\"\n",
    "# os.environ['WANDB_NOTEBOOK_NAME'] = 'VPP_simulator.ipynb'\n",
    "#wandb.login(relogin=True)\n",
    "\n",
    "#In local notebook, uncomment below:\n",
    "your_wandb_login_code = \"7bbad3c9c98621391634ef832a6c90f4e34f14b6\" #example length\n",
    "!wandb login {your_wandb_login_code}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 24,
     "status": "ok",
     "timestamp": 1666457454912,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "vAKJF5VDdvDO"
   },
   "outputs": [],
   "source": [
    "#Loading training model, from local directory or from wandb previous trainings\n",
    "RecurrentPPO_path = \"trained_models/RecurrentPPO_models/model_RecurrentPPO_\"\n",
    "\n",
    "#model_id = \"s37o8q0n\"\n",
    "model_id = \"333ckz0i\"\n",
    "model = RecurrentPPO.load(RecurrentPPO_path + model_id, env=env)  # 用 \"Monitor \"包装器来包装env,用DummyVecEnv来包装env。\n",
    "\n",
    "# run_id_restore = \"2y2dqvyn\"\n",
    "# model = wandb.restore(f'model_{run_id_restore}.zip', run_path=f\"francesco_maldonato/RL_VPP_Thesis/{run_id_restore}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q8gf6s79dvDP"
   },
   "source": [
    "## Testing dataset VPP Simulation using the loaded trained model\n",
    "测试数据集VPP仿真  使用加载的训练模型 #现在使用我们训练有素的模型进行确定性的预测[应该可以提高性能]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 58566,
     "status": "ok",
     "timestamp": 1666457513461,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "dFOU30oVt4Ri",
    "outputId": "7319e1dc-6757-490d-cd00-bd50ec97b168"
   },
   "outputs": [],
   "source": [
    "#TEST Model\n",
    "episodes = 1\n",
    "for episode in range(1, episodes+1):\n",
    "    obs = env.reset()\n",
    "    done = False\n",
    "    score = 0\n",
    "    # cell and hidden state of the LSTM\n",
    "    lstm_states = None\n",
    "    num_envs = 1\n",
    "    # Episode start signals are used to reset the lstm states\n",
    "    episode_starts = np.ones((num_envs,), dtype=bool)\n",
    "    while not done:\n",
    "        #env.render()\n",
    "        action_masks = get_action_masks(env)\n",
    "        action, lstm_states = model.predict(obs, state=lstm_states, episode_start=episode_starts, deterministic=True) #现在使用我们训练有素的模型进行确定性的预测[应该可以提高性能] Now using our trained model with deterministic prediction [should improve performances]\n",
    "        env.lstm_state = lstm_states\n",
    "        obs, reward, done, info = env.step(action)\n",
    "        episode_starts = done\n",
    "        score+=reward\n",
    "    print('Episode:{} Score:{}'.format(episode, score))\n",
    "\n",
    "#Save the VPP table\n",
    "VPP_table = env.save_VPP_table(save_path='data/environment_optimized_output/VPP_table.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 567,
     "output_embedded_package_id": "1yEem-pYCgavCwOUNCJ7UGJSEwMnVa0EZ"
    },
    "executionInfo": {
     "elapsed": 10495,
     "status": "ok",
     "timestamp": 1666457523949,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "8lXyEaa5e-BC",
    "outputId": "0b6cf82f-44a8-4219-e400-c7f26fe52e21"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_energies()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 455
    },
    "executionInfo": {
     "elapsed": 20,
     "status": "ok",
     "timestamp": 1666457523950,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "niwjvDS6YaV9",
    "outputId": "694759e4-5823-4146-a509-fa25a57c2d74"
   },
   "outputs": [],
   "source": [
    "VPP_table.head(15000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 17,
     "status": "ok",
     "timestamp": 1666457523950,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "wAx2MFUgdvDQ"
   },
   "outputs": [],
   "source": [
    "# env.plot_Elvis_results()  # 买电消费、EV功率、住宅还需负载、总负载曲线, 与下面的对比.   住宅+EV负载=总负载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 18,
     "status": "ok",
     "timestamp": 1666457523951,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "tsnj-3oQdvDQ"
   },
   "outputs": [],
   "source": [
    "# env.plot_VPP_results()  # 买电消费、EV功率、住宅还需负载、总负载（35041步结束后记录,由houseRW_load, ev_power and overcost叠加的输入数据集）.由随机不控制的动作的比较乱。RL模型控制的就不乱."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 817
    },
    "executionInfo": {
     "elapsed": 18,
     "status": "ok",
     "timestamp": 1666457523952,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "Vho7mEX6kHo3",
    "outputId": "06510ded-22d5-432d-b204-91ea13e6f666"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_autarky()  # 自消纳风光率、自给自足率无需买电对比:Elvis非控制性充电仿真和有控制性充电行动的VPP仿真(这里是RL控制的动作)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 767,
     "output_embedded_package_id": "1VZW6xnGdYOI1wqzG0sr95KsxagpvTgKs"
    },
    "executionInfo": {
     "elapsed": 24240,
     "status": "ok",
     "timestamp": 1666457548175,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "Rn93YDolXyDF",
    "outputId": "49e3ed60-6fdb-4589-8979-aaab1da20fd6"
   },
   "outputs": [],
   "source": [
    "# env.plot_VPP_supply_demand()  #  超多组  时间序列下的数据  正:#家庭负荷来源 ?2house #EV充电来源 ?2EV #消费实体 CO_?     # 负:#使用新能源的功率 RE2? #使用EV放电的功率 EV2? #产生的能源 PRO_?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 517
    },
    "executionInfo": {
     "elapsed": 13,
     "status": "ok",
     "timestamp": 1666457548176,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "tDvDx4_Se-BD"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_Elvis_comparison()  # Elvis和VPP对比:EV离开时的平均能量、买电量、新能源给EV未使用的部分、买电费用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 617,
     "output_embedded_package_id": "1aO1mVOd9sapQYItzop-1cioAy5SmloOs"
    },
    "executionInfo": {
     "elapsed": 2927,
     "status": "ok",
     "timestamp": 1666457551092,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "ImrOOG80e-BD",
    "outputId": "fcc73880-085c-4aa4-95d4-03c79fe6a5ed"
   },
   "outputs": [],
   "source": [
    "env.plot_rewards_results()  # VPP仿真中，EV奖励(每一步的total load，离开时电动车剩余电量, 可用EV数)随时间推移的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 517
    },
    "executionInfo": {
     "elapsed": 56,
     "status": "ok",
     "timestamp": 1666457551092,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "cdD8oYdge-BD"
   },
   "outputs": [],
   "source": [
    "env.plot_rewards_stats()  # 绘制奖励统计 见代码解释,cumulative = final_reward + step_total, step_total = step_EV_en + step_load"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 717
    },
    "executionInfo": {
     "elapsed": 55,
     "status": "ok",
     "timestamp": 1666457551093,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "uQ3TJneSe-BE"
   },
   "outputs": [],
   "source": [
    "env.plot_EVs_kpi()  # 在VPP仿真中，EV离开时能量区间计数图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 817
    },
    "executionInfo": {
     "elapsed": 55,
     "status": "ok",
     "timestamp": 1666457551094,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "9Uwo1Tcae-BE"
   },
   "outputs": [],
   "source": [
    "env.plot_load_kpi()  # 取一周2022-01-01~2022-01-08、一月2022-06-01~2022-07-01、一年的负载数据柱状图分析, ELVIS和VPP对比"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 717,
     "output_embedded_package_id": "1LXPI9Jtp-APOjApQOjYL0LSu9xGkQnxt"
    },
    "executionInfo": {
     "elapsed": 46,
     "status": "ok",
     "timestamp": 1666457551094,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "YrmJbMKWe-BE",
    "outputId": "23d4b9e1-3071-4b7b-aa0e-56e114874d1b"
   },
   "outputs": [],
   "source": [
    "env.plot_yearly_load_log()  # 按负载功率0.2kW为区间长度对step计数(年,叠加),画\"对数\"+直方图,对比ELVIS和VPP. 计算负载在-0.1~0.1区间的step百分比"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u5veXJ2YrXEJ"
   },
   "source": [
    "## Validating dataset VPP Simulation using the loaded trained model\n",
    "验证数据集VPP仿真，使用加载的训练模型(上面一组是测试)  #现在使用我们训练有素的模型进行确定性预测[应该会提高性能]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 24,
     "status": "ok",
     "timestamp": 1666457551095,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "35khJcQlqPuO"
   },
   "outputs": [],
   "source": [
    "#VALIDATING Environment initialization\n",
    "env = VPPEnv(VPP_validating_data_input_path, elvis_config_file, VPP_config_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 806,
     "output_embedded_package_id": "1oq2YlufACP97n0g28AUITF0AHgCbC9Xu"
    },
    "executionInfo": {
     "elapsed": 4596,
     "status": "ok",
     "timestamp": 1666457555673,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "zfbTOZNVrVRQ",
    "outputId": "a4011a3a-c7b0-47d1-bec5-76e10c5bb3bf"
   },
   "outputs": [],
   "source": [
    "#Function to check custom environment and output additional warnings if needed\n",
    "check_env(env)\n",
    "plot_VPP_input_data = env.plot_VPP_input_data()\n",
    "plot_VPP_input_data.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 13,
     "status": "ok",
     "timestamp": 1666457555674,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "S41TtfUEsHQo"
   },
   "outputs": [],
   "source": [
    "#model = PPO.load(PPO_path + model_run_ID, env = env)\n",
    "model = RecurrentPPO.load(RecurrentPPO_path + model_id, env=env)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 63282,
     "status": "ok",
     "timestamp": 1666457618948,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "Jiy7uEFzsG4w"
   },
   "outputs": [],
   "source": [
    "#TEST Model\n",
    "episodes = 1\n",
    "for episode in range(1, episodes+1):\n",
    "    obs = env.reset()\n",
    "    done = False\n",
    "    score = 0\n",
    "    # cell and hidden state of the LSTM\n",
    "    lstm_states = None\n",
    "    num_envs = 1\n",
    "    # Episode start signals are used to reset the lstm states\n",
    "    episode_starts = np.ones((num_envs,), dtype=bool)\n",
    "    while not done:\n",
    "        #env.render()\n",
    "        action_masks = get_action_masks(env)\n",
    "        action, lstm_states = model.predict(obs, state=lstm_states, episode_start=episode_starts, deterministic=True) #现在使用我们训练有素的模型进行确定性预测[应该会提高性能] Now using our trained model with deterministic prediction [should improve performances]\n",
    "        env.lstm_state = lstm_states\n",
    "        obs, reward, done, info = env.step(action)\n",
    "        episode_starts = done\n",
    "        score+=reward\n",
    "    print('Episode:{} Score:{}'.format(episode, score))\n",
    "\n",
    "VPP_table = env.VPP_table\n",
    "#print(env.lstm_states_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 567,
     "output_embedded_package_id": "1AD0Bw9PKdG9rUCdkTDAug9Fgdnxo1hqM"
    },
    "executionInfo": {
     "elapsed": 9993,
     "status": "ok",
     "timestamp": 1666457628927,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "yRLbO9xMe-BF",
    "outputId": "76165e57-dbe1-4467-b6e2-300b46f7d2c5"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_energies()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 455
    },
    "executionInfo": {
     "elapsed": 17,
     "status": "ok",
     "timestamp": 1666457628928,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "fjHr9gEfYXmC"
   },
   "outputs": [],
   "source": [
    "VPP_table.head(15000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "executionInfo": {
     "elapsed": 16,
     "status": "ok",
     "timestamp": 1666457628929,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "trYyI8I-qbXa"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_results()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "base_uri": "https://localhost:8080/",
     "height": 817
    },
    "executionInfo": {
     "elapsed": 17,
     "status": "ok",
     "timestamp": 1666457628930,
     "user": {
      "displayName": "Francesco Maldonato",
      "userId": "02511619010385133435"
     },
     "user_tz": -120
    },
    "id": "vGD9iEyFkU2t"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_autarky()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "output_embedded_package_id": "1H-BDAMY5gQghtJlsAbRMoN9gluxFd3Jz"
    },
    "id": "vjr6pajiX2Al",
    "outputId": "99a376af-f272-480e-8d14-fa1fafeba7ed"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_supply_demand()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "TnnijDgce-BG"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_Elvis_comparison()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "output_embedded_package_id": "1XRMf_C2JjMikp0CeS1ck71UFXfO2z7MK"
    },
    "id": "1Jx_H2M4e-BH",
    "outputId": "90e3ce6c-eb36-446b-b9c9-8d0352040943"
   },
   "outputs": [],
   "source": [
    "env.plot_rewards_results()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "BOQqHAcVe-BH"
   },
   "outputs": [],
   "source": [
    "env.plot_rewards_stats()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "G9mFndUMe-BH"
   },
   "outputs": [],
   "source": [
    "env.plot_EVs_kpi()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "wdNgTMv5e-BH"
   },
   "outputs": [],
   "source": [
    "env.plot_load_kpi()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "output_embedded_package_id": "1M1Ue0f8QHaQb-tc7MYV1kD4EByVip5YG"
    },
    "id": "ldE51e9Xe-BH",
    "outputId": "976a6dac-1534-419e-9088-4162f7adf688"
   },
   "outputs": [],
   "source": [
    "env.plot_yearly_load_log()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hl8IGpBItG8v"
   },
   "source": [
    "## Training dataset VPP Simulation using the loaded trained model\n",
    "训练数据集 使用加载的训练模型进行VPP模拟"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "EjXekmbGtJoP"
   },
   "outputs": [],
   "source": [
    "#TRAINING Environment initialization\n",
    "env = VPPEnv(VPP_training_data_input_path, elvis_config_file, VPP_config_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "output_embedded_package_id": "1-RQ1LdKrOsyXKU70rAsCcQH3xec7rclT"
    },
    "id": "k0ju86m3tUZB",
    "outputId": "6cac616f-fbbf-4bc6-bba6-52fb8fd28176"
   },
   "outputs": [],
   "source": [
    "#Function to check custom environment and output additional warnings if needed\n",
    "check_env(env)\n",
    "plot_VPP_input_data = env.plot_VPP_input_data()\n",
    "plot_VPP_input_data.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "TUzb7bCKtUuX"
   },
   "outputs": [],
   "source": [
    "#model = PPO.load(PPO_path + model_run_ID, env = env)\n",
    "model = RecurrentPPO.load(RecurrentPPO_path + model_id, env=env)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "Xa4AIRwktqlH"
   },
   "outputs": [],
   "source": [
    "#TEST Model\n",
    "episodes = 1\n",
    "for episode in range(1, episodes+1):\n",
    "    obs = env.reset()\n",
    "    done = False\n",
    "    score = 0\n",
    "    # cell and hidden state of the LSTM\n",
    "    lstm_states = None\n",
    "    num_envs = 1\n",
    "    # Episode start signals are used to reset the lstm states\n",
    "    episode_starts = np.ones((num_envs,), dtype=bool)\n",
    "    while not done:\n",
    "        #env.render()\n",
    "        action_masks = get_action_masks(env)\n",
    "        action, lstm_states = model.predict(obs, state=lstm_states, episode_start=episode_starts, deterministic=True) #Now using our trained model with deterministic prediction [should improve performances]\n",
    "        env.lstm_state = lstm_states\n",
    "        obs, reward, done, info = env.step(action)\n",
    "        episode_starts = done\n",
    "        score+=reward\n",
    "    print('Episode:{} Score:{}'.format(episode, score))\n",
    "\n",
    "VPP_table = env.VPP_table\n",
    "#print(env.lstm_states_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "output_embedded_package_id": "1rUzYxrHs3a945r9yGeF1PvrvkiS8EBc4"
    },
    "id": "ccro1Osze-BI",
    "outputId": "e3525514-a8ec-495e-9838-9709bcfb0296"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_energies()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "jPuBuR6eYBoH"
   },
   "outputs": [],
   "source": [
    "VPP_table.head(14995)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "DhEAmAost_C7"
   },
   "outputs": [],
   "source": [
    "#env.plot_VPP_results()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "RJMLjSCRkeO6"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_autarky()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "output_embedded_package_id": "1cQFct9jYdFd3-iR5D11ee1_OH_BMIFMy"
    },
    "id": "HSCK9TeMX5Sg",
    "outputId": "d381f21c-3496-49dd-9610-ee20107ac04a"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_supply_demand()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "qgxTLfv5e-BJ"
   },
   "outputs": [],
   "source": [
    "env.plot_VPP_Elvis_comparison()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "output_embedded_package_id": "18xe6TDzPd6Hl8Q1zwJeeK7hp3nG42pp-"
    },
    "id": "_TbzTL-ne-BJ",
    "outputId": "0304f255-e824-4db0-8236-5b38816b67da"
   },
   "outputs": [],
   "source": [
    "env.plot_rewards_results()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "0erLmvL6e-BK"
   },
   "outputs": [],
   "source": [
    "env.plot_rewards_stats()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "f2DARfA1e-BK"
   },
   "outputs": [],
   "source": [
    "env.plot_EVs_kpi()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "Ky_0HDIoe-BK"
   },
   "outputs": [],
   "source": [
    "env.plot_actions_kpi()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "FZlYczyke-BK"
   },
   "outputs": [],
   "source": [
    "env.plot_load_kpi()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true,
     "output_embedded_package_id": "1y-vxbpNkBKJ6DwHtRwVdFMHr95sTnaFN"
    },
    "id": "MbCcqHYWe-BK",
    "outputId": "e9fdf8df-c8f4-46c6-aa93-e62f87d12934"
   },
   "outputs": [],
   "source": [
    "env.plot_yearly_load_log()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "IMf4k5g8KFE7"
   },
   "outputs": [],
   "source": [
    "env.close()"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "machine_shape": "hm",
   "name": "",
   "version": ""
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3.7.13 ('RL3')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "7864169697d4ddc3dd4537d4d828a110971387e0f68e6d64f92dcaa80830ec39"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}