Step2_IRF.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "eb8526e8",
   "metadata": {},
   "source": [
    "# Step 2: IRFs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f52f6f1f",
   "metadata": {},
   "source": [
    "In this notebook, we will run magic-cta-pipe scripts on a small data sample. Due to time constraints, we are unable to run the pipeline on all the files needed to produce the main plots (e.g. SED, light curves), so we will provide you a complete dataset to get 'nice' plots and a few MCs and data *.h5* files to run the pipeline.\n",
    "You have to provide the paths where you want to save logging information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e6187f24",
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob\n",
    "import logging\n",
    "import os\n",
    "import subprocess\n",
    "import sys\n",
    "log = logging.getLogger()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59981c83",
   "metadata": {},
   "source": [
    "### Instrument Response Function  \n",
    "\n",
    "Here we will create point-like IRFs (using ring-wobble MC test sample, applying optimal gammaness and $\\theta$ cuts).\n",
    "The IRFs (e.g.: effective area and energy migration matrix,...) will be produced in the fitz.gz format which is then used in the next steps (DL3 and DL4).\n",
    "If gamma diffuse test sample is provided it is possible to calculate the offset-dependent IRFs (Full-enclosure). To get the point-like IRFs the  **fov_offset_bins** should be set to a single bin.\n",
    "\n",
    "We can run the script directly in the terminal with:\n",
    "\n",
    "$ python lst1_magic_create_irf.py -g yourfile -o Path1 -c configfile\n",
    "\n",
    "with the following options:\n",
    "\n",
    "-g: input DL2 gamma file (test sample)\n",
    "\n",
    "-o: output directory (to save IRFs)\n",
    "\n",
    "-c: configuration file:\n",
    "\n",
    "In the configuration file, we have to set the parameters as shown below:\n",
    "\n",
    "1) IRF **quality_cuts**: \"disp_diff_mean < 0.22\", which represent the standard quality cut on DISPs.\n",
    "2) **event_type**: \"software\", where the options are \"hardware\" which stands for a hardware trigger between MAGIC and LST-1; \"software\", which stands for any 2- or 3-telescope combinations except MAGIC-I + MAGIC-II; \"software_only_3tel\", which stands only for the combo of the type MI + MII+ LST-1; and \"magic_only\".\n",
    "3) **cut_type**: \"dynamic\". The options are \"global\", for a cut that is not energy dependent ; \"dynamic\", for a cut evaluated in every energy bin according to the given gamma and theta efficiencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8e6378e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "create_irf:\n",
      "    quality_cuts: \"disp_diff_mean < 0.22\"\n",
      "    event_type: \"software\"  # select \"software\", \"software_only_3tel\", \"magic_only\" or \"hardware\"\n",
      "    weight_type_dl2: \"intensity\"  # select \"simple\", \"variance\" or \"intensity\"\n",
      "    obs_time_irf: \"50 h\"  # used when creating a background HDU\n",
      "\n",
      "    energy_bins:  # log space\n",
      "        start: \"0.01 TeV\"\n",
      "        stop: \"1000 TeV\"\n",
      "        n_edges: 26\n",
      "\n",
      "    migration_bins:  # log space\n",
      "        start: 0.2\n",
      "        stop: 5\n",
      "        n_edges: 31\n",
      "\n",
      "    fov_offset_bins:  # linear space, used for diffuse MCs\n",
      "        start: \"0 deg\"\n",
      "        stop: \"1 deg\"\n",
      "        n_edges: 2\n",
      "\n",
      "    source_offset_bins:  # linear space, used when creating PSF HDU\n",
      "        start: \"0 deg\"\n",
      "        stop: \"1 deg\"\n",
      "        n_edges: 101\n",
      "\n",
      "    bkg_fov_offset_bins:  # linear space, used when creating background HDU\n",
      "        start: \"0 deg\"\n",
      "        stop: \"10 deg\"\n",
      "        n_edges: 21\n",
      "\n",
      "    gammaness:\n",
      "        cut_type: \"dynamic\"  # select \"global\" or \"dynamic\"\n",
      "        global_cut_value: 0.8  # used for the global cut\n",
      "        efficiency: 0.9  # used for the dynamic cuts\n",
      "        min_cut: 0.05  # used for the dynamic cuts\n",
      "        max_cut: 0.85  # used for the dynamic cuts\n",
      "\n",
      "    theta:\n",
      "        cut_type: \"global\"  # select \"global\" or \"dynamic\"\n",
      "        global_cut_value: \"0.2 deg\"  # used for the global cut\n",
      "        efficiency: 0.75  # used for the dynamic cuts\n",
      "        min_cut: \"0.1 deg\"  # used for the dynamic cuts\n",
      "        max_cut: \"0.25 deg\"  # used for the dynamic cuts\n"
     ]
    }
   ],
   "source": [
    "%%capture\n",
    "os.system(\"sed -n 214,257p /fefs/aswg/workspace/analysis-school-2024/MCP/irf/conf/config.yaml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08b7f98a",
   "metadata": {},
   "source": [
    "Let's start by setting up the configuration and data paths:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d456072e",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "dir_irf=('....../IRF')\n",
    "#dir_irf='/fefs/aswg/workspace/elisa.visentin/sw_school_2024/IRF'\n",
    "os.makedirs(dir_irf)\n",
    "\n",
    "f=open(f'{dir_irf}/IRF.log','w')\n",
    "\n",
    "input_gamma=('/fefs/aswg/workspace/analysis-school-2024/MCP/dl2/output/dl2_gamma_zd_37.814deg_az_270.0deg_LST1_MAGIC1_MAGIC2_run10102_to_20002.h5')\n",
    "\n",
    "config=('/fefs/aswg/workspace/analysis-school-2024/MCP/irf/conf/config.yaml')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3bbdfb6",
   "metadata": {},
   "source": [
    "Lines to get files from the data directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5c7c14b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_file_gamma = glob.glob(input_gamma)\n",
    "input_file_gamma.sort()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d16fff78",
   "metadata": {},
   "source": [
    "Here we use python subprocess.run to run the script and also get a log file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "eb02f724",
   "metadata": {},
   "outputs": [],
   "source": [
    "for input_file in input_file_gamma: \n",
    "    x=subprocess.run(['lst1_magic_create_irf', f'-g{input_file}', f'-o{dir_irf}',\\\n",
    "        f'-c{config}'], stdout=f, stderr=f) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "613cb50d",
   "metadata": {},
   "source": [
    "We can check the log file by typing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a559b610",
   "metadata": {},
   "outputs": [],
   "source": [
    "more $dir_irf/IRF.log"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "228ed449",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}