examples/api_widgets_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IPython Widgets Example\n",
    "\n",
    "### Purpose\n",
    "Demonstrate common uses of the `ipysliderule` module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Load necessary packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sliderule import icesat2, ipysliderule, io, sliderule\n",
    "import ipywidgets as widgets\n",
    "import logging\n",
    "import warnings\n",
    "# autoreload\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "# turn off warnings for demo\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initiate SlideRule API\n",
    "- Sets the URL for accessing the SlideRule service\n",
    "- Builds a table of servers available for processing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# set the url for the sliderule service\n",
    "# set the logging level\n",
    "icesat2.init(\"slideruleearth.io\", loglevel=logging.WARNING)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set options for making science data processing requests to SlideRule\n",
    "\n",
    "SlideRule follows a streamlined version of the [ATL06 land ice height algorithm](https://nsidc.org/sites/nsidc.org/files/technical-references/ICESat2_ATL06_ATBD_r005.pdf).\n",
    "\n",
    "SlideRule also can use different sources for photon classification before calculating the average segment height.  \n",
    "This is useful for example, in cases where there may be a vegetated canopy affecting the spread of the photon returns.\n",
    "- ATL03 photon confidence values, based on algorithm-specific classification types for land, ocean, sea-ice, land-ice, or inland water\n",
    "- [ATL08 Land and Vegetation Height product](https://nsidc.org/data/atl08) photon classification\n",
    "- Experimental YAPC (Yet Another Photon Classification) photon-density-based classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# display widgets for setting SlideRule parameters\n",
    "SRwidgets = ipysliderule.widgets()\n",
    "widgets.VBox([\n",
    "    SRwidgets.asset,\n",
    "    SRwidgets.release,\n",
    "    SRwidgets.classification,\n",
    "    SRwidgets.surface_type,\n",
    "    SRwidgets.confidence,\n",
    "    SRwidgets.quality,\n",
    "    SRwidgets.land_class,\n",
    "    SRwidgets.yapc_knn,\n",
    "    SRwidgets.yapc_win_h,\n",
    "    SRwidgets.yapc_win_x,\n",
    "    SRwidgets.yapc_min_ph,\n",
    "    SRwidgets.yapc_weight,\n",
    "    SRwidgets.length,\n",
    "    SRwidgets.step,\n",
    "    SRwidgets.iteration,\n",
    "    SRwidgets.spread,\n",
    "    SRwidgets.count,\n",
    "    SRwidgets.window,\n",
    "    SRwidgets.sigma,\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interactive Mapping with Leaflet\n",
    "\n",
    "Interactive maps within the SlideRule python API are built upon [ipyleaflet](https://ipyleaflet.readthedocs.io).\n",
    "\n",
    "#### Leaflet Basemaps and Layers\n",
    "\n",
    "There are 3 projections available within SlideRule for mapping ([Global](https://epsg.io/3857), [North](https://epsg.io/5936) and [South](https://epsg.io/3031)).  There are also contextual layers available for each projection.\n",
    "\n",
    "<table>\n",
    "  <tbody>\n",
    "    <tr>\n",
    "      <th align='center' max-width=\"30%\"><a href=\"https://epsg.io/3857\">Global (Web Mercator, EPSG:3857)</a></th>\n",
    "      <th align='center' max-width=\"30%\"><a href=\"https://epsg.io/5936\">North (Alaska Polar Stereographic, EPSG:5936)</a></th>\n",
    "      <th align='center' max-width=\"30%\"><a href=\"https://epsg.io/3031\">South (Antarctic Polar Stereographic, EPSG:3031)</a></th>\n",
    "    </tr>\n",
    "    <tr>\n",
    "      <td align='left' valign='top' width=\"30%\">\n",
    "        <ul style=\"line-height: 1.5em\">\n",
    "            <li><a href=\"https://apps.nationalmap.gov/3depdem/\">USGS 3DEP Hillshade</a></li>\n",
    "            <li><a href=\"https://asterweb.jpl.nasa.gov/gdem.asp\">ASTER GDEM Hillshade</a></li>\n",
    "            <li><a href=\"https://www.arcgis.com/home/item.html?id=10df2279f9684e4a9f6a7f08febac2a9\">ESRI Imagery</a></li>\n",
    "            <li><a href=\"http://glims.colorado.edu/glacierdata/\">Global Land Ice Measurements from Space (GLIMS)</a></li>\n",
    "            <li><a href=\"https://www.glims.org/RGI/\">Randolph Glacier Inventory (RGI)</a></li>\n",
    "        </ul>\n",
    "       </td>\n",
    "       <td align='left' valign='top' width=\"30%\">\n",
    "        <ul style=\"line-height: 1.5em\">\n",
    "            <li><a href=\"http://goto.arcgisonline.com/maps/Arctic_Imagery\">ESRI Imagery</a></li>\n",
    "            <li><a href=\"https://www.pgc.umn.edu/data/arcticdem\">ArcticDEM</a></li>\n",
    "        </ul>\n",
    "       </td>\n",
    "       <td align='left' valign='top' width=\"30%\">\n",
    "        <ul style=\"line-height: 1.5em\">\n",
    "            <li><a href=\"https://lima.usgs.gov/\">Landsat Image Mosaic of Antarctica (LIMA)</a></li>\n",
    "            <li><a href=\"https://nsidc.org/data/nsidc-0280\">MODIS Mosaic of Antarctica (MOA)</a></li>\n",
    "            <li><a href=\"https://nsidc.org/data/NSIDC-0103\">Radarsat Antarctic Mapping Project (RAMP)</a></li>\n",
    "            <li><a href=\"https://www.pgc.umn.edu/data/rema\">Reference Elevation Model of Antarctica (REMA)</a></li>\n",
    "        </ul>\n",
    "      </td>\n",
    "    </tr>\n",
    "  </tbody>\n",
    "</table>\n",
    "\n",
    "In addition, most [xyzservice providers](https://xyzservices.readthedocs.io/en/stable/introduction.html) can be added as contextual layers to the global Web Mercator maps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "widgets.VBox([\n",
    "    SRwidgets.projection,\n",
    "    SRwidgets.layers,\n",
    "    SRwidgets.raster_functions\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select regions of interest for submitting to SlideRule\n",
    "\n",
    "Here, we create polygons or bounding boxes for our regions of interest.  \n",
    "This map is also our viewer for inspecting our SlideRule ICESat-2 data returns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# create ipyleaflet map in specified projection\n",
    "m = ipysliderule.leaflet(SRwidgets.projection.value)\n",
    "# install click handler callback\n",
    "m.add_selected_callback(SRwidgets.atl06_click_handler)\n",
    "m.map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m.add_layer(\n",
    "    layers=SRwidgets.layers.value,\n",
    "    rendering_rule=SRwidgets.rendering_rule\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build and transmit requests to SlideRule\n",
    "\n",
    "- SlideRule will query the [NASA Common Metadata Repository (CMR)](https://cmr.earthdata.nasa.gov/) for ATL03 data within our region of interest\n",
    "- When using the `nsidc-s3` asset, the ICESat-2 ATL03 data are then accessed from the NSIDC AWS s3 bucket in `us-west-2`\n",
    "- The ATL03 granules is spatially subset within SlideRule to our exact region of interest\n",
    "- SlideRule then uses our specified parameters to calculate average height segments from the ATL03 data in parallel\n",
    "- The completed data is streamed concurrently back and combined into a geopandas GeoDataFrame within the Python client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "# sliderule asset and data release\n",
    "asset = SRwidgets.asset.value\n",
    "release = SRwidgets.release.value\n",
    "# build sliderule parameters using latest values from widget\n",
    "parms = SRwidgets.build_atl06()\n",
    "\n",
    "# create an empty geodataframe\n",
    "gdf = sliderule.emptyframe()\n",
    "\n",
    "# for each region of interest\n",
    "for poly in m.regions:\n",
    "    # add polygon from map to sliderule parameters\n",
    "    parms[\"poly\"] = poly \n",
    "    # make the request to the SlideRule (ATL06-SR) endpoint\n",
    "    # and pass it the request parameters to request ATL06 Data\n",
    "    gdf = gdf.append(icesat2.atl06p(parms, asset, version=release))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Review GeoDataFrame output\n",
    "Can inspect the columns, number of returns and returns at the top of the GeoDataFrame.\n",
    "\n",
    "See the [ICESat-2 documentation](https://slideruleearth.io/rtd/user_guide/ICESat-2.html#elevations) for descriptions of each column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(f'Returned {gdf.shape[0]} records')\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add GeoDataFrame to map\n",
    "\n",
    "For stability of the leaflet map, SlideRule will as a default limit the plot to have up to 10000 points from the GeoDataFrame\n",
    "\n",
    "GeoDataFrames can be plotted in any available [matplotlib colormap](https://matplotlib.org/stable/tutorials/colors/colormaps.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "widgets.VBox([\n",
    "    SRwidgets.variable,\n",
    "    SRwidgets.cmap,\n",
    "    SRwidgets.reverse,\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "# ATL06-SR fields for hover tooltip\n",
    "fields = m.default_atl06_fields()\n",
    "m.GeoData(gdf, column_name=SRwidgets.variable.value, cmap=SRwidgets.colormap,\n",
    "    max_plot_points=10000, tooltip=True, colorbar=True, fields=fields)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create plots for a single track\n",
    "- cycles: Will plot all available cycles of data returned by SlideRule for a single RGT and ground track\n",
    "- scatter: Will plot data returned by SlideRule for a single RGT, ground track and cycle\n",
    "- (to select a track from the leaflet plot above, click on one of the plotted elevations and the RGT and Cycle will automatically get populated below)\n",
    "\n",
    "The cycles plots should only be used in regions with [repeat Reference Ground Track (RGT) pointing](https://icesat-2.gsfc.nasa.gov/science/specs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "widgets.VBox([\n",
    "    SRwidgets.plot_kind,\n",
    "    SRwidgets.rgt,\n",
    "    SRwidgets.ground_track,\n",
    "    SRwidgets.cycle,\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%matplotlib widget\n",
    "# default is to skip cycles with significant off-pointing\n",
    "SRwidgets.plot(gdf, kind=SRwidgets.plot_kind.value, cycle_start=3,\n",
    "    legend=True, legend_frameon=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Save GeoDataFrame to output file\n",
    "- [pytables HDF5](https://www.pytables.org/): easily read back as a Geopandas GeoDataFrame\n",
    "- [netCDF](https://www.unidata.ucar.edu/software/netcdf): interoperable with other programs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "display(SRwidgets.filesaver)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# append sliderule api version to attributes\n",
    "version = icesat2.get_version()\n",
    "parms['version'] = version['icesat2']['version']\n",
    "parms['commit'] = version['icesat2']['commit']\n",
    "# save to file in format (HDF5 or netCDF)\n",
    "io.to_file(gdf, SRwidgets.file,\n",
    "    format=SRwidgets.format,\n",
    "    driver='pytables',\n",
    "    parameters=parms,\n",
    "    regions=m.regions,\n",
    "    verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Read GeoDataFrame from input file\n",
    "- [pytables HDF5](https://www.pytables.org/)\n",
    "- [netCDF](https://www.unidata.ucar.edu/software/netcdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "display(SRwidgets.fileloader)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# read from file in format (HDF5 or netCDF)\n",
    "gdf,parms,regions = io.from_file(SRwidgets.file,\n",
    "    format=SRwidgets.format,\n",
    "    driver='pytables',\n",
    "    return_parameters=True,\n",
    "    return_regions=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Review GeoDataFrame input from file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "gdf.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set parameters and add saved regions to map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "SRwidgets.set_values(parms)\n",
    "m.add_region(regions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}