DP02_04b_Intermediate_Butler_Queries.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Discovery and Query with the Butler\n",
    "\n",
    "<img align=\"left\" src = https://project.lsst.org/sites/default/files/Rubin-O-Logo_0.png width=250 style=\"padding: 10px\" alt=\"Rubin Observatory logo, a graphical representation of turning stars into data.\">\n",
    "<br>\n",
    "Contact author(s): Alex Drlica-Wagner, Melissa Graham <br>\n",
    "Last verified to run: 2024-12-17 <br>\n",
    "LSST Science Pipelines version: Weekly 2024_50 <br>\n",
    "Container Size: medium <br>\n",
    "Targeted learning level: intermediate <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Description:** Learn how to discover data and apply query constraints with the Butler."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Skills:** Use the Butler registry, dataIds, and spatial and temporal constraints."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**LSST Data Products:** calexps, deepCoadds, sources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Packages:** lsst.daf.butler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Credit:** Elements of this tutorial were originally developed by Alex Drlica-Wagner in the context of the LSST Stack Club."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Get Support:**\n",
    "Find DP0-related documentation and resources at <a href=\"https://dp0.lsst.io\">dp0.lsst.io</a>. Questions are welcome as new topics in the <a href=\"https://community.lsst.org/c/support/dp0\">Support - Data Preview 0 Category</a> of the Rubin Community Forum. Rubin staff will respond to all questions posted there."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Introduction\n",
    "\n",
    "In the introductory Butler tutorial, we learned how to access DP0 data given a specific data identifier (`dataId`). In this tutorial, we will explore how to use the Butler to find available data sets that match different sets of criteria (i.e., perform spatial and temporal searches). As a reminder, full Butler documentation can be found [in the documentation for lsst.dat.butler](https://pipelines.lsst.io/v/weekly/modules/lsst.daf.butler/index.html). For this notebook in particular, you might find this set of [Frequently Asked Questions](https://pipelines.lsst.io/middleware/faq.html) for the LSST Science Pipelines middleware to be useful."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Package Imports"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import general python packages and several packages from the LSST Science Pipelines, including the Butler package and AFW Display, which can be used to display images.\n",
    "More details and techniques regarding image display with `afwDisplay` can be found in the `rubin-dp0` GitHub Organization's [tutorial-notebooks](https://github.com/rubin-dp0/tutorial-notebooks) repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Generic python packages\n",
    "import numpy as np\n",
    "import pylab as plt\n",
    "import astropy.time\n",
    "\n",
    "# LSST Science Pipelines (Stack) packages\n",
    "import lsst.daf.butler as dafButler\n",
    "import lsst.afw.display as afwDisplay\n",
    "afwDisplay.setDefaultBackend('matplotlib')\n",
    "\n",
    "# Set a standard figure size to use\n",
    "plt.rcParams['figure.figsize'] = (8.0, 8.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2. Create an instance of the Butler\n",
    "\n",
    "Create an instance of the Butler pointing to the DP0.2 data by specifying the `dp02` configuration and the `2.2i/runs/DP0.2` collection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "butler = dafButler.Butler('dp02', collections='2.2i/runs/DP0.2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Explore the DP0 data repository\n",
    "\n",
    "Butler repositories have both a database component and a file-like storage component.\n",
    "The database component can be accessed through the Butler registry, while file-like storage can be local (i.e., pointing to a directory on the local file system) or remote (i.e., pointing to cloud storage resources).\n",
    "DP0 uses Simple Storage Service (S3) buckets, which are public cloud storage resources that are similar to file folders.\n",
    "The S3 buckets store objects, which consist of data and its descriptive metadata."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1. The Butler registry\n",
    "\n",
    "The database side of a data repository is called a `registry`.\n",
    "The registry contains entries for all data products, and organizes them by _collections_, _dataset types_, and _data IDs_.\n",
    "We can access a registry client directly as part of our Butler object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "registry = butler.registry"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional: learn more about the registry by uncommenting the following line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# help(registry)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1.1. Querying collections\n",
    "\n",
    "Collections are lightweight groups of datasets such as the set of raw images for a particular instrument, self-consistent calibration datasets, and the outputs of a processing run.\n",
    "For DP0.2, we use the `2.2i/runs/DP0.2` collection, which we specified when creating our instance of the Butler.\n",
    "\n",
    "It is possible to access other collections, which can be queried with `butler.collections.query`. \n",
    "More about collections can be found in the [lsst.daf.butler documentation](https://pipelines.lsst.io/v/weekly/modules/lsst.daf.butler/organizing.html#collections) and in the middleware [FAQ](https://pipelines.lsst.io/middleware/faq.html#querycollections).\n",
    "\n",
    "> **Risk reminder:** for DP0 there are no read/write restrictions on the Butler repository.\n",
    "\n",
    "The above risk means all users can see *everything* in the Butler, including intermediate processing steps, test runs, staff repositories, and other user repositories.\n",
    "This fact makes the butler.collections.query functionality less useful for data discovery than it will be in the future, due to the sheer number of Butler collections exposed to users."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional: print a giant list of every collection that exists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# for c in sorted(butler.collections.query('*')):\n",
    "#      print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The only collection users need is the DP0.2 collection, which is named \"2.2i/runs/DP0.2.\"\n",
    "\n",
    "Optional: Use `butler.collections.query_info` to print long list of collections that are children of the DP0.2 parent collection. For DP0, this includes a wide variety of different processing runs by Rubin Data Management, including intermediate steps and tests. Recall that for DP0 there are no read restrictions on the Butler repository, and that future releases will be cleaner."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# butler.collections.query_info('2.2i/runs/DP0.2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is also a way to list collections that match a string or regular expression. \n",
    "\n",
    "Optional: print a list collections with the word 'calib' in them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# for c in sorted(butler.collections.query(\"*calib*\")):\n",
    "#      print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### 2.1.2. queryDatasetTypes\n",
    "\n",
    "As shown in the introductory Butler notebook, useful DP0.2 datasetTypes for images include deepCoadd, calexp, and goodSeeingDiff_differenceExp, while useful datasetTypes for catalogs include sourceTable, objectTable, diaObjectTable_tract, etc.\n",
    "See the [DP0.2 Data Products Definitions Document](https://dp0-2.lsst.io/data-products-dp0-2/index.html#dp0-2-data-products-definition-document-dpdd) for more details about the DP0.2 data sets.\n",
    "\n",
    "The queryDatasetTypes function allows users to explore available datasetTypes.\n",
    "\n",
    "> **Notice:** as described in <a href=\"https://pipelines.lsst.io/middleware/faq.html#querydatasettypes\">the documentation page for `queryDatasetTypes`</a>, this method will report *all* datasetTypes that have been registered with a data repository, even if there aren’t any datasets of that type actually present.\n",
    "\n",
    "The queryDatasetTypes function is a useful tool when you know the name of the dataset type already, and want to see how it’s defined (i.e., what kind of dataId it accepts)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional: print a giant list of all the available dataset types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# for dt in sorted(registry.queryDatasetTypes()):\n",
    "#     print(dt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can provide a string or regular expression to queryDatasetTypes to list a subset of datasetTypes. For example, the following cell lists datasetTypes with names including '\\_tract', which indicates that this dataset can be queried by tract."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for dt in sorted(registry.queryDatasetTypes('*_tract')):\n",
    "    print(dt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional: list all the dataset types associated with deepCoadds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# for dt in sorted(registry.queryDatasetTypes('deepCoadd*')):\n",
    "#     print(dt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that queryDatasetTypes returns a generator function, which is a special kind of function that can be iterated.\n",
    "It contains objects that can be looped over like a list, as shown in the code cells above. \n",
    "Generator functions are used because they do not store their contents in memory, making them more suitable for really large data sets, like LSST."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1.3. getDatasetType\n",
    "\n",
    "If you want to retrieve a single datasetType rather than a generator function, you can use `getDatasetType`. Below, we get the datasetType associated with deepCoadd."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dt_deepCoadd = registry.getDatasetType('deepCoadd')\n",
    "print(dt_deepCoadd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1.4. query_dimension_records\n",
    "\n",
    "As described in the <a href=\"https://pipelines.lsst.io/middleware/faq.html#querydimensionrecords\">documentation for `queryDimensionRecords`</a>,\n",
    "this method provies a way to inspect metadata tables.\n",
    "\n",
    "Optional: print the different metadata elements that are available to be inspected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# for a in butler.dimensions.getStaticElements():\n",
    "#     print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A call to `query_dimension_records` will return a set of fields, depending on the element.\n",
    "\n",
    "Use `butler.dimensions.schema` to print a list of the fields that would be returned for a given element (in this case \"detector\")."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(butler.dimensions['detector'].schema)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `butler.query_dimension_records` method to return some detector metadata available for DP0 images (LSSTCam-imSim), for detectors 6, 7, and 8 only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "butler.query_dimension_records('detector', where=\"instrument='LSSTCam-imSim' \"\n",
    "                               \"AND detector.id IN (6..8)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional: use the `query_dimension_records` method to return the exposure, visit, and detector metadata available for DP0 visit number 971990 and detector 0. (Use \"[0]\" to print only the first item from the list.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for dim in ['exposure', 'visit', 'detector']:\n",
    "    print(butler.query_dimension_records(dim, where='visit = 971990 and detector = 0', limit=-1)[0])\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2. Use a dataId with query_datasets\n",
    "\n",
    "The dataId is a dictionary-like identifier for a data product (more information can be found in the [lsst.daf.butler documentation](https://pipelines.lsst.io/modules/lsst.daf.butler/dimensions.html#data-ids)).\n",
    "Each `DatasetType` (i.e., `calexp`, `deepCoadd`, `objectTable`, etc.) uses a different set of keys in its dataId, which are also called \"dimensions\".\n",
    "\n",
    "Use the registry to get the DatasetType for a specific named dataset, in this case a `calexp`, and list its dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dt = registry.getDatasetType('calexp')\n",
    "print(\"Name:\", dt.name)\n",
    "print(\"Dimensions:\", dt.dimensions)\n",
    "print(\"Storage Class:\", dt.storageClass)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataId contains both *implied* and *required* keys.\n",
    "For example, the value of `band` is *implied* by the `visit`, because a single visit refers to a single exposure at a single pointing in a single band.\n",
    "\n",
    "In other tutorial notebooks, we have seen how to access a specific data product using a fully specified dataId. A query for a fully specified dataId should return one unique entry (however, see the [FAQ](https://pipelines.lsst.io/middleware/faq.html#why-do-queries-return-duplicate-results) entry about duplicate results from chained collections).\n",
    "\n",
    "As described in the <a href=\"https://pipelines.lsst.io/v/weekly/middleware/faq.html#querydatasets\">documentation page for `query_datasets`</a>, this method returns `datasetRefs`, which can be passed directly to a call to `butler.get()` in order to retrieve the desired data (see next section)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "datasetType = 'calexp'\n",
    "dataId = {'visit': 192350, 'detector': 175}\n",
    "datasetRefs = butler.query_datasets(datasetType, data_id=dataId)\n",
    "\n",
    "for i, ref in enumerate(datasetRefs):\n",
    "    print(ref.dataId)\n",
    "    print(\"band:\", ref.dataId['band'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A dataId can be represented as regular Python `dict` object, but when they are returned from the `Butler` the `DataCoordinate` class is used instead.\n",
    "The value of a single key, in this case *band*, can also be printed by specifying the key name.\n",
    "\n",
    "Note that when we instantiated the Butler pointing to the DP0.2 collection (i.e., at the beginning of this notebook), we implicitly specified that we are interested in data associated with the `LSSTCam-imSim` instrument, since that is the only instrument contained in the DP0.2 collection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to query for all data products that match a partially specified dataId.\n",
    "For example, in the following cell we use a partially specified dataId to select all the `calexp` data associated with visit=192350.\n",
    "This search will return a separate entry for each CCD detector that was processed for this visit (this visit happens to be close to the edge of the simulated footprint, so only 187 detectors were processed). \n",
    "We'll print information about a few of them. \n",
    "\n",
    "(The following cell will fail and return an error if the query is requesting a `DatasetRef` for data that does not exist.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "datasetType = 'calexp'\n",
    "dataId = {'visit': 192350}\n",
    "datasetRefs = butler.query_datasets(datasetType, data_id=dataId)\n",
    "\n",
    "for i, ref in enumerate(datasetRefs):\n",
    "    print(ref.dataId)\n",
    "    if i > 5:\n",
    "         print('...')\n",
    "         break\n",
    "\n",
    "print(f\"Found {len(datasetRefs)} detectors\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The LSST Science Camera has 189 science detectors, total, but this visit is very near the edge of the DC2 simulation region (only 300 square degrees of sky), so not all detectors are within the dataset.\n",
    "\n",
    "### 2.2.1 Limiting the number of results\n",
    "\n",
    "You can limit the number of query results that are returned by passing a \"limit=N\" argument to `butler.query_datasets`. If you pass \"limit=N\", the query will return N results. If you instead add a minus sign in front of the number (\"limit=-N\"), the query will still return N results, but it will tell you that there are more results that were not returned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "datasetRefs = butler.query_datasets(datasetType, data_id=dataId, limit=-2)\n",
    "\n",
    "for i, ref in enumerate(datasetRefs):\n",
    "    print(ref.dataId)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3. Use butler.get() with a datasetRef\n",
    "\n",
    "One of the beauties of the Butler is that there is no need to know exactly where the data live in order to access it.\n",
    "In previous notebooks we've seen how to pass a dataId to Butler.get to return an instance of the appropriate object.\n",
    "When you already have a datasetRef, it is faster and more efficient to pass the datasetRef to Butler.get. \n",
    "Use Butler.get to retrieve the calexps from the previous query, and then get the detector Id from the calexp's properties."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for i, ref in enumerate(datasetRefs):\n",
    "    calexp = butler.get(ref)\n",
    "    print(' calexp.detector.getId(): ', calexp.detector.getId())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional: display the calexps retrieved from the Butler."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# for i, ref in enumerate(datasetRefs):\n",
    "#     calexp = butler.get(ref)\n",
    "#     print(i, ' calexp.detector.getId(): ', calexp.detector.getId())\n",
    "#     fig = plt.figure()\n",
    "#     display = afwDisplay.Display(frame=fig)\n",
    "#     display.scale('asinh', 'zscale')\n",
    "#     display.mtv(calexp.image)\n",
    "#     plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Query the DP0 data repository\n",
    "\n",
    "The following temporal and spatial query examples are all for images (calexps), and not catalogs\n",
    "because the TAP services is the recommended way to query the catalogs, not the butler.\n",
    "\n",
    "### 3.1. Basic image queries\n",
    "\n",
    "Our example above demonstrated a very simple use of `query_datasets`, but additional query terms can also be used, such as band and visit.\n",
    "When a query term is an equality, it can be specified as an argument like `band=''`. \n",
    "When a query term is an inequality, it can be specified with `where`.\n",
    "More details on Butler queries can be found [here in the lsst.daf.butler documentation](https://pipelines.lsst.io/v/weekly/modules/lsst.daf.butler/queries.html).\n",
    "\n",
    "In the following cell, we query for a list of all calexps corresponding to i-band observations of a single detector over a range of visits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "datasetRefs = butler.query_datasets(dataset_type='calexp', band='i', detector=175,\n",
    "                                    where='visit > 192000 and visit < 193000')\n",
    "\n",
    "for i, ref in enumerate(datasetRefs):\n",
    "    print(ref.dataId)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional: use the datasetRefs to retrieve and display the first two calexp images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# for i, ref in enumerate(datasetRefs[:2]):\n",
    "#     calexp = butler.get(ref)\n",
    "#     fig = plt.figure()\n",
    "#     display = afwDisplay.Display(frame=fig)\n",
    "#     display.scale('asinh', 'zscale')\n",
    "#     display.mtv(calexp.image)\n",
    "#     plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1.1. Optional: retrieving temporal and spatial metadata\n",
    "\n",
    "As a precursor to doing temporal and spatial queries below, we demonstrate the temporal and spatial metadata that can be retrieved for calexps via the Butler.\n",
    "\n",
    "Retrieving only the metadata (calexp.visitInfo, calexp.bbox, or calexp.wcs) can be faster than retreiving the full calexp and then extracting the metadata from it.\n",
    "\n",
    "Print the full visitInfo record for the first datasetRef returned by the previous query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "visitInfo = butler.get('calexp.visitInfo', dataId=datasetRefs[0].dataId)\n",
    "print(visitInfo)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print information from the visitInfo for all query results: the date, exposure time, and the boresight (the pointing coordinates of the telescope)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for i, ref in enumerate(datasetRefs):\n",
    "    visitInfo = butler.get('calexp.visitInfo', dataId=ref.dataId)\n",
    "    print(i, visitInfo.date, visitInfo.exposureTime, visitInfo.boresightRaDec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print spatial information from the bounding box (bbox) and world coordinate system (wcs) metadata: the four corners of the image. Note that the XY coordinates of the image corners must be converted to RA, Dec using the wcs.pixelToSky method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i, ref in enumerate(datasetRefs):\n",
    "    bbox = butler.get('calexp.bbox', dataId=ref.dataId)\n",
    "    wcs = butler.get('calexp.wcs', dataId=ref.dataId)\n",
    "    corners_xy = bbox.getCorners()\n",
    "    tmp = ''\n",
    "    for corn in corners_xy:\n",
    "        radec = wcs.pixelToSky(corn.x, corn.y)\n",
    "        tmp += f'({radec.getRa().asDegrees():.4f}, {radec.getDec().asDegrees():.4f}) '\n",
    "    print(i, tmp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional: view the help documentation for a datasetRef."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# help(ref)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Tip!** In the case where metadata is desired for many images at once, you can avoid the time-consuming use of butler.get for individual dataIds by using the \"with_dimension_records\" keyword when retrieving datasetRefs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "datasetRefs = butler.query_datasets(dataset_type='calexp', band='i', detector=175,\n",
    "                                    where='visit > 192000 and visit < 193000', with_dimension_records=True)\n",
    "\n",
    "for ref in datasetRefs:\n",
    "    print(ref)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the id, exposure time, timespan, and bounding box for each datasetRef:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for i, ref in enumerate(datasetRefs):\n",
    "    record = ref.dataId.records[\"visit\"]\n",
    "    print(i, record.id, record.exposure_time, record.timespan, record.region.getBoundingBox())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 3.2. Temporal image queries\n",
    "\n",
    "The following examples show how to query for data sets that include a desired coordinate and observation date.\n",
    "\n",
    "Since we only need to get the date and time that the exposure was taken, we can start by retrieving only the visitInfo associated with the calexp specified by our dataId (this will be faster than retrieving the full calexp)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dataId = {'visit': 192350, 'detector': 175}\n",
    "visitInfo = butler.get('calexp.visitInfo', dataId=dataId)\n",
    "print(visitInfo)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To query for dimension records or datasets that overlap an arbitrary time range, we can use the `bind` argument to pass times through to `where`.\n",
    "Using `bind` to define an alias for a variable saves us from having to string-format the times into the `where` expression.\n",
    "Note that a `dafButler.Timespan` will accept a `begin` or `end` value that is equal to `None` if it is unbounded on that side.\n",
    "\n",
    "Use `bind` and `where`, along with [astropy.time](https://docs.astropy.org/en/stable/time/index.html), to query for calexps that were obtained within +/- 10 minutes of the calexp defined by the dataId above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "time = astropy.time.Time(visitInfo.date.toPython())\n",
    "minute = astropy.time.TimeDelta(60, format=\"sec\")\n",
    "timespan = dafButler.Timespan(time - 10*minute, time + 10*minute)\n",
    "\n",
    "print(time)\n",
    "print(minute)\n",
    "print(timespan)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "datasetRefs = butler.query_datasets(\"calexp\",\n",
    "                                    where=\"visit.timespan OVERLAPS my_timespan\",\n",
    "                                    bind={\"my_timespan\": timespan})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for i, ref in enumerate(datasetRefs):\n",
    "    print(ref.dataId)\n",
    "    if i > 6:\n",
    "        print('...')\n",
    "        break\n",
    "\n",
    "print(f\"Found {len(list(datasetRefs))} calexps\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many unique visits were obtained within the DC2 area within +/- 10 minutes of the calexp defined above?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "temp = []\n",
    "for i, ref in enumerate(datasetRefs):\n",
    "    temp.append(ref.dataId['visit'])\n",
    "\n",
    "unique_visitIds = np.unique(np.sort(np.asarray(temp, dtype='int')))\n",
    "\n",
    "print('Number of unique visits: ', len(unique_visitIds))\n",
    "print('visitIds for the unique visits: ', unique_visitIds)\n",
    "\n",
    "del temp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The maximum number of visits that can be executed in 20 minutes is about 34 visits. The reason why we find only 13 is because for DC2 the [minion\\_1016 observing strategy](https://docushare.lsst.org/docushare/dsweb/View/Collection-4604) baseline simulation was used, and minion\\_1016 simulates observations over the whole sky but DC2 covers only 300 square degrees. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3. Spatial image queries\n",
    "\n",
    "#### 3.3.1. Overlapping images\n",
    "\n",
    "As a simple first example, we search for deepCoadd images that overlap a particular calexp. \n",
    "\n",
    "We again start by grabbing the visitInfo for the same calexp."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dataId = {'visit': 192350, 'detector': 175}\n",
    "visitInfo = butler.get('calexp.visitInfo', dataId=dataId)\n",
    "print(visitInfo)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we query for `deepCoadd` datasets with a `visit`+`detector` dataId, we'll get just the deepCoadd objects that overlap that observation and have the same band (because a visit implies a band).\n",
    "This is a very simple spatial query for data that overlaps other data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "refs = butler.query_datasets(\"deepCoadd\", data_id=dataId, order_by='tract')\n",
    "\n",
    "for ref in refs:\n",
    "    print(ref)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the corners of the `calexp` and of the overlapping `deepCoadd` patches to trace the image edges in a plot, and show the overlap.\n",
    "\n",
    "In the plot below, the `deepCoadd` tract and patch (4024, 48) does not quite overlap with the original `calexp`.\n",
    "That is because the visit and detector region from the `calexp` WCS and bounding box is not quite the same as the region in the database, which is based on the raw WCS with some padding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "calexp_wcs = butler.get('calexp.wcs', dataId=dataId)\n",
    "calexp_bbox = butler.get('calexp.bbox', dataId=dataId)\n",
    "\n",
    "calexp_corners_ra = []\n",
    "calexp_corners_dec = []\n",
    "\n",
    "for corn in calexp_bbox.getCorners():\n",
    "    radec = calexp_wcs.pixelToSky(corn.x, corn.y)\n",
    "    calexp_corners_ra.append(radec.getRa().asDegrees())\n",
    "    calexp_corners_dec.append(radec.getDec().asDegrees())\n",
    "\n",
    "calexp_corners_ra.append(calexp_corners_ra[0])\n",
    "calexp_corners_dec.append(calexp_corners_dec[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(6, 6))\n",
    "\n",
    "plt.plot(calexp_corners_ra, calexp_corners_dec, ls='solid', color='grey', label='visit detector')\n",
    "\n",
    "for r, ref in enumerate(set(butler.query_datasets(\"deepCoadd\", data_id=dataId))):\n",
    "    deepCoadd_dataId = ref.dataId\n",
    "    str_tract_patch = '(' + str(ref.dataId['tract']) + ', ' + str(ref.dataId['patch'])+')'\n",
    "    deepCoadd_wcs = butler.get('deepCoadd.wcs', dataId=deepCoadd_dataId)\n",
    "    deepCoadd_bbox = butler.get('deepCoadd.bbox', dataId=deepCoadd_dataId)\n",
    "    deepCoadd_corners_ra = []\n",
    "    deepCoadd_corners_dec = []\n",
    "\n",
    "    for corn in deepCoadd_bbox.getCorners():\n",
    "        radec = deepCoadd_wcs.pixelToSky(corn.x, corn.y)\n",
    "        deepCoadd_corners_ra.append(radec.getRa().asDegrees())\n",
    "        deepCoadd_corners_dec.append(radec.getDec().asDegrees())\n",
    "        \n",
    "    deepCoadd_corners_ra.append(deepCoadd_corners_ra[0])\n",
    "    deepCoadd_corners_dec.append(deepCoadd_corners_dec[0])\n",
    "\n",
    "    plt.plot(deepCoadd_corners_ra, deepCoadd_corners_dec, ls='solid', lw=1, label=str_tract_patch)\n",
    "\n",
    "plt.xlabel('RA')\n",
    "plt.ylabel('Dec')\n",
    "plt.legend(loc='upper left', ncol=3)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Figure 1: The bounding box of one detector from a single visit image (a `calexp`) is drawn in gray,\n",
    "> and six of the nearest `deepCoadd` patches are drawn in colors (as in legend), all but one overlapping the `calexp`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3.2. User-defined spatial constraints on images\n",
    "\n",
    "Often one wants to know what images overlap a given point on the sky. Such spatial queries can be accomplished using the \"region OVERLAPS POINT(ra, dec)\" syntax (e.g., see this [Butler queries](https://pipelines.lsst.io/v/weekly/modules/lsst.daf.butler/queries.html#overlaps-operator) documentation). Let us see how this works."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Specify the desired sky coordinate for the search. Below, the RA and Dec could be user-specified, but here we use the telescope boresight accessed from the calexp visitInfo retrieved above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ra, dec = visitInfo.boresightRaDec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Specify a small timespan for the query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "small_timespan = dafButler.Timespan(time - minute, time + minute)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pass the RA and Dec to the query_datasets command, using \"visit_detector_region.region OVERLAPS POINT(ra, dec)\" in the \"where\" clause of the query. Also apply the same timespan constraints as above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "datasetRefs = butler.query_datasets(\"calexp\", where=\"visit.timespan OVERLAPS my_timespan AND \\\n",
    "                                    visit_detector_region.region OVERLAPS POINT(ra, dec)\",\n",
    "                                    bind={\"my_timespan\": small_timespan, \"ra\": ra.asDegrees(), \"dec\": dec.asDegrees()})\n",
    "\n",
    "print(datasetRefs)\n",
    "\n",
    "print(f\"\\nFound {len(datasetRefs)} calexps\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, with the above query, we have *uniquely* recovered the visit for our desired temporal and spatial constraints."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3.3. A note about \"dimensions\"\n",
    "\n",
    "You may have wondered how we knew to use \"visit_detector_region.region\" in the above query. Constraints provided in the \"where\" clause of `butler.query_datasets` can be based on any of the attributes associated with each `dimension` or `element` (elements can be thought of as joins or combinations of dimension traits).\n",
    "\n",
    "See more about querying dimension records [here](https://pipelines.lsst.io/v/weekly/middleware/faq.html#querydimensionrecords).\n",
    "\n",
    "First, list the available dimensions and elements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Dimensions:\\n\", butler.dimensions.getStaticDimensions())\n",
    "print(\"\\nElements:\\n\", butler.dimensions.getStaticElements())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that these are the same except for the additional two entries at the end of the \"elements\" list.\n",
    "\n",
    "For an example dimension and element, use their \"schema\" attribute to see what fields are associated with them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(butler.dimensions[\"exposure\"].schema)\n",
    "print(\"\\n\", butler.dimensions.elements[\"visit_detector_region\"].schema)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These \"fields\" contain information that can be used in query constraints."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3.4. query_data_ids\n",
    "\n",
    "The `query_data_ids` method is a less general approach that will return the combinations of dimensions that _could be_ used to identify datasets. The <a href=\"https://pipelines.lsst.io/v/weekly/middleware/faq.html#querydataids\">documentation page for `query_data_ids`</a> outlines when not to use it.\n",
    "\n",
    "Use it to find the dataIds overlapping the small timespan defined above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for i, data_id in enumerate(butler.query_data_ids(\"visit\", where=\"visit.timespan OVERLAPS my_timespan\",\n",
    "                                                  bind={\"my_timespan\": small_timespan})):\n",
    "    print(data_id)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4. Catalog spatial and temporal queries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The recommended method for querying and retrieving catalog data is to use the TAP service, as demonstrated in other tutorials.\n",
    "However, it is also possible to query catalog data using the same spatial and temporal constraints as used above for images.\n",
    "\n",
    "The Butler's spatial reasoning is designed to work well for regions the size of full data products, like detector- or patch-level images and catalogs, and it's a poor choice for smaller-scale searches.\n",
    "The following search is a bit slow in part because `query_datasets` searches for all `src` datasets that overlap a larger region and then filters the results down to the specified region."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for i, src_ref in enumerate(butler.query_datasets(\"source\", band=\"i\",\n",
    "                                                  where=\"visit.timespan OVERLAPS my_timespan AND \\\n",
    "                                                  visit_detector_region.region OVERLAPS POINT(ra, dec)\",\n",
    "                                                  bind={\"my_timespan\": small_timespan, \"ra\": ra.asDegrees(), \"dec\": dec.asDegrees()})):\n",
    "    print(src_ref)\n",
    "    sources = butler.get(src_ref)\n",
    "    print('Number of sources: ', len(sources))\n",
    "    if i > 2:\n",
    "        print('...')\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show the contents of the last source table retrieved from the Butler.\n",
    "Notice that both the rows and the columns of the table are truncated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Explore metadata for butler-retrieved data products\n",
    "\n",
    "Data retrieved from the butler is enriched with metadata.\n",
    "\n",
    "The following provides a cursory overview of how to explore this metadata for image and catalog data retrieved via the butler.\n",
    "\n",
    "In some cases these options were already covered in the sections above, but have been gathered here for easy reference.\n",
    "\n",
    "### 4.1. Image data\n",
    "\n",
    "Retrieve a `calexp` for a given visit and detector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "calexp = butler.get('calexp', dataId={'visit': 192350, 'detector': 94})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the information (metadata) available for this calexp."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "calexp_info = calexp.getInfo()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-14T02:17:28.903936Z",
     "iopub.status.busy": "2023-09-14T02:17:28.903461Z",
     "iopub.status.idle": "2023-09-14T02:17:29.157670Z",
     "shell.execute_reply": "2023-09-14T02:17:29.156415Z",
     "shell.execute_reply.started": "2023-09-14T02:17:28.903905Z"
    },
    "tags": []
   },
   "source": [
    "**Option:** uncomment the following cell, put the cursor after the period, and press the tab key. \n",
    "A pop-up window will display the methods available for `calexp_info`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# calexp_info."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Option:** alternatively, print all options that would display in the pop-up window from the above cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# [m for m in dir(calexp_info) if not m.startswith('_')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obtain the visit information and summary statistics for this calexp."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "visit_info = calexp_info.getVisitInfo()\n",
    "summary_info = calexp_info.getSummaryStats()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the summary statistics for this visit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary_info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Option:** explore other aspects of the metadata, for example, detector information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# [m for m in dir(calexp_info.getDetector()) if not m.startswith('_')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2. Catalog data\n",
    "\n",
    "Retrieve sources from the `src` table for a given visit and detector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "src_cat = butler.get('src', dataId={'visit': 192350, 'detector': 94})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Option:** uncomment the following cell, put the cursor after the period, and press the tab key. \n",
    "A pop-up window will display the options available for the retrieved data product."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# src_cat."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-14T02:05:24.591615Z",
     "iopub.status.busy": "2023-09-14T02:05:24.591265Z",
     "iopub.status.idle": "2023-09-14T02:05:24.791936Z",
     "shell.execute_reply": "2023-09-14T02:05:24.790740Z",
     "shell.execute_reply.started": "2023-09-14T02:05:24.591592Z"
    },
    "tags": []
   },
   "source": [
    "**Option:** uncomment and execute the following cell in order to print the table retrieved (it will be truncated)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# src_cat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the names of the columns available in the schema for this data product."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "columns = src_cat.schema.getNames()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Option:** uncomment and execute the following cell in order to list all columns (it is a very long list)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List only the first part of all column names, before the underscore. This shows the different types of columns available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "src_cat.getSchema().getNames(topOnly=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To view the information for only a given element, use the `find` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "src_cat.getSchema().find('id')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}