Skip to content

Tutorial Jupyter Notebooks for Data Preview 0, created and maintained by the Rubin Observatory Community Science Team.

License

Notifications You must be signed in to change notification settings

rubin-dp0/tutorial-notebooks

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

tutorial-notebooks

Tutorial Jupyter Notebooks for Data Preview 0.2 and 0.3, created and maintained by the Rubin Observatory Community Science Team.

The prod branch will appear in IDF RSP users' home directories, in notebooks/tutorial-notebooks/.


DP0.2 Tutorials

Tutorial titles in bold have Spanish-language versions.

Title Brief Description
01. Introduction to DP0.2 Use the Jupyter Notebooks and Rubin python packages to access LSST data products.
02a. Introduction to TAP Explore the DP0.2 catalogs with the TAP service.
02b. Catalog Queries with TAP Execute complex ADQL queries with the TAP service. Visualize catalog data sets.
02c. Image Queries with TAP Retrieve and display images using the ObsTAP service.
03a. Image Display and Manipulation Learn how to display and manipulate images using the LSST Science Pipelines.
03b. Image Display with Firefly Use the Firefly interactive interface for image data.
03c. Big deepCoadd Cutout Use the GetTemplateTask to create a custom deepCoadd cutout that spans multiple patches and tracts.
04a. Introduction to the Butler Use the Butler to query DP0 images and catalogs.
04b. Intermediate Butler Queries Learn to discover data and apply query constraints with the Butler.
05. Source Detection and Measurement Access, display, and manipulate images; detect, deblend, and measure sources; and extract, plot, and use object footprints.
06a. Interactive Image Visualization Create interactive image visualizations with the HoloViews and Bokeh open-source python libraries.
06b. Interactive Catalog Visualization Create interactive catalog visualizations for large datasets with HoloViews, Bokeh, and Datashader.
07a. DiaObject Samples Use the DiaObject table parameters to identify a sample of time-variable objects of interest.
07b. Variable Star Lightcurves Use the DP0.2 catalogs to identify variable stars and plot their lightcurves.
08. Truth Tables Explore, retrieve, and compare data from the truth and measurement tables.
09a. Custom Coadd Create a custom "deepCoadd" using only a subset of the input visits.
09b. Custom Coadd Sources Detect and measure sources in a custom "deepCoadd" image.
10. Deblender Data Products Use the outputs of the multiband deblender to explore the footprints of parent and child objects.
11. Working with User Packages An example of how to install and set up user packages.
12a. Point Spread Function Data Products A demonstration of how to access calexp and deepCoadd PSF properties.
12b. Point Spread Function Science Demo Demonstration of the use of measured PSF properties in weak lensing analysis.
13a. Using The Image Cutout Tool With DP0.2 Demonstration of the use of the image cutout tool with a few science applications.
14. Injecting Synthetic Sources Into Single-Visit Images Inject artificial stars and galaxies into images.
15. Survey Property Maps Use the tools to visualize full-area survey property maps.

DP0.3 Tutorials

Title Brief Description
01. Introduction to DP0.3 An overview of the contents of the DP0.3 moving object catalogs.
02. Main Belt Asteroids A brief exploration of the orbital properties of Main Belt asteroids in DP0.3 catalogs.
03. Trans-Neptunian Objects Explore the trans-Neptunian object populations in DP0.3.
04a. Introduction to Phase Curves Explore phase curves for DP0.3 solar system objects.
04b. Advanced Phase Curve Modeling Explicitly investigate the derivation of phase curves for Main Belt asteroids.
05. Near-Earth Objects Exploration of the orbital properties of near-Earth objects in the DP0.3 catalogs.
06. User-Uploaded Catalogs Use the TAP upload functionality for user-supplied tables and join them with DP0.3 catalogs.
07. Interactive Catalog Visualization Create interactive catalog visualizations for large datasets with HoloViews, Bokeh, and Datashader.

Advisories

The tutorials in this repo will only work on the Rubin Science Platform deployed at the Interim Data Facility for Data Preview 0.

The tutorials in this repo are subject to change, as the Rubin Science Platform and the LSST Science Pipelines are in active development.

DP0.2 tutorial notebooks 09a and 09b can only be used with uncached RSP Image "Weekly 2022_40".

Branches main and prod will usually match, except for short periods of time when updates from multiple development branches are being collected in main before "releasing" updated notebooks in a single PR to prod. It is the prod branch which appears automatically in IDF RSP users' home directories. Any user who obtains this repository with git clone can switch from main to prod using git checkout prod to ensure they are using the "released" versions.

The documentation for DP0 can be found at dp0-2.lsst.io and dp0-3.lsst.io.

Get support or report issues

To ask a question or report an issue (e.g., a bug), use either the Data Preview 0 category of the Rubin Community Forum, or submit a GitHub Issue on the Support repository.

Please don't submit GitHub Issues in this repository! Rubin staff only monitor the Support repository and the Community Forum.

Acknowledgements

These notebooks have drawn from these repositories:

Many of the tutorial notebooks in this repository were originally developed by Stack Club members or Rubin staff, and have been altered and updated to be appropriate for DP0.

Contribute

Want to contribute a tutorial? Contact Melissa Graham via direct message at Community.lsst.org.

Licensing and Re-use

The content of these notebooks are licensed under the Apache 2.0 License. That said, the use of the Vera C. Rubin logo is reserved. Thus, the content of these notebooks may be re-used, but the header and logo at the top of each notebook should not. Those re-using the contents of these notebooks should remove the original header and logo and provide their own, and not use the Rubin Observatory branding.

Notebook descriptions, learning levels, packages, data products, and skills

Skills in DP0.2 Tutorial Notebooks
01. Introduction to DP0.2
Level: Beginner
Description: Use the Jupyter Notebooks and Rubin python packages to access LSST data products.
Skills: Execute python code in a Jupyter Notebook. Use the TAP service to retrieve Object catalog data. Use the Butler to retrieve and display a deepCoadd image.
Data Products: Object table; deepCoadd images
Packages: lsst.rsp.get_tap_service, lsst.rsp.retrieve_query, lsst.daf.butler, lsst.afw.display, lsst.geom, pandas, matplotlib
02a. Introduction to TAP
Level: Beginner
Description: Explore the DP0.2 catalogs with the TAP service.
Skills: Use the TAP service. Make simple ADQL queries. Visualize retrieved datasets.
Data Products: Catalog schema. Object table.
Packages: lsst.rsp
02b. Catalog Queries with TAP
Level: Intermediate
Description: Execute complex ADQL queries with the TAP service. Visualize catalog data sets.
Skills: Use advanced ADQL and TAP functionality. Visualize retrieved datasets with bokeh and holoviews
Data Products: Object, ForcedSource, CcdVisit tables.
Packages: lsst.rsp, bokeh, pandas
02c. Image Queries with TAP
Level: Intermediate
Description: Retrieve and display images using the ObsTAP service.
Skills: Query for image data via the TAP service; retrieve image data with PyVO and Datalink; display and save images.
Data Products: calexp, deepCoadd
Packages: lsst.rsp, lsst.afw, pyvo.dal.adhoc, urllib.request
03a. Image Display and Manipulation
Level: Beginner
Description: Learn how to display and manipulate images using the LSST Science Pipelines.
Skills: Display and manipulate images, explore image mask planes, create cutouts and RGB images.
Data Products: calexp and deepCoadd images; Object table
Packages: lsst.afw.display, lsst.daf.butler, lsst.geom, lsst.afw.image
03b. Image Display with Firefly
Level: Beginner
Description: Use the Firefly interactive interface for image data.
Skills: Using Firefly as the display interface; visualizing images and their masks; overlaying sources on images.
Data Products: calexp, deepCoadd_calexp images; src, deepCoadd_forced_src, deepCoadd_ref tables
Packages: lsst.afw.display, lsst.daf.butler
03c. Big deepCoadd Cutout
Level: Intermediate
Description: Create a large custom deepCoadd cutout.
Skills: Identify tracts and patches and combine them into a large custom deepCoadd cutout.
Data Products: deepCoadd images
Packages: lsst.ip.diffim.GetTemplateTask
04a. Introduction to the Butler
Level: Beginner
Description: Use the Butler to query DP0 images and catalogs.
Skills: Query and retrieve images and catalog data with the Butler.
Data Products: calexp, goodSeeingDiff_differenceExp, deepCoadd images; sourceTable, diaSourceTable, objectTable, forcedSourceTable, forcedSourceOnDiaObjectTable tables
Packages: lsst.daf.butler
04b. Intermediate Butler Queries
Level: Intermediate
Description: Learn to discover data and apply query constraints with the Butler.
Skills: Use the Butler registry, dataIds, and spatial and temporal constraints.
Data Products: calexps, deepCoadds, sources
Packages: lsst.daf.butler, lsst.sphgeom
05. Source Detection and Measurement
Level: Intermediate
Description: Access, display, and manipulate images; detect, deblend, and measure sources; and extract, plot, and use object footprints.
Skills: Run source detection, deblending and measurement. Use source "footprints". Do forced photometry.
Data Products: DP0.2 processed visit images and catalogs.
Packages: lsst.pipe.tasks.characterizeImage, lsst.meas.algorithms.detection.SourceDetectionTask, lsst.meas.deblender.SourceDeblendTask, lsst.meas.base.SingleFrameMeasurementTask, lsst.meas.base.ForcedMeasurementTask
06a. Interactive Image Visualization
Level: Intermediate
Description: Create interactive image visualizations with the HoloViews and Bokeh open-source python libraries.
Skills: Visualize exposure images with HoloViews, interact with images with HoloViews Streams and Dynamic Map.
Data Products: calexp, deepCoadd images; Source, Object tables
Packages: bokeh, holoviews
06b. Interactive Catalog Visualization
Level: Intermediate
Description: Create interactive catalog visualizations for large datasets with HoloViews, Bokeh, and Datashader.
Skills: Create linked interactive plots for large datasets. Use Bokeh, HoloViews, and Datashader.
Data Products: Object catalog
Packages: bokeh, holoviews, datashader
07a. DiaObject Samples
Level: Intermediate
Description: Use the DiaObject table parameters to identify a sample of time-variable objects of interest.
Skills: Use the TAP service and the DP0.2 DiaObject and DiaSource tables.
Data Products: DiaObject, DiaSource tables
Packages: lsst.rsp, astropy.cosmology, astropy.stats, numpy, matplotlib
07b. Variable Star Lightcurves
Level: Intermediate
Description: Use the DP0.2 catalogs to identify variable stars and plot their lightcurves.
Skills: Use various TAP tables, including joining multiple tables. Extract time-series photometry. Measure periods and plot phased lightcurves.
Data Products: Object, ForcedSource, CcdVisit, DiaObject, DiaSource tables
Packages: numpy, matplotlib, astropy.units, astropy.coordinates, astropy.io.fits, astropy.timeseries.LombScargle, lsst.rsp.get_tap_service
08. Truth Tables
Level: Beginner
Description: An introduction to using the truth data for the Dark Energy Science Collaboration's DC2 data set.
Skills: Use the TAP service with table joins to retrieve truth data matched to the Object catalog.
Data Products: TAP dp02_dc2_catalogs.Object, .MatchesTruth, and .TruthSummary tables
Packages: lsst.rsp.get_tap_service, lsst.rsp.retrieve_query
09a. Custom Coadd
Level: Advanced
Description: Create a custom "deepCoadd" using only a subset of the input visits.
Skills: Use of pipetasks for image coaddition. Creating and writing to Butler collections. Properties of deepCoadds.
Data Products: visitTable, deepCoadd images
Packages: lsst.daf.butler, lsst.ctrl.mpexec, lsst.pipe.base
09b. Custom Coadd Sources
Level: Advanced
Description: Detect and measure sources in a custom "deepCoadd" image.
Skills: Use the Butler on user-generated collections. Run source detection and measurement.
Data Products: user-generated deepCoadd; DP0.2 deepCoadd image and Object table
Packages: lsst.afw, lsst.pipe, lsst.meas
10. Deblender Data Products
Level: Beginner
Description: Use the outputs of the multiband deblender to explore the footprints of parent and child objects.
Skills: Use of the catalog data products related to deblending objects.
Data Products: TAP dp02_dc2_catalogs.Object, Butler objectTable and deepCoadd_deblendedFlux table
Packages: lsst.afw.image, lsst.afw.detection, lsst.rsp, lsst.daf.butler, lsst.geom
11. Working with User Packages
Level: Beginner
Description: An example of how to install and set up user packages.
Skills: Installing sofware, building libraries, and modifying paths.
Data Products: N/A
Packages: os, bagpipes, PyMultiNest, MultiNest, PyCuba, Cuba
12a. Point Spread Function Data Products
Level: Intermediate
Description: A demonstration of how to access calexp and deepCoadd PSF properties.
Skills: Use of single-epoch and coadded PSF models.
Data Products: DP0.2 calexp and deepCoadd images.
Packages: lsst.daf.butler, lsst.geom, lsst.afw.display
12b. Point Spread Function Science Demo
Level: Advanced
Description: Demonstration of the use of measured PSF properties in weak lensing analysis.
Skills: Use of the catalog data products for PSF analysis.
Data Products: DP0.2 Object and Source catalogs.
Packages: lsst.analysis.tools.atools
13a. Using The Image Cutout Tool With DP0.2
Level: Beginner
Description: This notebook demonstrates how to use the Rubin Image Cutout Service.
Skills: Run the Rubin Image Cutout Service for visual inspection of small cutouts of LSST images.
Data Products: Images (deepCoadd, calexp), catalogs (objectTable, diaObject, truthTables, ivoa.ObsCore).
Packages: PyVO, lsst.rsp.get_tap_service, lsst.pipe.tasks.registerImage, lsst.afw.display
14. Injecting Synthetic Sources Into Single-Visit Images
Level: Advanced
Description: This tutorial demonstrates a method to inject artificial sources (stars and galaxies) into calexp images using the measured point-spread function of the given calexp image. Confirmation that the synthetic sources were correctly injected into the image is done by running a difference imaging task from the pipelines.
Skills: Use the source_injection tools to inject synthetic sources into images. Create a difference image from a calexp with injected sources.
Data Products: Butler calexp images and corresponding src catalogs, goodSeeingDiff_templateExp images, and injection_catalogs.
Packages: lsst.source.injection
15. Survey Property Maps
Level: Intermediate
Description: Use the tools to visualize full-area survey property maps.
Skills: Load and visualize survey property maps using healsparse and skyproj.
Data Products: Survey property maps.
Packages: healsparse, skyproj, lsst.daf.butler
Skills in DP0.3 Tutorial Notebooks
01. Introduction to DP0.3
Level: Beginner
Description: An overview of the contents of the DP0.3 moving object catalogs.
Skills: Use the TAP service and ADQL to access the DP0.3 tables.
Data Products: TAP dp03_catalogs
Packages: lsst.rsp.get_tap_service
02. Properties of Main Belt Asteroids in DP0.3
Level: Beginner
Description: An exploration of the orbital properties of Main Belt asteroids in the DP0.3 catalogs.
Skills: Use the TAP service and ADQL to access the DP0.3 tables. Join information from multiple DP0.3 tables. Plot orbits of Solar System objects.
Data Products: TAP dp03_catalogs
Packages: lsst.rsp.get_tap_service
03. Trans-Neptunian Objects (TNOs)
Level: intermediate
Description: Explore the trans-Neptunian object populations in DP0.3.
Skills: Use of the DP0.3 catalogs to study TNO populations.
Data Products: DP0.3 catalogs SSObject, DiaSource, and MPCORB (10-year catalogs).
Packages: lsst.rsp
04a. Introduction to Phase Curves
Level: Intermediate
Description: Explore phase curves for DP0.3 solar system objects.
Skills: Use the TAP service and ADQL to access the DP0.3 tables. Join information from multiple DP0.3 tables. Plot phase curves.
Data Products: TAP dp03_catalogs
Packages: lsst.rsp.get_tap_service
04b. Advanced Phase Curve Modeling
Level: Advanced
Description: Explicitly investigate the derivation of phase curves for Main Belt asteroids.
Skills: Use the TAP service and ADQL to access the DP0.3 tables. Join information from multiple DP0.3 tables. Derive phase curves using three different models.
Data Products: TAP dp03_catalogs with added columns
Packages: lsst.rsp.get_tap_service
05. Orbital properties of Near-Earth Objects (NEOs) in DP0.3
Level: Beginner
Description: Exploration of the orbital properties of near-Earth objects in the DP0.3 catalogs.
Skills: Use TAP queries to retrieve Solar System objects. Plot propoerties and orbits of near-Earth objects.
Data Products: TAP DP0.3 MPCORB (10-year) and SSObject tables
Packages: lsst.rsp.get_tap_service
06. Using User-supplied Catalogs
Level: Beginner
Description: Use the TAP upload functionality for user-supplied tables and join them with DP0.3 catalogs.
Skills: Use the TAP service to upload a table and join it to an LSST table with ADQL.
Data Products: DP0.3 catalogs SSObject, DiaSource, and MPCORB (10-year catalogs).
Packages: lsst.rsp.get_tap_service
07. Interactive Catalog Visualization
Level: Intermediate
Description: Interactive Solar System catalog data visualizations with three open-source python libraries.
Skills: Create linked interactive plots for large datasets, and output interactive plots to interactive HTML files. Use Bokeh, HoloViews, and Datashader.
Data Products: Solar System Object catalogs
Packages: bokeh, holoviews, datashader

Looking for the DP0.1 tutorial notebooks?

DP0.1 was deprecated and is no longer available.

Although they cannot be executed, the tutorial notebooks can still be found in this archived repository: tutorial-notebooks-dp01-archive.

About

Tutorial Jupyter Notebooks for Data Preview 0, created and maintained by the Rubin Observatory Community Science Team.

Resources

License

Stars

Watchers

Forks

Packages

No packages published