Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rcal-969 Updates to add and use skycell coord info in asn header #1583

Merged
merged 7 commits into from
Jan 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changes/1583.associations.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
This adds additional info to the asn header keyword skycell_wcs_info and updates the mosaic pipeline to use
that information to construct the skycell data from the input exposures.
2 changes: 2 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,8 @@ def check_sphinx_version(expected_version):
"-Gfontname=Helvetica Neue, Helvetica, Arial, sans-serif",
]

# activate figure numbering
numfig = True
# If true, '()' will be appended to :func: etc. cross-reference text.
# add_function_parentheses = True

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
57 changes: 57 additions & 0 deletions docs/roman/pipeline/mosaic_pipeline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,63 @@ calibrated data for the the exposures. The most convenient way to pass the list
exposures to be processed with the mosaic level pipeline is to use an association.
Instructions on how to create an input association an be found at :ref:`asn-from-list`.

The mosaic pipeline can create different types of products. In one
mode you give it a list of calibrated images and the pipeline will run
the above steps and the final product is a mosaic of the input images
resampled to a regular grid. The mode is selected based on the target
field in the association header. If the input association contains a
target field which matches a skycell name (see TBD) then the mosaic
pipeline will resample the final product onto the skycell grid.

If the association has been generated with :ref:`skycell_asn` the
skycell name and projection coordinates should be available in the
association header. If the skycell name is available and corresponds
to a valid name in the database and the projection cell coordinates
are not available in the association header then the pipeline will
read the needed information from the data file containing the skycell
information.

.. _figure-skycell-mosaic:

.. figure:: graphics/wfi_4sca_skycell.png
:scale: 60 %

Image showing the four SCA's that overlap a given skycell.


The projection of the single WFI exposure resampled to a skycell is shown in :numref:`figure-skycell-mosaic`.
The image has the portion of the four SCAs that overlap
the given skycell resampled to a regular grid on the WCS of the skycell. The gaps
between the images show the placement of the SCAs in the detector. In general
these gaps will be filled in by additional exposures in the visit.

If the target field does not contain a valid skycell name then the
image or images will be resampled to a regular grid. To resample a
single image the input will need to be an association with a single
member.

.. list-table::

* - .. figure:: graphics/mosaic_regular_grid.png
:name: figure-mosaic-regular-grid
:scale: 25 %
:align: left

An SCA resampled to a regular grid.

- .. figure:: graphics/mosaic4_regular_grid.png
:name: figure-mosaic4-regular-grid
:scale: 25 %

Four SCAs resampled to a regular grid.

.. labels in list-tables do no appear to be used outside the list-table structure, hard codeing for now.
.. :numref:`figure-mosaic-regular-grid` and :numref:`figure-mosaic4-regular-grid` show

Figures 2 & 3 show the results of the mosaic pipeline on a single SCA and on four SCA's in the WFI array.
Using the code to mosaic large areas of the sky may result on the code needing large amounts of memory,
so care is needed not to exceed your local memory limits when constructing mosaics in this manner.


Outputs
----------
Expand Down
19 changes: 14 additions & 5 deletions romancal/associations/skycell_asn.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,23 @@ def skycell_asn(filelist, output_file_root, product_type, release_product):
# grab all the wcs parameters needed for generate_tan_wcs
projcell_info = dict(
[
("name", pm.PATCH_TABLE[item]["name"]),
("pixel_scale", float(pm.PATCH_TABLE[item]["pixel_scale"])),
("ra_cent", float(pm.PATCH_TABLE[item]["ra_projection_center"])),
("dec_cent", float(pm.PATCH_TABLE[item]["dec_projection_center"])),
("shiftx", float(pm.PATCH_TABLE[item]["x0_projection"])),
("shifty", float(pm.PATCH_TABLE[item]["y0_projection"])),
(
"ra_projection_center",
float(pm.PATCH_TABLE[item]["ra_projection_center"]),
),
(
"dec_projection_center",
float(pm.PATCH_TABLE[item]["dec_projection_center"]),
),
("x0_projection", float(pm.PATCH_TABLE[item]["x0_projection"])),
("y0_projection", float(pm.PATCH_TABLE[item]["y0_projection"])),
("ra_center", float(pm.PATCH_TABLE[item]["ra_center"])),
("dec_center", float(pm.PATCH_TABLE[item]["dec_center"])),
("nx", int(pm.PATCH_TABLE[item]["nx"])),
("ny", int(pm.PATCH_TABLE[item]["ny"])),
("orient", float(pm.PATCH_TABLE[item]["orientat"])),
("orientat", float(pm.PATCH_TABLE[item]["orientat"])),
(
"orientat_projection_center",
float(pm.PATCH_TABLE[item]["orientat_projection_center"]),
Expand Down
30 changes: 18 additions & 12 deletions romancal/pipeline/mosaic_pipeline.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env python
from __future__ import annotations

import json
import logging
import re
from os.path import basename, isfile
Expand Down Expand Up @@ -86,19 +87,24 @@ def process(self, input):
product_name = input.asn["products"][0]["name"]
try:
skycell_name = input.asn["target"]
except IndexError:
except KeyError:
skycell_name = ""
skycell_record = []

# if this is a valid skycell name load the database and get the skycell record
# if this is a valid skycell name get the skycell record
if re.match(r"r\d{3}\w{2}\d{2}x\d{2}y\d{2}", skycell_name):
if patch_match.PATCH_TABLE is None:
patch_match.load_patch_table()
if patch_match.PATCH_TABLE is None:
raise RuntimeError("No patch table has been loaded")
skycell_record = patch_match.PATCH_TABLE[
np.where(patch_match.PATCH_TABLE["name"][:] == skycell_name)[0][0]
]
# check to see if the skycell coords are in the asn header if
# so read the string and convert to a dictionary to match the patch table
try:
skycell_record = json.loads(input.asn["skycell_wcs_info"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an example association file with this input that you would share? Looking at this line of code makes me think this is expecting a json encoded object stored as a string inside the asn. Is that correct? If so, what's the reason for json encoding the skycell_wcs_info (instead of just allowing it to be encoded along with the other structured content of the asn)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can create an association using skycell_asn and any level 2 file.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming that means there is no example association. If I generate one with skycell_asn it's garbled which confirms the double json encoding I asked about above. Since this PR is merged I opened a ticket for the issue: https://jira.stsci.edu/browse/RCAL-992

except (KeyError, json.JSONDecodeError):
if patch_match.PATCH_TABLE is None:
patch_match.load_patch_table()
skycell_record = patch_match.PATCH_TABLE[
np.where(patch_match.PATCH_TABLE["name"][:] == skycell_name)[0][
0
]
]
log.info("Skycell record %s:", skycell_record)

if skycell_name in skycell_record["name"]:
Expand Down Expand Up @@ -140,7 +146,7 @@ def process(self, input):
result_catalog = self.sourcecatalog.run(result)
else:
raise NotImplementedError(
"resampling a mosaic file is not yet supported"
"Overwriting an exisiting file or resampling a mosaic file is not yet supported"
)

else:
Expand All @@ -157,8 +163,8 @@ def process(self, input):


def generate_tan_wcs(skycell_record):
# extract the wcs info from the record for generate_tan_wcs
# we need the scale, ra, dec, bounding_box
"""extract the wcs info from the record for generate_tan_wcs
we need the scale, ra, dec, bounding_box"""

scale = float(skycell_record["pixel_scale"])
ra_center = float(skycell_record["ra_projection_center"])
Expand Down
Loading