-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
JP-3794: Add trace-based extraction option for NIRSpec #9022
JP-3794: Add trace-based extraction option for NIRSpec #9022
Conversation
Expand trace-based extraction to all use_source_posn cases and add a general position_offset option.
I've updated the original comment above and documentation to reflect that this is now an update to generalize the extract_1d use_source_posn to work with non-resampled data (thanks @melanieclarke for the modification!). I'll go ahead and pull this out of draft, let me know if there is anything else that should be changed/addressed. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9022 +/- ##
==========================================
+ Coverage 76.90% 76.92% +0.02%
==========================================
Files 498 498
Lines 45769 45810 +41
==========================================
+ Hits 35199 35240 +41
Misses 10570 10570
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Thanks @hayescr! I'll start regression tests and tag someone else to review as well, since I contributed code. |
Co-authored-by: Melanie Clarke <[email protected]>
Regression tests running here: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are some unrelated failures for imaging modes in the regression tests; otherwise, the only change is to the NIRSpec BOTS extracted spectrum in spec2, as expected. I think this is good to go.
@tapastro - can you review as well, since I contributed some code for this one? |
Or maybe @jemorrison can review, since this touches on the PSF extraction work. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, though I'm curious - why set use_source_posn to False for the regression test? It appears as though the new code will be used by default for BOTS data - do we expect a parameter reference file update that reflects this, setting it to False?
No, we want use_source_posn to be True for BOTS with these changes, from now on. I set it to False for that regression test because it was specifically meant to test custom extraction parameters with polynomial coefficients for source and background regions. The coefficients I originally defined for that test were not intended to be shifted by the source location -- they include the source position directly. There is another BOTS regression test for spec2 that exercises the default extraction -- that one shows the intended changes. |
Ah, the downsides of reading the change snippets without context. 🙂 Thanks! |
@melanieclarke I have a basic question. Where in the code does it know the data is not resampled and a curved traced should be extracted. |
@jemorrison it doesn't explicitly check for resampled data to decide whether or not to extract a trace. In the case of resampled data the trace is straight and the trace array is filled with the "location" value that was previously used by the use_source_posn. So the behavior of use_source_posn is effectively the same, even though it may technically be using the "trace" for the extraction. |
@hayescr Ok can you point me to were the shape of the trace is determined - mainly for the unresampled data. |
return full_trace | ||
|
||
|
||
def _miri_trace_from_wcs(shape, bounding_box, wcs_ref, source_ra, source_dec): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jemorrison here (_miri_trace_from_wcs) is the relevant function that calculates the trace for MIRI data. The equivalent function for NIRSpec is similar other than the differences in MIRI/NIRSpec WCS and can be found above (_nirspec_trace_from_wcs).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And here is where _miri_trace_from_wcs is called in location_from_wcs, in case it is useful to work backward and see what inputs are provided:
jwst/jwst/extract_1d/extract.py
Line 1101 in 60eedab
trace = _miri_trace_from_wcs(shape, bb, wcs, dithra, dithdec) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me. I really like how the 'trace' is extracted depending on the wcs information.
Resolves JP-3794
Closes #8938
This PR expands the use_source_posn option in extract_1d to work for curved, non-resampled cal and calints spectra (specifically for NIRSpec BOTS/MOS/FS and MIRI LRS fixed slit). It does so by updating the use_source_posn option to derives a source trace from WCS and expected source positions and performs a box extraction around the trace if an extraction width has been supplied. This PR also turns on the use_source_position by default for NIRSpec BOTS data, and adds a "position_offset" step parameter that allows users to supply cross-dispersion offsets to the pipeline extraction regions.
Tasks
Build 11.3
(use the latest build if not sure)no-changelog-entry-needed
)changes/
:echo "changed something" > changes/<PR#>.<changetype>.rst
(see below for change types)docs/
pageokify_regtests
to update the truth filesnews fragment change types...
changes/<PR#>.general.rst
: infrastructure or miscellaneous changechanges/<PR#>.docs.rst
changes/<PR#>.stpipe.rst
changes/<PR#>.datamodels.rst
changes/<PR#>.scripts.rst
changes/<PR#>.fits_generator.rst
changes/<PR#>.set_telescope_pointing.rst
changes/<PR#>.pipeline.rst
stage 1
changes/<PR#>.group_scale.rst
changes/<PR#>.dq_init.rst
changes/<PR#>.emicorr.rst
changes/<PR#>.saturation.rst
changes/<PR#>.ipc.rst
changes/<PR#>.firstframe.rst
changes/<PR#>.lastframe.rst
changes/<PR#>.reset.rst
changes/<PR#>.superbias.rst
changes/<PR#>.refpix.rst
changes/<PR#>.linearity.rst
changes/<PR#>.rscd.rst
changes/<PR#>.persistence.rst
changes/<PR#>.dark_current.rst
changes/<PR#>.charge_migration.rst
changes/<PR#>.jump.rst
changes/<PR#>.clean_flicker_noise.rst
changes/<PR#>.ramp_fitting.rst
changes/<PR#>.gain_scale.rst
stage 2
changes/<PR#>.assign_wcs.rst
changes/<PR#>.badpix_selfcal.rst
changes/<PR#>.msaflagopen.rst
changes/<PR#>.nsclean.rst
changes/<PR#>.imprint.rst
changes/<PR#>.background.rst
changes/<PR#>.extract_2d.rst
changes/<PR#>.master_background.rst
changes/<PR#>.wavecorr.rst
changes/<PR#>.srctype.rst
changes/<PR#>.straylight.rst
changes/<PR#>.wfss_contam.rst
changes/<PR#>.flatfield.rst
changes/<PR#>.fringe.rst
changes/<PR#>.pathloss.rst
changes/<PR#>.barshadow.rst
changes/<PR#>.photom.rst
changes/<PR#>.pixel_replace.rst
changes/<PR#>.resample_spec.rst
changes/<PR#>.residual_fringe.rst
changes/<PR#>.cube_build.rst
changes/<PR#>.extract_1d.rst
changes/<PR#>.resample.rst
stage 3
changes/<PR#>.assign_mtwcs.rst
changes/<PR#>.mrs_imatch.rst
changes/<PR#>.tweakreg.rst
changes/<PR#>.skymatch.rst
changes/<PR#>.exp_to_source.rst
changes/<PR#>.outlier_detection.rst
changes/<PR#>.tso_photometry.rst
changes/<PR#>.stack_refs.rst
changes/<PR#>.align_refs.rst
changes/<PR#>.klip.rst
changes/<PR#>.spectral_leak.rst
changes/<PR#>.source_catalog.rst
changes/<PR#>.combine_1d.rst
changes/<PR#>.ami.rst
other
changes/<PR#>.wfs_combine.rst
changes/<PR#>.white_light.rst
changes/<PR#>.cube_skymatch.rst
changes/<PR#>.engdb_tools.rst
changes/<PR#>.guider_cds.rst