Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

desi_zcatalog --recoadd-fibermap option #2109

Merged
merged 1 commit into from
Sep 1, 2023
Merged

desi_zcatalog --recoadd-fibermap option #2109

merged 1 commit into from
Sep 1, 2023

Conversation

sbailey
Copy link
Contributor

@sbailey sbailey commented Aug 31, 2023

This PR fixes #2087 by adding a desi_zcatalog --recoadd-fibermap option to recoadd the spectra FIBERMAP -> FIBERMAP, EXP_FIBERMAP instead of propagating the previously coadded versions from the redrock files. This enables us to rebuild the stacked redshift catalogs while bringing in the bug fixes from e.g. PRs #2100 (IN_COADD_B/R/Z), #2094 (target bit bitwise OR), and #2093 (COADD_FIBERSTATUS bug fixes).

A companion option --ztile flags whether this is a tile-based coadd (vs. a healpix based coadd). I plan a separate PR to simplify the desi_zcatalog options and this might change again, but I've tried to focus this PR on the minimal changes needed to support adding re-coadding.

One non-minimal change for simplicity is dropping support for the "ZBEST" HDU (renamed "REDSHIFTS") and dropping support for missing TSNR2 and EXP_FIBERMAP; all of these cases were for internal releases that pre-date Fuji/EDR.

@sbailey
Copy link
Contributor Author

sbailey commented Aug 31, 2023

I forgot to mention: example outputs are in /pscratch/sd/s/sjbailey/desi/dev/zcat_recoadd/:

  • ztile-sv2-bright-orig.fits: regenerating the iron ztile-sv2-bright-cumulative catalog without re-coadding
  • ztile-sv2-bright-recoadd.fits: including --recoadd-fibermap --ztile

the differences are due to the extra columns introduced in #2065 and #2100, and changes to COADD_FIBERSTATUS and MEAN/STD coadded columns due to bug fixes to fibermap coadds #2065, #2093, #2094, #2100.

@sbailey sbailey mentioned this pull request Sep 1, 2023
Copy link
Member

@akremin akremin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the changes are minimal and look good to me. The differences show that the new coadd fibermap has the correct values thanks to the updated code, while not showing any other differences.
The loss of backward compatibility is unfortunate, but after more than two years I think that is appropriate. We can use an older code tag if we need to work with earlier data.

@akremin akremin merged commit 3d32447 into main Sep 1, 2023
24 checks passed
@akremin akremin deleted the zcat_recoadd branch September 1, 2023 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add desi_zcatalog --recoadd-fibermap option
2 participants