Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

desi_group_spectra header propagation cleanup #2302

Merged
merged 3 commits into from
Jul 24, 2024
Merged

Conversation

sbailey
Copy link
Contributor

@sbailey sbailey commented Jul 22, 2024

This PR fixes the desi_group_spectra large number of warnings from incorrectly merging per-exposure keywords reported in #2200.

The vast majority of keywords in the frame FIBERMAP HDUs are inherited from the raw data, and either conflict when combined (humidities, temperatures, exposure times...) or are fairly meaningless for a fibermap even if they don't conflict (CCD dimensions, clock settings...). This PR treats it as an opt-in list to propagate: by default only

    keep_keywords = ['SURVEY', 'PROGRAM', 'EXTNAME', 'LONGSTRN',
                     'INSTRUME', 'OBSERVAT', 'OBS-LAT', 'OBS-LONG', 'OBS-ELEV', 'TELESCOP']

and if onetile=True it also propagates keywords related to the input fiberassignment, which apply to all observations of a single tile but not coadds across tiles (healpix).

I added some basic unit tests to check this functionality, and also tested by rerunning two Jura cases in /global/cfs/cdirs/desi/users/sjbailey/dev/mergehdr:

import desispec.scripts.group_spectra

desispec.scripts.group_spectra.main(['--inframes', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240118/00219813/cframe-b0-00219813.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240118/00219813/cframe-r0-00219813.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240118/00219813/cframe-z0-00219813.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222119/cframe-b0-00222119.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222119/cframe-r0-00222119.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222119/cframe-z0-00222119.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222120/cframe-b0-00222120.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222120/cframe-r0-00222120.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222120/cframe-z0-00222120.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222121/cframe-b0-00222121.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222121/cframe-r0-00222121.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20240128/00222121/cframe-z0-00222121.fits.gz', '--outfile', 'spectra-0-3884-thru20240128.fits.gz', '--coaddfile', 'coadd-0-3884-thru20240128.fits', '--onetile', '--header', 'SPGRP=cumulative', 'SPGRPVAL=20240128', 'NIGHT=20240128', 'TILEID=3884', 'SPECTRO=0', 'PETAL=0'])

desispec.scripts.group_spectra.main(['--inframes', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089278/cframe-b7-00089278.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089278/cframe-r7-00089278.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089278/cframe-z7-00089278.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089278/cframe-b8-00089278.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089278/cframe-r8-00089278.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089278/cframe-z8-00089278.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089280/cframe-b2-00089280.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089280/cframe-r2-00089280.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20210519/00089280/cframe-z2-00089280.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230808/00189277/cframe-b0-00189277.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230808/00189277/cframe-r0-00189277.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230808/00189277/cframe-z0-00189277.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230808/00189277/cframe-b9-00189277.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230808/00189277/cframe-r9-00189277.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230808/00189277/cframe-z9-00189277.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230815/00190734/cframe-b0-00190734.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230815/00190734/cframe-r0-00190734.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230815/00190734/cframe-z0-00190734.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230815/00190734/cframe-b9-00190734.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230815/00190734/cframe-r9-00190734.fits.gz', '/global/cfs/cdirs/desi/spectro/redux/jura/exposures/20230815/00190734/cframe-z9-00190734.fits.gz', '--outfile', 'spectra-main-dark-10000.fits.gz', '--coaddfile', 'coadd-main-dark-10000.fits', '--healpix', '10000', '--header', 'SURVEY=main', 'PROGRAM=dark'])

(apologies for the very long command lines; those are taken directly from the Jura logs except for editing the output directory location).

The spectra*.fits.gz and coadd*.fits files in /global/cfs/cdirs/desi/users/sjbailey/dev/mergehdr can be compared to the versions in jura/tiles/cumulative/3884/20240128 and jura/healpix/main/dark/100/10000 .

@segasai please check.

@sbailey sbailey requested a review from segasai July 22, 2024 23:45
py/desispec/pixgroup.py Outdated Show resolved Hide resolved
Copy link
Contributor

@segasai segasai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me, other than a minor comment that I left.

@sbailey sbailey merged commit d3bff04 into main Jul 24, 2024
26 checks passed
@sbailey sbailey deleted the frames2spectra_headers branch July 24, 2024 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants