desi_group_spectra header propagation cleanup #2302
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes the desi_group_spectra large number of warnings from incorrectly merging per-exposure keywords reported in #2200.
The vast majority of keywords in the frame FIBERMAP HDUs are inherited from the raw data, and either conflict when combined (humidities, temperatures, exposure times...) or are fairly meaningless for a fibermap even if they don't conflict (CCD dimensions, clock settings...). This PR treats it as an opt-in list to propagate: by default only
and if
onetile=True
it also propagates keywords related to the input fiberassignment, which apply to all observations of a single tile but not coadds across tiles (healpix).I added some basic unit tests to check this functionality, and also tested by rerunning two Jura cases in /global/cfs/cdirs/desi/users/sjbailey/dev/mergehdr:
(apologies for the very long command lines; those are taken directly from the Jura logs except for editing the output directory location).
The spectra*.fits.gz and coadd*.fits files in /global/cfs/cdirs/desi/users/sjbailey/dev/mergehdr can be compared to the versions in jura/tiles/cumulative/3884/20240128 and jura/healpix/main/dark/100/10000 .
@segasai please check.