Process JEDI radiance files as a single tarball

[RussTreadon-NOAA]

What new functionality do you need?

At present JEDI radiance bias correction uses three files per satellite_sensor. This result in many individual files to move, copy, link, archive. While JEDI applications need to work with individual files, file operations are more easily applied to a single file.

This can be achived by tar'ing all JEDI radiance bias correction files into a single file. This single file is what gets moved, copied, linked, and archived. Individual radiance bias correction files can be obtained by untar'ing the radiance tarball in the run directory of specific jobs.

What are the requirements for the new functionality?

Update JEDI DA scripts which currently process individual radiance bias correction files to process a single radiance tarball.

Acceptance Criteria

Once work for this issue is complete, the COM directory will contain a single radiance bias correction file per cycle. Analysis results when using this tarball should be identical to results obtained by using individual radiance bias correction files.

Suggest a solution (optional)

No response

[RussTreadon-NOAA]

Work for this issue will be done in RussTreadon-NOAA:feature/radbcor

[RussTreadon-NOAA]

Hera g-w C96C48_ufs_hybatmDA CI

Install feature/radbcor at 20fb88c on Hera. Set up g-w CI for C96C48_ufs_hybatmDA.

Note, for these test specify ICSDIR_ROOT=/scratch1/NCEPDEV/global/Russ.Treadon/data/ICSDIR. This is copy of the glopara ICSDIR_ROOT with the satbias and tlapse radiance bias correction files for C96C48_ufs_hybatmDA replaced with a single gdas.t18z.radbcor tarball.

All enkfgdas, gdas, and gfs DA jobs successfully run to completion. Only 20240224 00Z gfs forecast and post jobs remain running.

The enkfgdas, gdas, and gfs init jobs extracted satellite_sensor radiance bias correction files from the radbcor tarball. The final job created a radbcor tarball with updated coefficient and copied the file to the appropriate COM directory. The gdas archive job included the radbcor tarball in gdas_restarta.tar.

This test will be repeated using the official ICSDIR_ROOT once issue #2869 is closed.

[RussTreadon-NOAA]

GDASApp PR #1267 updates the drivers for the atmospheric local ensemble and variational ctests to process the radiance bias correction tarball instead of individual satellite_sensor bias correction files. The sorc/gdas.cd hash in g-w feature/radbcor will be updated a new GDASApp hash once PR #1267 is approved and merged into GDASApp develop.