Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update hashes for JEDI submodules #1117

Closed
RussTreadon-NOAA opened this issue May 20, 2024 · 13 comments · Fixed by #1123
Closed

Update hashes for JEDI submodules #1117

RussTreadon-NOAA opened this issue May 20, 2024 · 13 comments · Fixed by #1123
Assignees

Comments

@RussTreadon-NOAA
Copy link
Contributor

RussTreadon-NOAA commented May 20, 2024

The hashes for the core JEDI component submodules in sorc/ are several weeks old. This issue is opened to document the updating of the hashes for these JEDI components.

Note: The updating of sorc/soca is documented in GDASApp issue #1093

@RussTreadon-NOAA RussTreadon-NOAA self-assigned this May 20, 2024
@RussTreadon-NOAA
Copy link
Contributor Author

Work for this issue will be done in RussTreadon-NOAA:/feature/submodules

RussTreadon-NOAA added a commit to RussTreadon-NOAA/GDASApp that referenced this issue May 20, 2024
@RussTreadon-NOAA
Copy link
Contributor Author

Commit d480014 to feature/submodules points oops at bugfix/lgetkf_addincvars at 912da6d. The current head of oops develop, 8b50896, causes test_gdasapp_atm_jjob_ens_run to abort with the error message described in OOPS PR #2645. The changes in bugfix/lgetkf_addincvars are required for this ctest to pass. The oops submodule hash in feature/submodules will be updated once PR #2645 is merged into oops develop

@RussTreadon-NOAA
Copy link
Contributor Author

RussTreadon-NOAA commented May 20, 2024

The updated JEDI hashes bring in changes to the names of several radiance bias correction variables.

old new
lapse_rate lapseRate
lapse_rate_order_2 lapseRate_order_2
emissivity emissivityJacobian
scan_angle sensorScanAngle
scan_angle_order_2 sensorScanAngle_order_2
scan_angle_order_3 sensorScanAngle_order_3
scan_angle_order_4 sensorScanAngle_order_4

As such, the GDASApp radiance bias correction conversion script needs to be updated accordingly. Required changes are made to

  • test/atm/test_convert_gsi_satbias.sh - correct path to satbias2ioda.x
  • ush/run_satbias_conv.py - update bias correction variable names

at ba81096 and 18b6058

The var and ens ctests assimilate amsua_n19 data with bias correction. The bias correction variable names encoded in the following bias correction files need to be updated in order for the var and ens analysis ctests to pass.

  • $GDASAPP_TESTDATA/lowres/gdas.20210323/12/analysis/atmos/gdas.t12z.amsua_n19.satbias.nc4
  • $GDASAPP_TESTDATA/lowres/gdas.20210323/12/analysis/atmos/gdas.t12z.amsua_n19.satbias_cov.nc4

These files have been updated and are available in the following directory on Orion

  • /work2/noaa/da/rtreadon/CI/GDASApp/data/lowres/gdas.20210323/12/analysis/atmos
Orion-login-3:/work2/noaa/da/rtreadon/CI/GDASApp/data/lowres/gdas.20210323/12/analysis/atmos$ ls -l gdas.t12z.amsua_n19.satbias*
-rw-r--r-- 1 rtreadon da 22260 May 18 13:00 gdas.t12z.amsua_n19.satbias_cov.nc4
-rw-r--r-- 1 rtreadon da 22260 May 18 13:00 gdas.t12z.amsua_n19.satbias.nc4

@CoryMartin-NOAA : We need to update the amsua_n19 satbias files when feature/submodules is merged into develop

@RussTreadon-NOAA
Copy link
Contributor Author

In addition to updating variable names in the test data amsua_n19 bias correction files and the GDASApp bias correction conversion scripts, we also need to update bias correction variable names in the following jcb-gdas yamls

  • observations/atmosphere-lgetkf/amsua_n19.yaml.j2
  • observations/atmosphere/amsua_n19.yaml.j2

Note that similar changes need to be made to the yamls for other radiance data in observations/atmosphere/ The changes for amsua_n19 are mentioned in this issue since GDASApp ctests process amsua_n19 data.

Updating the JEDI submodule hashes also requires changes be made to the following jcb-gdas yamls

  • model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2
  • model/atmosphere/background_error_static_gsibec.yaml.j2

In both of these yamls saber block name gsi covariance is replaced with gsi static covariance

jcb-gdas issue #6 documents the addition of the above changes to jcb-gdas branch feature/submodules.

RussTreadon-NOAA added a commit to RussTreadon-NOAA/GDASApp that referenced this issue May 20, 2024
RussTreadon-NOAA added a commit to RussTreadon-NOAA/GDASApp that referenced this issue May 20, 2024
RussTreadon-NOAA added a commit to RussTreadon-NOAA/GDASApp that referenced this issue May 20, 2024
@guillaumevernieres
Copy link
Contributor

@RussTreadon-NOAA , soca fixes are in the feature/submodules_soca branch. I was going to issue a pr into your branch ... but you're working on a fork. Any-chance you could merge my branch into yours?

@RussTreadon-NOAA
Copy link
Contributor Author

@guillaumevernieres , thank you for the soca updates. I merged feature/submodules_soca into feature/submodules at f3cae35.

sorc/soca in feature/submodules dates back to 3/27/2024 (hash 18f7813d). This is the same soca hash as in the authoritative GDASApp develop.

The current head of soca develop is 98e2be9 from last week. Should I update to a more recent soca hash?

@RussTreadon-NOAA
Copy link
Contributor Author

RussTreadon-NOAA commented May 20, 2024

Update to soca @ 98e2be9. Build of feature/submodules at d01dac0 aborted with

In file included from /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/bundle/gdas-utils/soca/gdas_socahyb\
ridweights.cc(1):
/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/bundle/gdas-utils/soca/gdas_socahybridweights.h(34): erro\
r: class "soca::Geometry" has no member "latlon"
        geom.latlon(lats, lons, halo);
             ^

compilation aborted for /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/bundle/gdas-utils/soca/gdas_socah\
ybridweights.cc (code 2)
make[2]: *** [gdas-utils/soca/CMakeFiles/gdas_socahybridweights.x.dir/gdas_socahybridweights.cc.o] Error 2
make[2]: Leaving directory `/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build'
make[1]: *** [gdas-utils/soca/CMakeFiles/gdas_socahybridweights.x.dir/all] Error 2

@guillaumevernieres - ignore this comment. My local working copy did not contain your soca fixes. Let me retest and edit this comment after tests are complete.

@RussTreadon-NOAA
Copy link
Contributor Author

Update to soca @ 98e2be9. Build of feature/submodules at d01dac0 successfully completed.

test_gdasapp tests run with the following results

98% tests passed, 1 tests failed out of 46

Label Time Summary:
gdas-utils    =  22.48 sec*proc (9 tests)
script        =  22.48 sec*proc (9 tests)

Total Test time (real) = 1539.73 sec

The following tests FAILED:
        1827 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)

Look in log file for test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST and see the following traceback

+ JGDAS_GLOBAL_OCEAN_ANALYSIS_POST[35]: /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/scripts/exgdas_global_marine_analysis_post.py
2024-05-20 23:54:55,647 - INFO     - root        : ---------------- Copy from RUNDIR to COMOUT
Traceback (most recent call last):
  File "/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/scripts/exgdas_global_marine_analysis_post.py", line 104, in <module>
    os.path.join(com_ice_analysis, f'{cice_rst_date}.cice_model_anl.res.nc')])
  File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/gdasapp/lib/python3.7/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Script exgdas_global_marine_analysis_post.py contains

com_ocean_analysis = os.getenv('COM_OCEAN_ANALYSIS')
com_ice_analysis = os.getenv('COM_ICE_ANALYSIS')
com_ice_restart = os.getenv('COM_ICE_RESTART')

but variable COM_ICE_ANALYSIS is not defined in the parent j-job. g-w PR #2584 adds COM_ICE_ANALYSIS to JGDAS_GLOBAL_OCEAN_ANALYSIS_POST.

We need g-w PRs #2584 and #2592 merged into g-w develop to see all test_gdasapp ctests pass when using forked GDASApp branch feature/submodules.

GDASApp develop is ahead of what g-w can actually run.

Attention @guillaumevernieres , @DavidNew-NOAA , @CoryMartin-NOAA , and @danholdaway

@guillaumevernieres
Copy link
Contributor

Ha, sorry. You need to merge develop into your branch @RussTreadon-NOAA . I commented out the post job for now, I'll turn it back on after my g-w pr is merged.

@RussTreadon-NOAA
Copy link
Contributor Author

RussTreadon-NOAA commented May 21, 2024

Merge GDASApp develop at 39cf2b2 into RussTreadon-NOAA:feature/submodules. Done at 11e8e3b.

Update working copy of DavidNew-NOAA:feature/atmensanlfv3inc to hash 7ce5b3c. This hash is in sync with the current head, 9aad86f, of g-w develop.

feature/submodules is manually cloned into sorc/gdas.cd in the working copy of David's branch. Build GDASApp inside g-w and rerun test_gdasapp. All soca tests now pass. Unfortunately, three different tests now fail.

Test project /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build
      Start 1468: test_gdasapp_util_coding_norms
 1/45 Test #1468: test_gdasapp_util_coding_norms ........................   Passed    4.41 sec
      Start 1469: test_gdasapp_util_ioda_example
 2/45 Test #1469: test_gdasapp_util_ioda_example ........................   Passed    5.59 sec
      Start 1470: test_gdasapp_util_prepdata
 3/45 Test #1470: test_gdasapp_util_prepdata ............................   Passed    2.54 sec
      Start 1471: test_gdasapp_util_rads2ioda
 4/45 Test #1471: test_gdasapp_util_rads2ioda ...........................   Passed    0.87 sec
      Start 1472: test_gdasapp_util_ghrsst2ioda
 5/45 Test #1472: test_gdasapp_util_ghrsst2ioda .........................   Passed    0.15 sec
      Start 1473: test_gdasapp_util_smap2ioda
 6/45 Test #1473: test_gdasapp_util_smap2ioda ...........................   Passed    0.12 sec
      Start 1474: test_gdasapp_util_smos2ioda
 7/45 Test #1474: test_gdasapp_util_smos2ioda ...........................   Passed    0.38 sec
      Start 1475: test_gdasapp_util_viirsaod2ioda
 8/45 Test #1475: test_gdasapp_util_viirsaod2ioda .......................   Passed    0.14 sec
      Start 1476: test_gdasapp_util_icecamsr2ioda
 9/45 Test #1476: test_gdasapp_util_icecamsr2ioda .......................   Passed    0.12 sec
      Start 1811: test_gdasapp_check_python_norms
10/45 Test #1811: test_gdasapp_check_python_norms .......................   Passed    2.54 sec
      Start 1812: test_gdasapp_check_yaml_keys
11/45 Test #1812: test_gdasapp_check_yaml_keys ..........................   Passed    1.42 sec
      Start 1813: test_gdasapp_jedi_increment_to_fv3
12/45 Test #1813: test_gdasapp_jedi_increment_to_fv3 ....................   Passed   11.86 sec
      Start 1814: test_gdasapp_setup_cycled_exp
13/45 Test #1814: test_gdasapp_setup_cycled_exp .........................   Passed    1.75 sec
      Start 1815: test_gdasapp_fv3jedi_fv3inc
14/45 Test #1815: test_gdasapp_fv3jedi_fv3inc ...........................***Failed   27.33 sec
      Start 1816: test_gdasapp_soca_nsst_increment_to_mom6
15/45 Test #1816: test_gdasapp_soca_nsst_increment_to_mom6 ..............   Passed   29.80 sec
      Start 1817: test_gdasapp_soca_prep
16/45 Test #1817: test_gdasapp_soca_prep ................................   Passed    4.45 sec
      Start 1818: test_gdasapp_soca_run_clean
17/45 Test #1818: test_gdasapp_soca_run_clean ...........................   Passed    0.02 sec
      Start 1819: test_gdasapp_soca_setup_obsprep
18/45 Test #1819: test_gdasapp_soca_setup_obsprep .......................   Passed   10.16 sec
      Start 1820: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS
19/45 Test #1820: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS ..............   Passed   42.75 sec
      Start 1821: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
20/45 Test #1821: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP ....   Passed   42.18 sec
      Start 1822: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT
21/45 Test #1822: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT ....   Passed  138.21 sec
      Start 1823: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
22/45 Test #1823: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN .....   Passed   42.20 sec
      Start 1824: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN
23/45 Test #1824: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN ....   Passed   42.18 sec
      Start 1825: test_gdasapp_soca_copy_scratch
24/45 Test #1825: test_gdasapp_soca_copy_scratch ........................   Passed    1.19 sec
      Start 1826: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT
25/45 Test #1826: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT ...   Passed   42.18 sec
      Start 1827: test_gdasapp_soca_socahybridweights
26/45 Test #1827: test_gdasapp_soca_socahybridweights ...................   Passed   10.15 sec
      Start 1828: test_gdasapp_soca_incr_handler
27/45 Test #1828: test_gdasapp_soca_incr_handler ........................   Passed   10.14 sec
      Start 1829: test_gdasapp_soca_ens_handler
28/45 Test #1829: test_gdasapp_soca_ens_handler .........................   Passed   10.16 sec
      Start 1830: test_gdasapp_snow_create_ens
29/45 Test #1830: test_gdasapp_snow_create_ens ..........................   Passed    0.77 sec
      Start 1831: test_gdasapp_snow_imsproc
30/45 Test #1831: test_gdasapp_snow_imsproc .............................   Passed    3.72 sec
      Start 1832: test_gdasapp_snow_apply_jediincr
31/45 Test #1832: test_gdasapp_snow_apply_jediincr ......................***Failed    1.46 sec
      Start 1833: test_gdasapp_snow_letkfoi_snowda
32/45 Test #1833: test_gdasapp_snow_letkfoi_snowda ......................***Failed    1.57 sec
      Start 1834: test_gdasapp_convert_bufr_adpsfc_snow
33/45 Test #1834: test_gdasapp_convert_bufr_adpsfc_snow .................   Passed    3.02 sec
      Start 1838: test_gdasapp_convert_bufr_adpsfc
34/45 Test #1838: test_gdasapp_convert_bufr_adpsfc ......................   Passed    4.27 sec
      Start 1839: test_gdasapp_convert_gsi_satbias
35/45 Test #1839: test_gdasapp_convert_gsi_satbias ......................   Passed    4.72 sec
      Start 1840: test_gdasapp_setup_atm_cycled_exp
36/45 Test #1840: test_gdasapp_setup_atm_cycled_exp .....................   Passed    0.57 sec
      Start 1841: test_gdasapp_atm_jjob_var_init
37/45 Test #1841: test_gdasapp_atm_jjob_var_init ........................   Passed   44.14 sec
      Start 1842: test_gdasapp_atm_jjob_var_run
38/45 Test #1842: test_gdasapp_atm_jjob_var_run .........................   Passed  106.13 sec
      Start 1843: test_gdasapp_atm_jjob_var_inc
39/45 Test #1843: test_gdasapp_atm_jjob_var_inc .........................   Passed   42.13 sec
      Start 1844: test_gdasapp_atm_jjob_var_final
40/45 Test #1844: test_gdasapp_atm_jjob_var_final .......................   Passed   42.28 sec
      Start 1845: test_gdasapp_atm_jjob_ens_init
41/45 Test #1845: test_gdasapp_atm_jjob_ens_init ........................   Passed   43.71 sec
      Start 1846: test_gdasapp_atm_jjob_ens_run
42/45 Test #1846: test_gdasapp_atm_jjob_ens_run .........................   Passed  458.30 sec
      Start 1847: test_gdasapp_atm_jjob_ens_inc
43/45 Test #1847: test_gdasapp_atm_jjob_ens_inc .........................   Passed  106.13 sec
      Start 1848: test_gdasapp_atm_jjob_ens_final
44/45 Test #1848: test_gdasapp_atm_jjob_ens_final .......................   Passed   42.13 sec
      Start 1849: test_gdasapp_aero_gen_3dvar_yaml
45/45 Test #1849: test_gdasapp_aero_gen_3dvar_yaml ......................   Passed    1.50 sec

93% tests passed, 3 tests failed out of 45

Label Time Summary:
gdas-utils    =  14.33 sec*proc (9 tests)
script        =  14.33 sec*proc (9 tests)

Total Test time (real) = 1394.66 sec

The following tests FAILED:
        1815 - test_gdasapp_fv3jedi_fv3inc (Failed)
        1832 - test_gdasapp_snow_apply_jediincr (Failed)
        1833 - test_gdasapp_snow_letkfoi_snowda (Failed)
Errors while running CTest

Interestingly, rerunning the three failed tests after the above failure results in all three tests passing.

(gdasapp) Orion-login-2:/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build$ ctest -R test_gdasapp_fv3jedi_fv3inc
Test project /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build
    Start 1815: test_gdasapp_fv3jedi_fv3inc
1/1 Test #1815: test_gdasapp_fv3jedi_fv3inc ......   Passed    4.23 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   4.75 sec
(gdasapp) Orion-login-2:/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build$ ctest -R test_gdasapp_snow_apply_jediincr
Test project /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build
    Start 1832: test_gdasapp_snow_apply_jediincr
1/1 Test #1832: test_gdasapp_snow_apply_jediincr ...   Passed    2.22 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   2.68 sec
(gdasapp) Orion-login-2:/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build$ ctest -R test_gdasapp_snow_letkfoi_snowda
Test project /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build
    Start 1833: test_gdasapp_snow_letkfoi_snowda
1/1 Test #1833: test_gdasapp_snow_letkfoi_snowda ...   Passed    3.42 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   3.88 sec

This behavior needs to be examined and better understood.

RussTreadon-NOAA added a commit to RussTreadon-NOAA/GDASApp that referenced this issue May 21, 2024
RussTreadon-NOAA added a commit to RussTreadon-NOAA/GDASApp that referenced this issue May 30, 2024
@RussTreadon-NOAA
Copy link
Contributor Author

Hera test
Install RussTreadon-NOAA:feature/submodules at b30d103. Run test_gdasapp. Two tests failed

        1856 - test_gdasapp_atm_jjob_var_run (Failed)
        1860 - test_gdasapp_atm_jjob_ens_run (Failed)

due to the reference check after each application finished. This failure is not unexpected. feature/submodules updates the hash for several JEDI repositories. @DavidNew-NOAA explained how to update the references files. Did so and reran with all tests passing

Test project /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/update_gdasapp/sorc/gdas.cd/build
      Start 1483: test_gdasapp_util_coding_norms
 1/47 Test #1483: test_gdasapp_util_coding_norms ........................   Passed    2.93 sec
      Start 1484: test_gdasapp_util_ioda_example
 2/47 Test #1484: test_gdasapp_util_ioda_example ........................   Passed    0.34 sec
      Start 1485: test_gdasapp_util_prepdata
 3/47 Test #1485: test_gdasapp_util_prepdata ............................   Passed    1.03 sec
      Start 1486: test_gdasapp_util_rads2ioda
 4/47 Test #1486: test_gdasapp_util_rads2ioda ...........................   Passed    0.25 sec
      Start 1487: test_gdasapp_util_ghrsst2ioda
 5/47 Test #1487: test_gdasapp_util_ghrsst2ioda .........................   Passed    0.14 sec
      Start 1488: test_gdasapp_util_smap2ioda
 6/47 Test #1488: test_gdasapp_util_smap2ioda ...........................   Passed    0.16 sec
      Start 1489: test_gdasapp_util_smos2ioda
 7/47 Test #1489: test_gdasapp_util_smos2ioda ...........................   Passed    0.15 sec
      Start 1490: test_gdasapp_util_viirsaod2ioda
 8/47 Test #1490: test_gdasapp_util_viirsaod2ioda .......................   Passed    0.14 sec
      Start 1491: test_gdasapp_util_icecamsr2ioda
 9/47 Test #1491: test_gdasapp_util_icecamsr2ioda .......................   Passed    0.14 sec
      Start 1826: test_gdasapp_check_python_norms
10/47 Test #1826: test_gdasapp_check_python_norms .......................   Passed    3.15 sec
      Start 1827: test_gdasapp_check_yaml_keys
11/47 Test #1827: test_gdasapp_check_yaml_keys ..........................   Passed    0.28 sec
      Start 1828: test_gdasapp_jedi_increment_to_fv3
12/47 Test #1828: test_gdasapp_jedi_increment_to_fv3 ....................   Passed    0.83 sec
      Start 1829: test_gdasapp_setup_cycled_exp
13/47 Test #1829: test_gdasapp_setup_cycled_exp .........................   Passed    0.86 sec
      Start 1830: test_gdasapp_fv3jedi_fv3inc
14/47 Test #1830: test_gdasapp_fv3jedi_fv3inc ...........................   Passed   14.12 sec
      Start 1831: test_gdasapp_soca_nsst_increment_to_mom6
15/47 Test #1831: test_gdasapp_soca_nsst_increment_to_mom6 ..............   Passed    3.43 sec
      Start 1832: test_gdasapp_soca_prep
16/47 Test #1832: test_gdasapp_soca_prep ................................   Passed    3.80 sec
      Start 1833: test_gdasapp_soca_run_clean
17/47 Test #1833: test_gdasapp_soca_run_clean ...........................   Passed    0.58 sec
      Start 1834: test_gdasapp_soca_setup_obsprep
18/47 Test #1834: test_gdasapp_soca_setup_obsprep .......................   Passed   23.57 sec
      Start 1835: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS
19/47 Test #1835: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS ..............   Passed   75.62 sec
      Start 1836: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
20/47 Test #1836: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP ....   Passed   42.47 sec
      Start 1837: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT
21/47 Test #1837: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT ....   Passed   74.53 sec
      Start 1838: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
22/47 Test #1838: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN .....   Passed   42.49 sec
      Start 1839: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN
23/47 Test #1839: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN ....   Passed   74.61 sec
      Start 1840: test_gdasapp_soca_copy_scratch
24/47 Test #1840: test_gdasapp_soca_copy_scratch ........................   Passed    4.46 sec
      Start 1841: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT
25/47 Test #1841: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT ...   Passed   42.63 sec
      Start 1842: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST
26/47 Test #1842: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST ....   Passed   42.51 sec
      Start 1843: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY
27/47 Test #1843: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY ....   Passed  234.77 sec
      Start 1844: test_gdasapp_soca_socahybridweights
28/47 Test #1844: test_gdasapp_soca_socahybridweights ...................   Passed   10.47 sec
      Start 1845: test_gdasapp_soca_incr_handler
29/47 Test #1845: test_gdasapp_soca_incr_handler ........................   Passed   10.38 sec
      Start 1846: test_gdasapp_soca_ens_handler
30/47 Test #1846: test_gdasapp_soca_ens_handler .........................   Passed   10.44 sec
      Start 1847: test_gdasapp_snow_create_ens
31/47 Test #1847: test_gdasapp_snow_create_ens ..........................   Passed    2.39 sec
      Start 1848: test_gdasapp_snow_imsproc
32/47 Test #1848: test_gdasapp_snow_imsproc .............................   Passed    4.94 sec
      Start 1849: test_gdasapp_snow_apply_jediincr
33/47 Test #1849: test_gdasapp_snow_apply_jediincr ......................   Passed    6.74 sec
      Start 1850: test_gdasapp_snow_letkfoi_snowda
34/47 Test #1850: test_gdasapp_snow_letkfoi_snowda ......................   Passed   12.36 sec
      Start 1851: test_gdasapp_convert_bufr_adpsfc_snow
35/47 Test #1851: test_gdasapp_convert_bufr_adpsfc_snow .................   Passed    4.31 sec
      Start 1852: test_gdasapp_convert_bufr_adpsfc
36/47 Test #1852: test_gdasapp_convert_bufr_adpsfc ......................   Passed    4.18 sec
      Start 1853: test_gdasapp_convert_gsi_satbias
37/47 Test #1853: test_gdasapp_convert_gsi_satbias ......................   Passed    5.82 sec
      Start 1854: test_gdasapp_setup_atm_cycled_exp
38/47 Test #1854: test_gdasapp_setup_atm_cycled_exp .....................   Passed    1.35 sec
      Start 1855: test_gdasapp_atm_jjob_var_init
39/47 Test #1855: test_gdasapp_atm_jjob_var_init ........................   Passed   47.79 sec
      Start 1856: test_gdasapp_atm_jjob_var_run
40/47 Test #1856: test_gdasapp_atm_jjob_var_run .........................   Passed  106.44 sec
      Start 1857: test_gdasapp_atm_jjob_var_inc
41/47 Test #1857: test_gdasapp_atm_jjob_var_inc .........................   Passed   74.35 sec
      Start 1858: test_gdasapp_atm_jjob_var_final
42/47 Test #1858: test_gdasapp_atm_jjob_var_final .......................   Passed   42.43 sec
      Start 1859: test_gdasapp_atm_jjob_ens_init
43/47 Test #1859: test_gdasapp_atm_jjob_ens_init ........................   Passed   47.56 sec
      Start 1860: test_gdasapp_atm_jjob_ens_run
44/47 Test #1860: test_gdasapp_atm_jjob_ens_run .........................   Passed  458.34 sec
      Start 1861: test_gdasapp_atm_jjob_ens_inc
45/47 Test #1861: test_gdasapp_atm_jjob_ens_inc .........................   Passed  106.27 sec
      Start 1862: test_gdasapp_atm_jjob_ens_final
46/47 Test #1862: test_gdasapp_atm_jjob_ens_final .......................   Passed   42.25 sec
      Start 1863: test_gdasapp_aero_gen_3dvar_yaml
47/47 Test #1863: test_gdasapp_aero_gen_3dvar_yaml ......................   Passed    0.81 sec

100% tests passed, 0 tests failed out of 47

Label Time Summary:
gdas-utils    =   5.29 sec*proc (9 tests)
script        =   5.29 sec*proc (9 tests)

Total Test time (real) = 1691.89 sec

@RussTreadon-NOAA
Copy link
Contributor Author

REMINDER

The names used for certain radiance bias correction terms change with updated JEDI hashes in feature/submodules. As such, the amsua_n19 bias correction files used in the atm_jjob tests must be updated. The updated bias correction files on Hera are in /scratch1/NCEPDEV/da/Russ.Treadon/CI/GDASApp/data/lowres/gdas.20210323/12/analysis/atmos. Variable GDASAPP_TESTDATA was changed in modulefiles/GDAS/hera.intel.lua prior to running the above ctests.

-setenv("GDASAPP_TESTDATA","/scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/data")
+setenv("GDASAPP_TESTDATA","/scratch1/NCEPDEV/da/Russ.Treadon/CI/GDASApp/data")

The updated amsua_n19 bias correction files will be copied to /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/data when PR #1123 is merged into develop. As such, the above change to hera.intel.lua will not be committed to feature/submodules.

A change to GDASAPP_TESTDATA was committed to modulefiles/GDAS/orion.intel.lua. This change must be reverted before PR #1123 is merged into develop

@CoryMartin-NOAA
Copy link
Contributor

Test data has been copied from rtreadon to cmartin on Orion and Hera

RussTreadon-NOAA added a commit to RussTreadon-NOAA/GDASApp that referenced this issue Jun 5, 2024
RussTreadon-NOAA added a commit to RussTreadon-NOAA/GDASApp that referenced this issue Jun 5, 2024
CoryMartin-NOAA pushed a commit that referenced this issue Jun 5, 2024
This PR updates the hashes for select JEDI repositories in the GDASApp
`sorc` directory. Updating the submodule hashes necessitates
corresponding changes elsewhere in GDASApp. These additional changes
include
- soca updates (see GDASApp issue
[#1093](#1093))
- update names for select radiance bias correction terms
- correcting the path to `satbias2ioda.x` in
`test/atm/test_convert_gsi_satbias.sh`

Resolves #1117

---------

Co-authored-by: Guillaume Vernieres <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants