Skip to content

Commit

Permalink
Add optional testing block to 3dvar and lgetkf input YAMLs for jjobs (#…
Browse files Browse the repository at this point in the history 
…1129)

This PR turns on the testing blocks, added in jcb-algorithms
#[1](NOAA-EMC/jcb-algorithms#1), in the input
YAMLs for the 3dvar and lgetkf jjob test, turning them into JEDI
application tests that check for changes in the output states. This PR
updates the jcb-algorithms hash to the latest develop as well.

The motivation for this PR is that we would like to know not only if
changes to GDASApp cause run failures, but also whether they change
outputs.

I used the same relative and absolute tolerance used in
https://github.com/JCSDA-internal/fv3-jedi/blob/develop/test/testinput/lgetkf.yaml
and
https://github.com/JCSDA-internal/fv3-jedi/blob/develop/test/testinput/hyb-3dvar_gfs.yaml
from FV3-JEDI. I don't know why one uses 1e-7 and the other 1e-6 for the
absolute tolerances. Reviewers can tell me what they think is
reasonable.

I ran all the jjob tests and they passed on Orion with the
feature/atmensanlfv3inc branch of Global Workflow. That branch needs to
be merged into develop with
#[2592](NOAA-EMC/global-workflow#2592).

Note: The JCB submodule in Global Workflow will need to updated to the
latest develop branch due to change in this commit of jcb-algorithms
NOAA-EMC/jcb-algorithms@1a1497c
  • Loading branch information
@DavidNew-NOAA
DavidNew-NOAA authored May 28, 2024
1 parent 95218e7 commit 41cecc9
Show file tree
Hide file tree
Showing 5 changed files with 306 additions and 1 deletion.
2 changes: 1 addition & 1 deletion parm/jcb-algorithms
Submodule jcb-algorithms updated 4 files
+18 −0 3dvar.yaml.j2
+18 −0 local_ensemble_da.yaml.j2
+3 −0 observer_components.yaml
+0 −1 test_obs_filters.yaml.j2
94 changes: 94 additions & 0 deletions test/atm/global-workflow/3dvar.ref
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
CostJb : Nonlinear Jb = 0.0000000000000000e+00
CostJo : Nonlinear Jo(AMSUA N19) = 3.8984270736441467e+04, nobs = 74275, Jo/n = 5.2486396144653613e-01, err = 6.3650648936327991e+00
CostJo : Nonlinear Jo(sondes) = 1.0738490911289638e+04, nobs = 4255, Jo/n = 2.5237346442513839e+00, err = 1.0981085630539836e+01
CostFunction: Nonlinear J = 4.9722761647731109e+04
DRPCGMinimizer: reduction in residual norm = 9.6484392730339263e-01
CostFunction::addIncrement: Analysis:
----------------------------------------------------------------------------------------------------
State print | number of fields = 22 | cube sphere face size: C48
eastward_wind | Min:-5.5495644220059390e+01 Max:+8.4934651204487366e+01 RMS:+1.6388250288534099e+01
northward_wind | Min:-7.3712421229093422e+01 Max:+7.6984825502186652e+01 RMS:+8.1424144891570887e+00
air_temperature | Min:+1.7482158077318462e+02 Max:+3.1467235233685437e+02 RMS:+2.4978424883089124e+02
air_pressure_thickness | Min:+6.0599999874109178e-01 Max:+1.7760098554198755e+03 RMS:+9.5680601590204810e+02
surface_pressure | Min:+5.3297486514404744e+04 Max:+1.0397927292902800e+05 RMS:+9.8896232762674656e+04
specific_humidity | Min:+0.0000000000000000e+00 Max:+2.0041369334954085e-02 RMS:+4.7792636981582193e-03
cloud_liquid_ice | Min:+0.0000000000000000e+00 Max:+1.2024392024662985e-03 RMS:+1.4396968866569995e-05
cloud_liquid_water | Min:+0.0000000000000000e+00 Max:+1.5935686618789048e-03 RMS:+4.1931345590543333e-05
ozone_mass_mixing_ratio | Min:+1.1998327271379204e-08 Max:+1.7808431790670831e-05 RMS:+4.4947917149861064e-06
sfc_geopotential_height_times_grav | Min:-2.6495993813133236e+02 Max:+5.1678555530273436e+04 RMS:+6.1813840931281156e+03
slmsk | Min:+0.0000000000000000e+00 Max:+2.0000000000000000e+00 RMS:+7.3450427349184144e-01
sheleg | Min:+0.0000000000000000e+00 Max:+1.8650088674622128e+02 RMS:+2.2457791153397761e+01
sea_surface_temperature | Min:+2.1068090851678403e+02 Max:+3.3125178729099270e+02 RMS:+2.8949442340676961e+02
vtype | Min:+0.0000000000000000e+00 Max:+2.0000000000000000e+01 RMS:+6.5214489858466269e+00
stype | Min:+0.0000000000000000e+00 Max:+1.6000000000000000e+01 RMS:+4.5380018343041852e+00
vfrac | Min:+0.0000000000000000e+00 Max:+9.7215068788150627e-01 RMS:+2.2249321391025326e-01
stc | Min:+2.1550390656839755e+02 Max:+3.2609885436959308e+02 RMS:+2.8945169639231835e+02
soilMoistureVolumetric | Min:+1.1321643448483208e-02 Max:+1.0000000000000000e+00 RMS:+8.3306585674111977e-01
totalSnowDepth | Min:+0.0000000000000000e+00 Max:+1.7098998490425442e+03 RMS:+1.8885640927654018e+02
surface_eastward_wind | Min:-1.6966302087131233e+01 Max:+2.1273647117120184e+01 RMS:+5.4892133439475188e+00
surface_northward_wind | Min:-1.7367330522224339e+01 Max:+2.2091894088184446e+01 RMS:+4.4723535847538756e+00
f10m | Min:+9.4613586695410434e-01 Max:+1.0695628272426509e+00 RMS:+9.9526981313535046e-01
----------------------------------------------------------------------------------------------------

Obs bias coefficients:
---------------------------------------------------------------
constant: Min= -1.2490630149841309, Max= 1.6242779492736057, Norm= 3.1216962856027304
lapse_rate_order_2: Min= -6.0771207806339094, Max= 52.3767318715110477, Norm= 52.7793275314599910
lapse_rate: Min= -6.1173920629465659, Max= 0.4750269950800435, Norm= 6.1658157312728985
emissivity: Min= -0.2395000010728836, Max= 10.5581263135712575, Norm= 10.5655898122368264
scan_angle_order_4: Min= -7.9506011008096911, Max= 2.3034050466999081, Norm= 13.1727465012547444
scan_angle_order_3: Min= -1.1903680560584049, Max= 2.3802189822803275, Norm= 3.1970135720803450
scan_angle_order_2: Min= -5.1212630271747548, Max= 2.9977130889532191, Norm= 7.3569496454063597
scan_angle: Min= -0.6766870022405442, Max= 0.0732169970537269, Norm= 1.1485053152110289
---------------------------------------------------------------


CostJb : Nonlinear Jb = 0.0000003263133461
CostJo : Nonlinear Jo(AMSUA N19) = 38981.7238460305161425, nobs = 74275, Jo/n = 0.5248296714376374, err = 6.3650648936327991
CostJo : Nonlinear Jo(sondes) = 10687.2756506645455374, nobs = 4255, Jo/n = 2.5116981552678133, err = 10.9810856305398357
CostFunction: Nonlinear J = 49668.9994970213738270
DRPCGMinimizer: reduction in residual norm = 1.0147575953754833
CostFunction::addIncrement: Analysis:
----------------------------------------------------------------------------------------------------
State print | number of fields = 22 | cube sphere face size: C48
eastward_wind | Min:-5.5495644220059390e+01 Max:+8.4934651204487366e+01 RMS:+1.6388250288542711e+01
northward_wind | Min:-7.3712421229093422e+01 Max:+7.6984825502186652e+01 RMS:+8.1424144891586359e+00
air_temperature | Min:+1.7482158077318311e+02 Max:+3.1467235233685437e+02 RMS:+2.4978424883200961e+02
air_pressure_thickness | Min:+6.0599999874109178e-01 Max:+1.7760098554198755e+03 RMS:+9.5680601590204810e+02
surface_pressure | Min:+5.3297486514404744e+04 Max:+1.0397927292902800e+05 RMS:+9.8896232762674656e+04
specific_humidity | Min:+0.0000000000000000e+00 Max:+2.0041369334954085e-02 RMS:+4.7792638362681448e-03
cloud_liquid_ice | Min:+0.0000000000000000e+00 Max:+1.2024392024662985e-03 RMS:+1.4396968866569995e-05
cloud_liquid_water | Min:+0.0000000000000000e+00 Max:+1.5935686618789048e-03 RMS:+4.1931345590543333e-05
ozone_mass_mixing_ratio | Min:+1.1998327271379204e-08 Max:+1.7808431790670831e-05 RMS:+4.4947917149861064e-06
sfc_geopotential_height_times_grav | Min:-2.6495993813133236e+02 Max:+5.1678555530273436e+04 RMS:+6.1813840931281156e+03
slmsk | Min:+0.0000000000000000e+00 Max:+2.0000000000000000e+00 RMS:+7.3450427349184144e-01
sheleg | Min:+0.0000000000000000e+00 Max:+1.8650088674622128e+02 RMS:+2.2457791153397761e+01
sea_surface_temperature | Min:+2.1068090851678403e+02 Max:+3.3125178729099270e+02 RMS:+2.8949442340676961e+02
vtype | Min:+0.0000000000000000e+00 Max:+2.0000000000000000e+01 RMS:+6.5214489858466269e+00
stype | Min:+0.0000000000000000e+00 Max:+1.6000000000000000e+01 RMS:+4.5380018343041852e+00
vfrac | Min:+0.0000000000000000e+00 Max:+9.7215068788150627e-01 RMS:+2.2249321391025326e-01
stc | Min:+2.1550390656839755e+02 Max:+3.2609885436959308e+02 RMS:+2.8945169639231835e+02
soilMoistureVolumetric | Min:+1.1321643448483208e-02 Max:+1.0000000000000000e+00 RMS:+8.3306585674111977e-01
totalSnowDepth | Min:+0.0000000000000000e+00 Max:+1.7098998490425442e+03 RMS:+1.8885640927654018e+02
surface_eastward_wind | Min:-1.6966302087131233e+01 Max:+2.1273647117120184e+01 RMS:+5.4892133439475188e+00
surface_northward_wind | Min:-1.7367330522224339e+01 Max:+2.2091894088184446e+01 RMS:+4.4723535847538756e+00
f10m | Min:+9.4613586695410434e-01 Max:+1.0695628272426509e+00 RMS:+9.9526981313535046e-01
----------------------------------------------------------------------------------------------------

Obs bias coefficients:
---------------------------------------------------------------
constant: Min= -1.2490630149841309, Max= 1.6242779490057870, Norm= 3.1216962826704004
lapse_rate_order_2: Min= -6.0771207792168234, Max= 52.3767318668067006, Norm= 52.7793275266354911
lapse_rate: Min= -6.1173920620497713, Max= 0.4750269957036114, Norm= 6.1658157301802641
emissivity: Min= -0.2395000010728836, Max= 10.5581256964462273, Norm= 10.5655891955375107
scan_angle_order_4: Min= -7.9506011003013111, Max= 2.3034050477855654, Norm= 13.1727464989080296
scan_angle_order_3: Min= -1.1903680549739593, Max= 2.3802189803928178, Norm= 3.1970135704212637
scan_angle_order_2: Min= -5.1212630270944297, Max= 2.9977130887912211, Norm= 7.3569496444930564
scan_angle: Min= -0.6766870025059790, Max= 0.0732169969027750, Norm= 1.1485053159201970
---------------------------------------------------------------


CostJb : Nonlinear Jb = 0.0000033688688972
CostJo : Nonlinear Jo(AMSUA N19) = 38970.3026556682743831, nobs = 74275, Jo/n = 0.5246759024660824, err = 6.3650648936327991
CostJo : Nonlinear Jo(sondes) = 10639.1209516101644112, nobs = 4255, Jo/n = 2.5003809521998037, err = 10.9810856305398357
CostFunction: Nonlinear J = 49609.4236106473108521
7 changes: 7 additions & 0 deletions test/atm/global-workflow/jcb-prototype_3dvar.yaml.j2
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,10 @@ observations:

# The observation files in the testing are appended using the yyymmddhh similar to JEDI tests
atm_obsdatain_suffix: ".{{ current_cycle | to_YMDH }}.nc"

# Testing things
# --------------
test_reference_filename: {{ HOMEgfs }}/sorc/gdas.cd/test/atm/global-workflow/3dvar.ref
test_output_filename: ./3dvar.out
float_relative_tolerance: 1.0e-3
float_absolute_tolerance: 1.0e-6
7 changes: 7 additions & 0 deletions test/atm/global-workflow/jcb-prototype_lgetkf.yaml.j2
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,10 @@ observations:

# The observation files in the testing are appended using the yyymmddhh similar to JEDI tests
atm_obsdatain_suffix: ".{{ current_cycle | to_YMDH }}.nc"

# Testing things
# --------------
test_reference_filename: {{ HOMEgfs }}/sorc/gdas.cd/test/atm/global-workflow/lgetkf.ref
test_output_filename: ./lgetkf.out
float_relative_tolerance: 1.0e-3
float_absolute_tolerance: 1.0e-7
Loading

0 comments on commit 41cecc9

Please sign in to comment.