Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an OOPS-based application to recenter snow analysis #1102

Merged
merged 15 commits into from
May 16, 2024

Conversation

CoryMartin-NOAA
Copy link
Contributor

Closes #1097

This adds a new GDASApp utility that will:

  • read deterministic guess
  • read ensemble guess, compute ensemble mean
  • read deterministic increment
  • subtract ens mean from guess
  • add this difference to the increment
  • write out a new increment
    Effectively recentering the ensemble to the deterministic

This is just the recentering utility, the workflow will need to be modified to use fregrid to interpolate, and then the land-apply-inc code will need to be exercised on every member

@CoryMartin-NOAA CoryMartin-NOAA marked this pull request as ready for review May 9, 2024 17:32
@CoryMartin-NOAA
Copy link
Contributor Author

This is what a YAML would look like for this app:

geometry:
  fms initialization:
    namelist filename: Data/fv3files/fmsmpp.nml
    field table filename: Data/fv3files/field_table_gfdl
  akbk: Data/fv3files/akbk127.nc4
  npx: 13
  npy: 13
  npz: 127
  field metadata override: Data/fieldmetadata/gfs-restart.yaml
date: "2020-12-15T00:00:00Z"
variables: [T]
deterministic increment:
  datetime: 2020-12-15T00:00:00Z
  filetype: fms restart
  datapath: ./inc/
  filename_core: 20201215.000000.fv_core.res.nc
  filename_trcr: 20201215.000000.fv_tracer.res.nc
  filename_sfcd: 20201215.000000.sfc_data.nc
  filename_sfcw: 20201215.000000.fv_srf_wnd.res.nc
  filename_cplr: 20201215.000000.coupler.res
deterministic background:
  datetime: 2020-12-15T00:00:00Z
  filetype: fms restart
  state variables: [u,v,ua,va,T,DELP,sphum,ice_wat,liq_wat,o3mr,phis,
                    slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph,
                    u_srf,v_srf,f10m]
  datapath: ./bkg/bkg/
  filename_core: 20201215.000000.fv_core.res.nc
  filename_trcr: 20201215.000000.fv_tracer.res.nc
  filename_sfcd: 20201215.000000.sfc_data.nc
  filename_sfcw: 20201215.000000.fv_srf_wnd.res.nc
  filename_cplr: 20201215.000000.coupler.res
output increment:
  datapath: ./
  datetime: 2020-12-15T00:00:00Z
  filetype: fms restart
  filename_core: 20201215.000000.fv_core.res.nc
  filename_trcr: 20201215.000000.fv_tracer.res.nc
  filename_sfcd: 20201215.000000.sfc_data.nc
  filename_sfcw: 20201215.000000.fv_srf_wnd.res.nc
  filename_cplr: 20201215.000000.coupler.res
ensemble backgrounds:
  number of members: 3
  pattern: '%mem%'
  zero padding: 3
  template:
    datetime: 2020-12-15T00:00:00Z
    filetype: fms restart
    state variables: [u,v,ua,va,T,DELP,sphum,ice_wat,liq_wat,o3mr,phis,
                      slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph,
                      u_srf,v_srf,f10m]
    datapath: ./bkg/mem%mem%/
    filename_core: 20201215.000000.fv_core.res.nc
    filename_trcr: 20201215.000000.fv_tracer.res.nc
    filename_sfcd: 20201215.000000.sfc_data.nc
    filename_sfcw: 20201215.000000.fv_srf_wnd.res.nc
    filename_cplr: 20201215.000000.coupler.res

Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

Copy link
Collaborator

@ClaraDraper-NOAA ClaraDraper-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few thoughts, that may or may not be relevant:

  1. I don't see any land masks. In GFS, the land variables are defined (and usually zero) over non-land points. Is there any risk that this code could cause problems if we're not careful with the input files over non-land points?
  2. If we're recentering the snow depth, we should really re-center the other variables too. Is there other code in place to do this?
    Also, see comment about making the routine names generic.

utils/snow/snow_ensrecenter.h Outdated Show resolved Hide resolved
utils/snow/snow_ensrecenter.h Outdated Show resolved Hide resolved
@CoryMartin-NOAA
Copy link
Contributor Author

Thanks @ClaraDraper-NOAA .

For 1, I didn't consider a mask since this will write out an increment and presumably the apply increment code will handle the masking, but I can look into how to add a mask here and see how difficult it would be.

For 2, what other variables do you have in mind? The atmospheric fields will be recentered as part of the GSI Var/EnKF machinery. For land fields, we can add extra variables in the YAML with this executable. The original increment will be 0, but the recentering will give the ensemble an increment to be consistent with the deterministic.

@CoryMartin-NOAA
Copy link
Contributor Author

@ClaraDraper-NOAA A quick look seems like through the magic of JEDI I can pretty easily read in time invariant fields and then I think I should be able to add to this utility something like:

mask field: slmsk
minvalue: 0.9
maxvalue: 1.1

or something like that to make it generic to zero out the increments where the mask value is between min and max

@jiaruidong2017
Copy link
Collaborator

For the increment file, we just have the surface file available. It is better to just keep the sfc_data files for the land recentering applications in the yaml file with bold font below.

datapath: ./inc/
filename_core: 20201215.000000.fv_core.res.nc
filename_trcr: 20201215.000000.fv_tracer.res.nc
> filename_sfcd: 20201215.000000.sfc_data.nc
filename_sfcw: 20201215.000000.fv_srf_wnd.res.nc
> filename_cplr: 20201215.000000.coupler.res

And the state variables in current JEDI based snow analysis are [snodl,vtype,slmsk] for the Noah-MP LSM.

state variables: [u,v,ua,va,T,DELP,sphum,ice_wat,liq_wat,o3mr,phis,
slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph,
u_srf,v_srf,f10m]

@ClaraDraper-NOAA
Copy link
Collaborator

@ClaraDraper-NOAA A quick look seems like through the magic of JEDI I can pretty easily read in time invariant fields and then I think I should be able to add to this utility something like:

mask field: slmsk
minvalue: 0.9
maxvalue: 1.1

or something like that to make it generic to zero out the increments where the mask value is between min and max

If the masking can be done easily (1/2 a day), I think's it worth it, just to keep things neat. However, you're right that since you're writing out increments here, the code that adds the increments will take care of the masking.

@ClaraDraper-NOAA
Copy link
Collaborator

Thanks @ClaraDraper-NOAA .

For 1, I didn't consider a mask since this will write out an increment and presumably the apply increment code will handle the masking, but I can look into how to add a mask here and see how difficult it would be.

For 2, what other variables do you have in mind? The atmospheric fields will be recentered as part of the GSI Var/EnKF machinery. For land fields, we can add extra variables in the YAML with this executable. The original increment will be 0, but the recentering will give the ensemble an increment to be consistent with the deterministic.

I answered 1 below. For 2, when we meet tomorrow let's talk quickly about how this code would interact with the soil analysis code - which also requires recentering (which I haven't done). Long term, I think it'd be really useful if this code had the flexibility to recenter vars W and X without adding an increment (or adding a zero increment), while also recentering Y and Z with increments.

@CoryMartin-NOAA
Copy link
Contributor Author

I think I've got the masking working:
Screenshot 2024-05-13 at 3 37 13 PM
Will push the changes now/soon.

Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to approve a pr that ignores the ocean! But ...

@CoryMartin-NOAA
Copy link
Contributor Author

@ClaraDraper-NOAA see if my latest additions match what you were thinking based on our discussion yesterday. I still want to do some quick spot checks to confirm its working as intended but otherwise I think this is ready for review again.

Copy link
Collaborator

@ClaraDraper-NOAA ClaraDraper-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks Cory.

@CoryMartin-NOAA CoryMartin-NOAA merged commit c87f536 into develop May 16, 2024
5 checks passed
@CoryMartin-NOAA CoryMartin-NOAA deleted the feature/snow-recenter branch May 16, 2024 21:01
ypwang19 pushed a commit that referenced this pull request May 21, 2024
…e/superob_avg

* origin/feature/superob_avg:
  Added YAML, JSON, python files for assimilating LEOGEO satwinds (#1099)
  Add an OOPS-based application to recenter snow analysis (#1102)
  Fix ice water FV3 increment variable name and add a few more (#1112)
danholdaway added a commit that referenced this pull request Jun 4, 2024
* origin/develop:
  Update wcoss2.intel.lua (#1142)
  Add yaml file for prepobsaero task (#1141)
  Diffusion parameter yaml template (#1140)
  Adjust absolute float tolerance for variational and ensemble DA jjob tests (#1136)
  Insitu temp and salt (#1135)
  Adds letkf.yaml(.j2)  (#1134)
  Add optional testing block to 3dvar and lgetkf input YAMLs for jjobs (#1129)
  updates to build and run some ctests on WCOSS2 (#1122)
  Addin a GFSv17 ctest (#1130)
  Feature/rtofs in situ (#1110)
  update jcb-gdas hash to bring in satwnd yaml changes (#1128)
  Optionally run specified rocoto task as part of the ctests  (#1121)
  Add build capability to Gaea-C5 (#1101)
  Python Converter and Json for the ADPUPA BUFR DUMP (#1115)
  Revert "Fix ice water FV3 increment variable name and add a few more … (#1125)
  Satwnd and variable name-convention updates for jcb implementation in GDASApp (#1119)
  Add inverse distance weighting option to superob function (#1116)
  Resume the Marine Vrfy Task on ctests (#1107)
  Replicate the creation of the gw-ci cycling test (#1114)
  Added YAML, JSON, python files for assimilating LEOGEO satwinds (#1099)
  Add an OOPS-based application to recenter snow analysis (#1102)
  Fix ice water FV3 increment variable name and add a few more (#1112)
  Update hera.intel.lua (#1109)
danholdaway added a commit that referenced this pull request Jun 4, 2024
* develop:
  Update wcoss2.intel.lua (#1142)
  Add yaml file for prepobsaero task (#1141)
  Diffusion parameter yaml template (#1140)
  Adjust absolute float tolerance for variational and ensemble DA jjob tests (#1136)
  Insitu temp and salt (#1135)
  Adds letkf.yaml(.j2)  (#1134)
  Add optional testing block to 3dvar and lgetkf input YAMLs for jjobs (#1129)
  updates to build and run some ctests on WCOSS2 (#1122)
  Addin a GFSv17 ctest (#1130)
  Feature/rtofs in situ (#1110)
  update jcb-gdas hash to bring in satwnd yaml changes (#1128)
  Optionally run specified rocoto task as part of the ctests  (#1121)
  Add build capability to Gaea-C5 (#1101)
  Python Converter and Json for the ADPUPA BUFR DUMP (#1115)
  Revert "Fix ice water FV3 increment variable name and add a few more … (#1125)
  Satwnd and variable name-convention updates for jcb implementation in GDASApp (#1119)
  Add inverse distance weighting option to superob function (#1116)
  Resume the Marine Vrfy Task on ctests (#1107)
  Replicate the creation of the gw-ci cycling test (#1114)
  Added YAML, JSON, python files for assimilating LEOGEO satwinds (#1099)
  Add an OOPS-based application to recenter snow analysis (#1102)
  Fix ice water FV3 increment variable name and add a few more (#1112)
  Update hera.intel.lua (#1109)
danholdaway added a commit that referenced this pull request Jun 4, 2024
 into feature/rename_atm

* 'feature/rename_atm' of https://github.com/NOAA-EMC/GDASApp:
  Update wcoss2.intel.lua (#1142)
  Add yaml file for prepobsaero task (#1141)
  Diffusion parameter yaml template (#1140)
  Adjust absolute float tolerance for variational and ensemble DA jjob tests (#1136)
  Insitu temp and salt (#1135)
  Adds letkf.yaml(.j2)  (#1134)
  Add optional testing block to 3dvar and lgetkf input YAMLs for jjobs (#1129)
  updates to build and run some ctests on WCOSS2 (#1122)
  Addin a GFSv17 ctest (#1130)
  Feature/rtofs in situ (#1110)
  update jcb-gdas hash to bring in satwnd yaml changes (#1128)
  Optionally run specified rocoto task as part of the ctests  (#1121)
  Add build capability to Gaea-C5 (#1101)
  Python Converter and Json for the ADPUPA BUFR DUMP (#1115)
  Revert "Fix ice water FV3 increment variable name and add a few more … (#1125)
  Satwnd and variable name-convention updates for jcb implementation in GDASApp (#1119)
  Add inverse distance weighting option to superob function (#1116)
  Resume the Marine Vrfy Task on ctests (#1107)
  Replicate the creation of the gw-ci cycling test (#1114)
  Added YAML, JSON, python files for assimilating LEOGEO satwinds (#1099)
  Add an OOPS-based application to recenter snow analysis (#1102)
  Fix ice water FV3 increment variable name and add a few more (#1112)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add utility to recenter snow ensemble increments
4 participants