[rrfs-mpas-jedi] Updates for running rrfs-workflow on WCOSS2 #803
Conversation
SamuelDegelia-NOAA
requested review from
MatthewPyle-NOAA,
ShunLiu-NOAA,
JacobCarley-NOAA and
BenjaminBlake-NOAA
as code owners
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
README.md can be updated to include WCOSS2 now, :) |
|
@SamuelDegelia-NOAA I agree with you that But that can be done in a separate PR. Thanks! |
…e, and update README
Done! |
There was a problem hiding this comment.
I feel like these definitions of things like NDATE and FSYNC shouldn't be needed for WCOSS2. If the prod_util module is loaded, they should be available. Or is something different with the spack libraries?
There was a problem hiding this comment.
I just kept the same method for loading the prod_util commands that we used for the other machines. This modulefile manually defines these variables. But it does look like we could just load prod_util through spack-stack (or as a default module such as available on WCOSS2) and these commands and variables would be available without needing this extra modulefile/prod_util directory.
@guoqing-noaa Do you know why we went with this method that manually defines the NDATE etc. variables instead of just loading prod_util through spack-stack?
There was a problem hiding this comment.
@MatthewPyle-NOAA and @SamuelDegelia-NOAA
As @SamuelDegelia-NOAA mentioned, the wcoss2.lua is essentially a copy of the original prod_util lua file of whatever available on each platform.
I did not recall all the details. But I think the reason to load this separately is that we only want to load modules as needed. The workflow only uses err_exit, err_chk, NDATE, cpreq from the prod_util, so we don't need any module dependencies in the original prod_util. Also, I think we will load a few extra modules if we load prod_util directly from the spack-stack.
We may revisit this solution in the future. Thanks!
There was a problem hiding this comment.
Thanks for the explanations, @SamuelDegelia-NOAA and @guoqing-noaa
DESCRIPTION OF CHANGES:
This PR adds config files and various other updates to allow running rrfs-workflow (version 2) on WCOSS2. Results from the 2024052700 retro are compared against results from Hera in #773. The workflow appears to be working as expected on WCOSS2.
A few notes:
cc,CC, andftn). These compilers are needed to correctly handle MPI on WCOSS2. IFor MPASSIT, this is handled by a hash update. For MPAS-Model, since we only plan to update the model at certain times, I instead replaceMakefileusing the_workaround_method.versions/build.verwhich overwrote the module versions loaded for UPP. To solve this, I addedversions/unset.verwhich clears these vars before loadingsorc/UPP/modulefiles/wcoss2.lua.NTASKS,PPN) and thus is defined inworkflow/sideload/launch.shinstead of in theexp.setupfile. This means we can probably removeMPI_RUN_CMDfrom theexp.setupfiles in a future update.ioda_bufr. So instead we source a python virtual environment used for RRFSv1.TESTS CONDUCTED:
Ran 24-h of cycling with the default
exp/exp.conus12kmconfiguration file. Results are shown in issue #773.Machines/Platforms:
ISSUE:
Resolves #773