Skip to content

[rrfs-mpas-jedi] Updates for running rrfs-workflow on WCOSS2 #803

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
May 27, 2025
Merged

[rrfs-mpas-jedi] Updates for running rrfs-workflow on WCOSS2 #803

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
e9f3717
Initial commit for running on WCOSS2
May 21, 2025
758439c
Use mpiexec for submitting fcst job and update NODES/PPN calculation
May 21, 2025
de0fb83
Add MPASSIT workaround and update MPI_RUN_CMD for all jobs on WCOSS2
May 22, 2025
cc2590e
Revert changes to forecast script
May 22, 2025
373074e
Fix linter error
May 22, 2025
2c3ba90
Load cray-pals in launch.sh since rrfs/wcoss2.intel.lua is not loaded…
May 22, 2025
bebda91
Add UPP workaround (loads libjpeg, libfabric) needed to run
May 22, 2025
0f6725d
Unset some module versions to prevent issues with submodules
May 22, 2025
c14f377
Dont set exclusive tag for wcoss2
May 22, 2025
aee919d
Update exp configs with new path to staged data on WCOSS2
May 22, 2025
3e02285
Update unset.ver
May 22, 2025
8d879cc
Remove old comments
May 22, 2025
6ada96d
Add workaround for loading python modules for offline domaim check
May 22, 2025
0b08e2f
Remove temporary workarounds, load UPP modules in launch.sh
May 23, 2025
6a21f9d
Fix linter error
May 23, 2025
f32978a
Fix more linter errors
May 23, 2025
afb9620
One more linter fix
May 23, 2025
d61ccee
Move Makefile workaround under WCOSS2 if-block
May 23, 2025
e339018
Clean up launching UPP task
May 23, 2025
3adcde6
Add context for MPAS-Model workaround on WCOSS2
May 23, 2025
657717a
Switch order of sourcing config files and set exclusive tag by machin…
May 24, 2025
cdeb8ed
Merge branch 'rrfs-mpas-jedi' into feature/wcoss2_run
MatthewPyle-NOAA May 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Team Charter (draft): https://docs.google.com/document/d/1uLbPx-pOWp7eECz_7VHRt_
- NOAA RDHPCS Jet
- NOAA RDHPCS Gaea C6
- MSU-HPC systems (Hercules/Orion)
- WCOSS2

## [How to build and run rrfs experiments?](doc/build_and_run.md)
## [How to deploy a realtime run in Jet?](https://github.com/NOAA-EMC/rrfs-workflow/wiki/deploy-a-realtime-run-in-Jet)
Expand Down
26 changes: 26 additions & 0 deletions modulefiles/prod_util/wcoss2.lua
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like these definitions of things like NDATE and FSYNC shouldn't be needed for WCOSS2. If the prod_util module is loaded, they should be available. Or is something different with the spack libraries?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just kept the same method for loading the prod_util commands that we used for the other machines. This modulefile manually defines these variables. But it does look like we could just load prod_util through spack-stack (or as a default module such as available on WCOSS2) and these commands and variables would be available without needing this extra modulefile/prod_util directory.

@guoqing-noaa Do you know why we went with this method that manually defines the NDATE etc. variables instead of just loading prod_util through spack-stack?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MatthewPyle-NOAA and @SamuelDegelia-NOAA
As @SamuelDegelia-NOAA mentioned, the wcoss2.lua is essentially a copy of the original prod_util lua file of whatever available on each platform.
I did not recall all the details. But I think the reason to load this separately is that we only want to load modules as needed. The workflow only uses err_exit, err_chk, NDATE, cpreq from the prod_util, so we don't need any module dependencies in the original prod_util. Also, I think we will load a few extra modules if we load prod_util directly from the spack-stack.
We may revisit this solution in the future. Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanations, @SamuelDegelia-NOAA and @guoqing-noaa

Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
-- -*- lua -*-
-- Module file created by spack (https://github.com/spack/spack) on 2025-01-27 23:01:25.166951
--
-- [email protected]%[email protected]~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-haswell/4vpcrpl
--

whatis([[Name : prod-util]])
whatis([[Version : 2.1.1]])
whatis([[Target : haswell]])
whatis([[Short description : Product utilities for the NCEP models.]])

help([[Name : prod-util]])
help([[Version: 2.1.1]])
help([[Target : haswell]])
help()
help([[ Product utilities for the NCEP models. This is part of NOAA's NCEPLIBS
project.]])


prepend_path("PATH", "/apps/ops/prod/nco/core/prod_util.v2.0.14/ush", ":")
prepend_path("CMAKE_PREFIX_PATH", "/apps/ops/prod/nco/core/prod_util.v2.0.14/.", ":")
setenv("prod_util_ROOT", "/apps/ops/prod/nco/core/prod_util.v2.0.14")
setenv("MDATE", "/apps/ops/prod/nco/core/prod_util.v2.0.14/exec/mdate")
setenv("NDATE", "/apps/ops/prod/nco/core/prod_util.v2.0.14/exec/ndate")
setenv("NHOUR", "/apps/ops/prod/nco/core/prod_util.v2.0.14/exec/nhour")
setenv("FSYNC", "/apps/ops/prod/nco/core/prod_util.v2.0.14/exec/fsync_file")
1 change: 1 addition & 0 deletions modulefiles/rrfs/wcoss2.intel.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ load(pathJoin("PrgEnv-intel", os.getenv("PrgEnv_intel_ver")))
load(pathJoin("stack-intel", os.getenv("intel_ver")))
load(pathJoin("craype", os.getenv("craype_ver")))
load(pathJoin("stack-cray-mpich", os.getenv("cray_mpich_ver")))
load(pathJoin("stack-python", os.getenv("python_ver") or "3.10.13"))
load(pathJoin("cmake", os.getenv("cmake_ver")))
prepend_path("MODULEPATH", os.getenv("modulepath_compiler"))
prepend_path("MODULEPATH", os.getenv("modulepath_mpi"))
Expand Down
11 changes: 11 additions & 0 deletions parm/config/config.wcoss2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
# machine-specific settings
#
topdir=topdir_not_defined
export COMINgfs=${COMINgfs:-"${topdir}/GFS"}
export COMINrap=${COMINrap:-"${topdir}/RAP"}
export COMINgefs=${COMINgefs:-"${topdir}/GEFS"}
export COMINrrfs=${COMINrrfs:-"/COMINrrfs_not_defined"}
export COMINhrrr=${COMINhrrr:-"/COMINhrrr_not_defined"}
#
export OBSINprepbufr=${OBSINprepbufr:-"${topdir}/obs_rap"}
2 changes: 1 addition & 1 deletion sorc/MPASSIT
Open in desktop
Submodule MPASSIT updated 1 files
+3 −1 build.sh
Loading