Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop]: Update ConfigWorkflow chapter to HEAD of develop #915

Merged
merged 183 commits into from
Oct 5, 2023
Merged
Show file tree
Hide file tree
Changes from 167 commits
Commits
Show all changes
183 commits
Select commit Hold shift + click to select a range
ff80851
update FAQ
gspetro-NOAA Jul 5, 2023
82b2e52
split up/revise Intro; reorg chapters
gspetro-NOAA Jul 5, 2023
b905241
update components section; also monor updates to intro, tech ov, glos…
gspetro-NOAA Jul 5, 2023
fd9efc8
restructure SRW docs
gspetro-NOAA Jul 6, 2023
438814f
finish restructuring, rm images
gspetro-NOAA Jul 6, 2023
a74f4d4
minor updates
gspetro-NOAA Jul 6, 2023
e778e1b
edit tables, add component info
gspetro-NOAA Jul 6, 2023
c2a772a
add info on components
gspetro-NOAA Jul 7, 2023
48e6038
rename Tech Details section
gspetro-NOAA Jul 7, 2023
57c8b08
edit build chapter
gspetro-NOAA Jul 7, 2023
d8e3c30
add cpld_gridgen def
gspetro-NOAA Jul 10, 2023
de60461
rename/reorganize/minor fixes
gspetro-NOAA Jul 10, 2023
1cfba00
minor updates
gspetro-NOAA Jul 10, 2023
593c4c6
minor updates
gspetro-NOAA Jul 10, 2023
e4ba1bc
change regional_workflow to workflow_tools
gspetro-NOAA Jul 10, 2023
e5df5c0
change regional workflow to workflow or SRW App workflow
gspetro-NOAA Jul 10, 2023
92dd46d
update info on loading wflow env for L1 v L2-4 systems
gspetro-NOAA Jul 10, 2023
0a1e7a2
move config_defaults table out of Run chapter
gspetro-NOAA Jul 10, 2023
36b2901
update run ch & tables
gspetro-NOAA Jul 12, 2023
fdf7ed2
config section updates/reorg
gspetro-NOAA Jul 12, 2023
ffc7f86
minor run updates
gspetro-NOAA Jul 12, 2023
0729d1b
plotting & VX updates
gspetro-NOAA Jul 12, 2023
9237772
minor updates
gspetro-NOAA Jul 13, 2023
1860238
add info about NCO-compliant structure
gspetro-NOAA Jul 13, 2023
b88cf59
update wflow & VX task tables
gspetro-NOAA Jul 13, 2023
db20aa9
update VX task table w/ensemble info
gspetro-NOAA Jul 17, 2023
2adb67b
add slides for Purser ESG grid
gspetro-NOAA Jul 17, 2023
552d57b
RunSRW edits
gspetro-NOAA Jul 17, 2023
8f976cf
some QS updates
gspetro-NOAA Jul 17, 2023
43fa966
First set of updates in RunSRW.rst
mkavulich Jul 19, 2023
e6fca92
More updates for PR 864
mkavulich Jul 19, 2023
07ee7a5
'GET_OBS' isn't a valid config.yaml variable
mkavulich Jul 19, 2023
27095ed
Add section links for more information
mkavulich Jul 19, 2023
f13fec5
Better description of MRMS data staging script
mkavulich Jul 19, 2023
c074431
Merge pull request #1 from mkavulich/PR_864_doc_updates
gspetro-NOAA Jul 20, 2023
5a2c266
Merge branch 'ufs-community:develop' into text/ug-updates
gspetro-NOAA Jul 20, 2023
f0656eb
minor formatting
gspetro-NOAA Jul 21, 2023
519a427
add details on AQM executables & tasks
gspetro-NOAA Jul 29, 2023
78d7c6d
minor Tutorial & QS updates
gspetro-NOAA Jul 29, 2023
2bbc3ac
update AQM section
gspetro-NOAA Aug 1, 2023
79e9dbd
formatting fixes
gspetro-NOAA Aug 1, 2023
0b9547a
minor fixes
gspetro-NOAA Aug 1, 2023
4b323de
add AQM expt data info & wflow SUCCESS info
gspetro-NOAA Aug 2, 2023
f004d1f
minor WE2E updates
gspetro-NOAA Aug 2, 2023
2fe0f99
update testing ch
gspetro-NOAA Aug 3, 2023
c52ee61
reorganize WE2E, condense
gspetro-NOAA Aug 4, 2023
911ffdc
WE2E edits
gspetro-NOAA Aug 4, 2023
d74a1d1
update Container QS
gspetro-NOAA Aug 4, 2023
6865890
update VX cases ch
gspetro-NOAA Aug 4, 2023
5f0bcf5
update Tutorial
gspetro-NOAA Aug 4, 2023
9374a3e
update 1st half of FAQ
gspetro-NOAA Aug 4, 2023
b3e1c5f
update FAQ
gspetro-NOAA Aug 7, 2023
7570503
add comma
gspetro-NOAA Aug 7, 2023
f23e441
update Rocoto ch & minor details in other chs
gspetro-NOAA Aug 7, 2023
188a149
rename Tech Deteails ch
gspetro-NOAA Aug 7, 2023
e1ef957
update index file w/new dir names
gspetro-NOAA Aug 7, 2023
7c5818d
add ufs wm intersphinx
gspetro-NOAA Aug 8, 2023
3c2c727
I/O updates
gspetro-NOAA Aug 8, 2023
9806215
update file /path/to convention
gspetro-NOAA Aug 8, 2023
995ccf2
update file /path/to convention
gspetro-NOAA Aug 8, 2023
67c17af
update I/O ch
gspetro-NOAA Aug 8, 2023
b917456
resolve merge conflict from develop
gspetro-NOAA Aug 8, 2023
351141a
add FAQ update & link update
gspetro-NOAA Aug 8, 2023
bf48e21
Defining Wflow updates
gspetro-NOAA Aug 8, 2023
c3e0ba0
Defining Wflow updates
gspetro-NOAA Aug 9, 2023
f6d0b9b
Intro updates
gspetro-NOAA Aug 9, 2023
a8335a1
Intro updates
gspetro-NOAA Aug 9, 2023
a8f0b81
minor details
gspetro-NOAA Aug 10, 2023
6d30954
fix typo
gspetro-NOAA Aug 10, 2023
49c42af
resolve merge conflicts, add AQM data info
gspetro-NOAA Aug 10, 2023
0ea850a
revert changes to align w/v2.1.0 stable code
gspetro-NOAA Aug 10, 2023
ab6d093
add prdgen def
gspetro-NOAA Aug 11, 2023
95f7179
rm mention of any required file structure for *_OBS_DIR paths
gspetro-NOAA Aug 14, 2023
a686045
rm mentions of GSI/rrfs-utl
gspetro-NOAA Aug 15, 2023
ef77903
Merge branch 'ufs-community:develop' into text/ug-updates
gspetro-NOAA Aug 16, 2023
36b338f
Updates to WE2E tests
mkavulich Aug 16, 2023
7df67d5
Update WE2E testing doc umentation
mkavulich Aug 16, 2023
9aaafc4
fix typos/formatting
gspetro-NOAA Aug 16, 2023
51d7350
update # of CCPP suites
gspetro-NOAA Aug 16, 2023
349a304
update config_default user/platform sections
gspetro-NOAA Aug 16, 2023
c48a07c
fix merge conflicts
gspetro-NOAA Aug 16, 2023
6b11b2b
update table of comm config vals
gspetro-NOAA Aug 17, 2023
3084869
Add task_get_obs_nohrsc to VX tasks table
gspetro-NOAA Aug 17, 2023
b7c917b
Fix VX task name: metatask_PcpCombine_fcst_APCP_all_accums_all_mems
gspetro-NOAA Aug 17, 2023
8684019
Add VX task to table: metatask_PcpCombine_fcst_ASNOW_all_accums_all_mems
gspetro-NOAA Aug 17, 2023
fd0f849
Add VX task to table: metatask_GenEnsProd_EnsembleStat_NOHRSC
gspetro-NOAA Aug 17, 2023
389ab11
Fix VX task table: metatask_GridStat_NOHRSC_ensmeanprob_all_accums
gspetro-NOAA Aug 17, 2023
e2138b3
rm GSI from Glossary
gspetro-NOAA Aug 17, 2023
dcbd4c2
Merge branch 'text/ug-updates' of github.com:gspetro-NOAA/ufs-srweath…
gspetro-NOAA Aug 17, 2023
15eb7ae
fix typo
gspetro-NOAA Aug 17, 2023
c18f2b5
minor edits
gspetro-NOAA Aug 17, 2023
f4b7be8
Merge branch 'text/ug-updates' of github.com:gspetro-NOAA/ufs-srweath…
gspetro-NOAA Aug 17, 2023
9ab5d76
update file names/paths & defs
gspetro-NOAA Aug 17, 2023
7296127
Merge branch 'ufs-community:develop' into text/ug-updates
gspetro-NOAA Aug 21, 2023
b59aa4c
update file & path vars
gspetro-NOAA Aug 22, 2023
dbd71ab
updates to config defaults dirs
gspetro-NOAA Aug 25, 2023
c1d250c
update to head of develop
gspetro-NOAA Aug 25, 2023
406693a
Merge branch 'text/ug-updates' of github.com:gspetro-NOAA/ufs-srweath…
gspetro-NOAA Aug 25, 2023
a242b3b
Update RRFS mention in Intro
gspetro-NOAA Aug 25, 2023
6403974
minor cholt edits
gspetro-NOAA Aug 25, 2023
a24e663
update I/O NOMADS mention to S3 bucket
gspetro-NOAA Aug 25, 2023
40c794b
remove stray Run ch from PR 883 & fix table formatting
gspetro-NOAA Aug 25, 2023
79ef602
Update 18h to 18 UTC
gspetro-NOAA Aug 25, 2023
82202e1
rm stray RRFS/gsi mentions
gspetro-NOAA Aug 25, 2023
9f1aa91
Merge branch 'text/ug-updates' of github.com:gspetro-NOAA/ufs-srweath…
gspetro-NOAA Aug 25, 2023
0eba803
expand list of supported components
gspetro-NOAA Aug 25, 2023
5a8eb8d
update notes in Container ch
gspetro-NOAA Aug 25, 2023
1c8c3b9
QS and container QS updates
gspetro-NOAA Aug 25, 2023
0d0257d
rm package list; add uwtools env instructions
gspetro-NOAA Aug 25, 2023
31cc9ef
clarify re EXPTDIR
gspetro-NOAA Aug 28, 2023
629186b
rm confusing csh ref
gspetro-NOAA Aug 28, 2023
4713c3e
clarify bash/csh env commands
gspetro-NOAA Aug 28, 2023
044e94a
updates to dirs, CCPP, grids, fcst params
gspetro-NOAA Aug 28, 2023
6f96f7f
fix typo
gspetro-NOAA Aug 28, 2023
9837c97
minor fixes
gspetro-NOAA Aug 29, 2023
1a84595
Update RunSRW to rm $PWD in config_utils cmd
gspetro-NOAA Aug 29, 2023
e606bda
Update Intro QS Description
gspetro-NOAA Aug 29, 2023
109b88c
Update CQS crosslink
gspetro-NOAA Aug 29, 2023
a6fb29d
Update instructions on providing error msg
gspetro-NOAA Aug 29, 2023
92aea33
update software prereqs
gspetro-NOAA Aug 29, 2023
edb8531
update ESMF SCRIP abbrev
gspetro-NOAA Aug 29, 2023
d0c8ee1
link to j-jobs in glossary
gspetro-NOAA Aug 29, 2023
0d95dcc
update instructions on filing GitHub issues
gspetro-NOAA Aug 29, 2023
705ea15
Update Intro info on QS
gspetro-NOAA Aug 29, 2023
3823a3c
rm redundant umbrella repo def
gspetro-NOAA Aug 29, 2023
d8c8407
Merge branch 'text/ug-updates' of github.com:gspetro-NOAA/ufs-srweath…
gspetro-NOAA Aug 29, 2023
1201e24
Update intro w/user support recs
gspetro-NOAA Aug 29, 2023
09e10ca
rm note re container binding on Jet
gspetro-NOAA Aug 29, 2023
62e54eb
Merge branch 'text/ug-updates' of github.com:gspetro-NOAA/ufs-srweath…
gspetro-NOAA Aug 29, 2023
4261f08
Update container docs w/Apptainer info
gspetro-NOAA Aug 29, 2023
6e55e60
fix j-job xref
gspetro-NOAA Aug 29, 2023
a4ca1d0
fix link
gspetro-NOAA Aug 29, 2023
61515b8
Change name of Ch 3 to 'Customizing the Workflow'
gspetro-NOAA Aug 30, 2023
5b045a0
Update CQS ch w/Gaea c5 info
gspetro-NOAA Aug 30, 2023
653fe01
rename to Customizing the Workflow
gspetro-NOAA Aug 30, 2023
4f6ae16
update link to ccpp sci docs
gspetro-NOAA Aug 30, 2023
4e0cd8f
Merge branch 'text/ug-updates' into text/config
gspetro-NOAA Aug 30, 2023
8c56c32
resolve conflicts
gspetro-NOAA Aug 30, 2023
7d1ffb6
explain FCST_LEN_CYCL & related vars
gspetro-NOAA Aug 30, 2023
fb22f70
rm RRFS/GSI vars from config defaults per PR #893
gspetro-NOAA Aug 30, 2023
3a7300a
update NCO vars
gspetro-NOAA Aug 31, 2023
581d4cc
edit grid dir/esggrid params
gspetro-NOAA Aug 31, 2023
471f6cd
update FCST_LEN_CYCL notes/hints
gspetro-NOAA Aug 31, 2023
215faf4
update GFDLgrid params, condense definitions
gspetro-NOAA Sep 5, 2023
a073313
update vars for initial wflow tasks
gspetro-NOAA Sep 5, 2023
4ef6735
update most of last third of config defaults
gspetro-NOAA Sep 7, 2023
a6036aa
add VX, AQM, and rocoto params
gspetro-NOAA Sep 11, 2023
1318b20
edit/rm comments pt1
gspetro-NOAA Sep 11, 2023
e31a194
update default workflow vars table
gspetro-NOAA Sep 14, 2023
9fd9d0d
update Default vars tbl; other minor edits
gspetro-NOAA Sep 18, 2023
d3533fd
update testing categories and HPSS info
gspetro-NOAA Sep 18, 2023
baaa246
minor updates
gspetro-NOAA Sep 20, 2023
e8afcce
add CCPP_PHYS_DIR
gspetro-NOAA Sep 20, 2023
709abe8
update a few remaining defs from config defaults
gspetro-NOAA Sep 20, 2023
8c50fae
container ch updates
gspetro-NOAA Sep 20, 2023
9b81439
fix formatting
gspetro-NOAA Sep 21, 2023
d1c5e24
fix typo in test ch
gspetro-NOAA Sep 21, 2023
d048847
add comment
gspetro-NOAA Sep 21, 2023
bcf0d4f
fix formatting; add definitions
gspetro-NOAA Sep 21, 2023
391ef33
fix formatting, tweak containter ch
gspetro-NOAA Sep 21, 2023
d4b57ec
add MERRA2 note
gspetro-NOAA Sep 21, 2023
1b31362
Mike L's fixes
gspetro-NOAA Sep 22, 2023
266e353
update note about tests needing HPSS access
gspetro-NOAA Sep 22, 2023
99a6c4b
Merge branch 'develop' into text/config
gspetro-NOAA Sep 22, 2023
36ca401
minor edits
gspetro-NOAA Sep 22, 2023
5ce632f
fix *_OBS_DIR defs
gspetro-NOAA Sep 22, 2023
a180461
fix *_OBS_DIR defs in config defaults
gspetro-NOAA Sep 22, 2023
5d38a67
change 'file' to 'script'
gspetro-NOAA Sep 22, 2023
160d567
add missing TEST_* vars and minor formatting
gspetro-NOAA Sep 29, 2023
0aec769
fix typo
gspetro-NOAA Oct 2, 2023
992b4a0
fix typos
gspetro-NOAA Oct 2, 2023
2fdb80f
Merge branch 'ufs-community:develop' into text/config
gspetro-NOAA Oct 4, 2023
8f7a217
escape chars
gspetro-NOAA Oct 4, 2023
edb565b
Merge branch 'text/config' of github.com:gspetro-NOAA/ufs-srweather-a…
gspetro-NOAA Oct 4, 2023
77e0de7
Update test info re HPSS access
gspetro-NOAA Oct 4, 2023
138131b
Reword MPI/preprocessing paragraph
gspetro-NOAA Oct 4, 2023
37c5296
Rm unnecessary info
gspetro-NOAA Oct 4, 2023
ead642e
Add note re: FIXaer/Thompson MP
gspetro-NOAA Oct 4, 2023
4846797
Add note re: FIXlut/Thompson MP
gspetro-NOAA Oct 4, 2023
81e5d23
minor updates; note about vars not to configure
gspetro-NOAA Oct 4, 2023
9f25833
Merge branch 'text/config' of github.com:gspetro-NOAA/ufs-srweather-a…
gspetro-NOAA Oct 4, 2023
c62945f
rm outdated note about comp params
gspetro-NOAA Oct 4, 2023
a3002e0
note that AQM features are not supported for community
gspetro-NOAA Oct 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/UsersGuide/source/BuildingRunningTesting/BuildSRW.rst
Original file line number Diff line number Diff line change
Expand Up @@ -294,8 +294,8 @@ If the ``devbuild.sh`` approach failed, users need to set up their environment t

.. code-block:: console

source etc/lmod-setup.sh gaea
source etc/lmod-setup.csh gaea
source /path/to/ufs-srweather-app/etc/lmod-setup.sh gaea
source /path/to/ufs-srweather-app/etc/lmod-setup.csh gaea

.. note::

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,12 @@ Build the Container
------------------------

.. hint::
If a ``singularity: command not found`` error message appears when working on Level 1 platforms, try running: ``module load singularity``.
If a ``singularity: command not found`` error message appears when working on Level 1 platforms, try running: ``module load singularity`` or (on Derecho) ``module load apptainer``.

Level 1 Systems
^^^^^^^^^^^^^^^^^^

On most Level 1 systems, a container named ``ubuntu20.04-intel-srwapp-develop.img`` has already been built at the following locations:
On most Level 1 systems, a container named ``ubuntu20.04-intel-ue-1.4.1-srw-dev.img`` has already been built at the following locations:

.. table:: Locations of pre-built containers

Expand All @@ -95,29 +95,27 @@ On most Level 1 systems, a container named ``ubuntu20.04-intel-srwapp-develop.im
+--------------+--------------------------------------------------------+

.. note::
* Singularity is only available on the Gaea C5 partition, and therefore container use is only supported on Gaea C5.
* On Gaea, Singularity/Apptainer is only available on the C5 partition, and therefore container use is only supported on Gaea C5.
* The NOAA Cloud containers are accessible only to those with EPIC resources.

Users can simply set an environment variable to point to the container:

.. code-block:: console

export img=/path/to/ubuntu20.04-intel-srwapp-develop.img
export img=/path/to/ubuntu20.04-intel-ue-1.4.1-srw-dev.img

Users may convert the container ``.img`` file to a writable sandbox. This step is required when running on Cheyenne but is optional on other systems:

.. code-block:: console

singularity build --sandbox ubuntu20.04-intel-srwapp $img

.. COMMENT: What about on Derecho?

When making a writable sandbox on Level 1 systems, the following warnings commonly appear and can be ignored:

.. code-block:: console

INFO: Starting build...
INFO: Verifying bootstrap image ubuntu20.04-intel-srwapp-develop.img
INFO: Verifying bootstrap image ubuntu20.04-intel-ue-1.4.1-srw-dev.img
WARNING: integrity: signature not found for object group 1
WARNING: Bootstrap image could not be verified, but build will continue.

Expand Down Expand Up @@ -241,7 +239,7 @@ To generate the forecast experiment, users must:
#. :ref:`Set experiment parameters <SetUpConfigFileC>`
#. :ref:`Run a script to generate the experiment workflow <GenerateWorkflowC>`

The first two steps depend on the platform being used and are described here for Level 1 platforms. Users will need to adjust the instructions to their machine if their local machine is a Level 2-4 platform.
The first two steps depend on the platform being used and are described here for Level 1 platforms. Users will need to adjust the instructions to match their machine configuration if their local machine is a Level 2-4 platform.

.. _SetUpPythonEnvC:

Expand Down Expand Up @@ -277,8 +275,6 @@ The ``wflow_<platform>`` modulefile will then output instructions to activate th

then the user should run ``conda activate workflow_tools``. This will activate the ``workflow_tools`` conda environment. The command(s) will vary from system to system, but the user should see ``(workflow_tools)`` in front of the Terminal prompt at this point.

.. COMMENT: Containers are old and still say regional_workflow...

.. _SetUpConfigFileC:

Configure the Workflow
Expand All @@ -295,13 +291,13 @@ where:
* ``-c`` indicates the compiler on the user's local machine (e.g., ``intel/2022.1.2``)
* ``-m`` indicates the :term:`MPI` on the user's local machine (e.g., ``impi/2022.1.2``)
* ``<platform>`` refers to the local machine (e.g., ``hera``, ``jet``, ``noaacloud``, ``mac``). See ``MACHINE`` in :numref:`Section %s <user>` for a full list of options.
* ``-i`` indicates the container image that was built in :numref:`Step %s <BuildC>` (``ubuntu20.04-intel-srwapp`` or ``ubuntu20.04-intel-srwapp-develop.img`` by default).
* ``-i`` indicates the container image that was built in :numref:`Step %s <BuildC>` (``ubuntu20.04-intel-srwapp`` or ``ubuntu20.04-intel-ue-1.4.1-srw-dev.img`` by default).

For example, on Hera, the command would be:

.. code-block:: console

./stage-srw.sh -c=intel/2022.1.2 -m=impi/2022.1.2 -p=hera -i=ubuntu20.04-intel-srwapp-develop.img
./stage-srw.sh -c=intel/2022.1.2 -m=impi/2022.1.2 -p=hera -i=ubuntu20.04-intel-ue-1.4.1-srw-dev.img

.. attention::

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,36 +11,28 @@ Table of Variables in ``config_defaults.yaml``
* - Group Name
- Configuration variables
* - User
- RUN_ENVIR, MACHINE, ACCOUNT, HOMEdir, USHdir, SCRIPTSdir, JOBSdir, SORCdir, PARMdir, MODULESdir, EXECdir, VX_CONFIG_DIR, METPLUS_CONF, MET_CONFIG, UFS_WTHR_MDL_DIR, ARL_NEXUS_DIR
- RUN_ENVIR, MACHINE, ACCOUNT, HOMEdir, USHdir, SCRIPTSdir, JOBSdir, SORCdir, PARMdir, MODULESdir, EXECdir, METPLUS_CONF, UFS_WTHR_MDL_DIR, ARL_NEXUS_DIR
mkavulich marked this conversation as resolved.
Show resolved Hide resolved
* - Platform
- WORKFLOW_MANAGER, NCORES_PER_NODE, TASKTHROTTLE, BUILD_MOD_FN, WFLOW_MOD_FN, BUILD_VER_FN, RUN_VER_FN, SCHED,PARTITION_DEFAULT, QUEUE_DEFAULT, PARTITION_HPSS,
QUEUE_HPSS, PARTITION_FCST, QUEUE_FCST, REMOVE_MEMORY, RUN_CMD_SERIAL, RUN_CMD_UTILS, RUN_CMD_FCST, RUN_CMD_POST, RUN_CMD_PRDGEN, RUN_CMD_AQM,
RUN_CMD_AQMLBC, SCHED_NATIVE_CMD, CCPA_OBS_DIR, MRMS_OBS_DIR, NDAS_OBS_DIR, NOHRSC_OBS_DIR, DOMAIN_PREGEN_BASEDIR, PRE_TASK_CMDS,
RUN_CMD_AQMLBC, SCHED_NATIVE_CMD, PRE_TASK_CMDS, CCPA_OBS_DIR, NOHRSC_OBS_DIR, MRMS_OBS_DIR, NDAS_OBS_DIR, DOMAIN_PREGEN_BASEDIR,
TEST_EXTRN_MDL_SOURCE_BASEDIR, TEST_AQM_INPUT_BASEDIR, TEST_PREGEN_BASEDIR, TEST_ALT_EXTRN_MDL_SYSBASEDIR_ICS, TEST_ALT_EXTRN_MDL_SYSBASEDIR_LBCS,
TEST_VX_FCST_INPUT_BASEDIR, FIXgsm, FIXaer, FIXlut, FIXorg, FIXsfc, FIXshp, FIXgsi, FIXcrtm, FIXcrtmupp, EXTRN_MDL_DATA_STORES
TEST_VX_FCST_INPUT_BASEDIR, FIXgsm, FIXaer, FIXlut, FIXorg, FIXsfc, FIXshp, FIXcrtm, FIXcrtmupp, EXTRN_MDL_DATA_STORES
* - Workflow
- WORKFLOW_ID, RELATIVE_LINK_FLAG, USE_CRON_TO_RELAUNCH, CRON_RELAUNCH_INTVL_MNTS, CRONTAB_LINE, LOAD_MODULES_RUN_TASK_FP, EXPT_BASEDIR, EXPT_SUBDIR, EXEC_SUBDIR,
EXPTDIR, DOT_OR_USCORE, EXPT_CONFIG_FN, CONSTANTS_FN, RGNL_GRID_NML_FN, FV3_NML_FN, FV3_NML_BASE_SUITE_FN, FV3_NML_YAML_CONFIG_FN, FV3_NML_BASE_ENS_FN,
FV3_EXEC_FN, DATA_TABLE_FN, DIAG_TABLE_FN, FIELD_TABLE_FN, DIAG_TABLE_TMPL_FN, FIELD_TABLE_TMPL_FN, MODEL_CONFIG_FN, NEMS_CONFIG_FN, AQM_RC_FN, AQM_RC_TMPL_FN,
FV3_NML_BASE_SUITE_FP, FV3_NML_YAML_CONFIG_FP, FV3_NML_BASE_ENS_FP, DATA_TABLE_TMPL_FP, DIAG_TABLE_TMPL_FP, FIELD_TABLE_TMPL_FP,
MODEL_CONFIG_TMPL_FP, NEMS_CONFIG_TMPL_FP, AQM_RC_TMPL_FP, DATA_TABLE_FP, FIELD_TABLE_FP, NEMS_CONFIG_FP, FV3_NML_FP, FV3_NML_CYCSFC_FP,
FV3_NML_RESTART_FP, FV3_NML_STOCH_FP, FV3_NML_RESTART_STOCH_FP, FCST_MODEL, WFLOW_XML_FN, GLOBAL_VAR_DEFNS_FN, ROCOTO_YAML_FN, EXTRN_MDL_VAR_DEFNS_FN,
MODEL_CONFIG_TMPL_FP, NEMS_CONFIG_TMPL_FP, AQM_RC_TMPL_FP, DATA_TABLE_FP, FIELD_TABLE_FP, NEMS_CONFIG_FP, FV3_NML_FP,
FV3_NML_STOCH_FP, FCST_MODEL, WFLOW_XML_FN, GLOBAL_VAR_DEFNS_FN, ROCOTO_YAML_FN, EXTRN_MDL_VAR_DEFNS_FN,
WFLOW_LAUNCH_SCRIPT_FN, WFLOW_LAUNCH_LOG_FN, GLOBAL_VAR_DEFNS_FP, ROCOTO_YAML_FP, WFLOW_LAUNCH_SCRIPT_FP, WFLOW_LAUNCH_LOG_FP, FIXdir, FIXam,
FIXclim, FIXlam, THOMPSON_MP_CLIMO_FN, THOMPSON_MP_CLIMO_FP, CCPP_PHYS_SUITE, CCPP_PHYS_SUITE_FN, CCPP_PHYS_SUITE_IN_CCPP_FP, CCPP_PHYS_SUITE_FP, CCPP_PHYS_DIR,
FIELD_DICT_FN, FIELD_DICT_IN_UWM_FP, FIELD_DICT_FP, GRID_GEN_METHOD, PREDEF_GRID_NAME, DATE_FIRST_CYCL, DATE_LAST_CYCL, INCR_CYCL_FREQ, FCST_LEN_HRS,
FCST_LEN_CYCL, LONG_FCST_LEN, CYCL_HRS_SPINSTART, CYCL_HRS_PRODSTART, BOUNDARY_LEN_HRS, BOUNDARY_LONG_LEN_HRS, BOUNDARY_PROC_GROUP_NUM,
PREEXISTING_DIR_METHOD, VERBOSE, DEBUG, COMPILER, SYMLINK_FIX_FILES, DO_REAL_TIME, COLDSTART, WARMSTART_CYCLE_DIR,
FCST_LEN_CYCL, LONG_FCST_LEN, PREEXISTING_DIR_METHOD, VERBOSE, DEBUG, COMPILER, SYMLINK_FIX_FILES, DO_REAL_TIME, COLDSTART, WARMSTART_CYCLE_DIR,
* - NCO
- envir_default, NET_default, RUN_default, model_ver_default, OPSROOT_default, COMROOT_default, DATAROOT_default, DCOMROOT_default, LOGBASEDIR_default,
COMIN_BASEDIR, COMOUT_BASEDIR, NWGES, NWGES_BASEDIR, DBNROOT_default, SENDECF_default, SENDDBN_default, SENDDBN_NTC_default, SENDCOM_default,
COMIN_BASEDIR, COMOUT_BASEDIR, DBNROOT_default, SENDECF_default, SENDDBN_default, SENDDBN_NTC_default, SENDCOM_default,
SENDWEB_default, KEEPDATA_default, MAILTO_default, MAILCC_default
* - gsi
- niter1, niter2, l_obsprvdiag, diag_radardbz, write_diag_2, bkgerr_vs, bkgerr_hzscl, usenewgfsberror, netcdf_diag, binary_diag, readin_localization,
beta1_inv, ens_h, ens_v, regional_ensemble_option, grid_ratio_fv3, grid_ratio_ens, i_en_perts_io, q_hyb_ens, ens_fast_read, l_PBL_pseudo_SurfobsT,
l_PBL_pseudo_SurfobsQ, i_use_2mQ4B, i_use_2mT4B, i_T_Q_adjust, l_rtma3d, i_precip_vertical_check, HYBENSMEM_NMIN, ANAVINFO_FN, ANAVINFO_DBZ_FN,
ENKF_ANAVINFO_FN, ENKF_ANAVINFO_DBZ_FN, CONVINFO_FN, BERROR_FN, OBERROR_FN, HYBENSINFO_FN, cld_bld_hgt, l_precip_clear_only, l_qnr_from_qr, beta_recenter
* - rrfs
- DO_RRFS_DEV, DO_NLDN_LGHT, DO_ENKFUPDATE, DO_DACYCLE, DO_SURFACE_CYCLE
* - task_make_grid
- GRID_DIR, ESGgrid_LON_CTR, ESGgrid_LAT_CTR, ESGgrid_DELX, ESGgrid_DELY, ESGgrid_NX, ESGgrid_NY, ESGgrid_WIDE_HALO_WIDTH, ESGgrid_PAZI,
GFDLgrid_LON_T6_CTR, GFDLgrid_LAT_T6_CTR, GFDLgrid_NUM_CELLS, GFDLgrid_STRETCH_FAC, GFDLgrid_REFINE_RATIO, GFDLgrid_ISTART_OF_RGNL_DOM_ON_T6G,
Expand All @@ -53,7 +45,7 @@ Table of Variables in ``config_defaults.yaml``
- EXTRN_MDL_NAME_ICS, EXTRN_MDL_ICS_OFFSET_HRS, FV3GFS_FILE_FMT_ICS, EXTRN_MDL_SYSBASEDIR_ICS, USE_USER_STAGED_EXTRN_FILES,
EXTRN_MDL_SOURCE_BASEDIR_ICS, EXTRN_MDL_FILES_ICS
* - task_get_extrn_lbcs
- EXTRN_MDL_NAME_LBCS, LBC_SPEC_INTVL_HRS, EXTRN_MDL_LBCS_OFFSET_HRS, FV3GFS_FILE_FMT_LBCS, LBCS_SEARCH_HRS, EXTRN_MDL_LBCS_SEARCH_OFFSET_HRS, EXTRN_MDL_SYSBASEDIR_LBCS,
- EXTRN_MDL_NAME_LBCS, LBC_SPEC_INTVL_HRS, EXTRN_MDL_LBCS_OFFSET_HRS, FV3GFS_FILE_FMT_LBCS, EXTRN_MDL_SYSBASEDIR_LBCS,
USE_USER_STAGED_EXTRN_FILES,EXTRN_MDL_SOURCE_BASEDIR_LBCS, EXTRN_MDL_FILES_LBCS
* - task_make_ics
- KMP_AFFINITY_MAKE_ICS, OMP_NUM_THREADS_MAKE_ICS, OMP_STACKSIZE_MAKE_ICS, USE_FVCOM, FVCOM_WCSTART, FVCOM_DIR, FVCOM_FILE, VCOORD_FILE
Expand All @@ -71,14 +63,6 @@ Table of Variables in ``config_defaults.yaml``
- KMP_AFFINITY_RUN_PRDGEN, OMP_NUM_THREADS_RUN_PRDGEN, OMP_STACKSIZE_RUN_PRDGEN, DO_PARALLEL_PRDGEN, ADDNL_OUTPUT_GRIDS: []
* - task_plot_allvars:
- COMOUT_REF, PLOT_FCST_START, PLOT_FCST_INC, PLOT_FCST_END, PLOT_DOMAINS
* - task_analysis_gsi
- TN_ANALYSIS_GSI, TN_OBSERVER_GSI, TN_OBSERVER_GSI_ENSMEAN, KMP_AFFINITY_ANALYSIS, OMP_NUM_THREADS_ANALYSIS, OMP_STACKSIZE_ANALYSIS, OBSPATH_TEMPLATE
* - task_process_radarref
- RADAR_REF_THINNING, RADARREFL_MINS, RADARREFL_TIMELEVEL, OBS_SUFFIX
* - task_get_da_obs
- NLDN_NEEDED, NLDN_LIGHTNING, NSSLMOSAIC, RAP_OBS_BUFR
* - task_process_bufrobs
- OBSPATH_TEMPLATE
* - task_nexus_emission
- PPN_NEXUS_EMISSION, KMP_AFFINITY_NEXUS_EMISSION, OMP_NUM_THREADS_NEXUS_EMISSION, OMP_STACKSIZE_NEXUS_EMISSION
* - task_bias_correction_o3
Expand All @@ -88,19 +72,19 @@ Table of Variables in ``config_defaults.yaml``
* - Global
- USE_CRTM, CRTM_DIR, DO_ENSEMBLE, NUM_ENS_MEMBERS, ENSMEM_NAMES, FV3_NML_ENSMEM_FPS, ENS_TIME_LAG_HRS, DO_SHUM, DO_SPPT, DO_SKEB, ISEED_SHUM, ISEED_SPPT, ISEED_SKEB, NEW_LSCALE, SHUM_MAG, SHUM_LSCALE, SHUM_TSCALE, SHUM_INT,
SPPT_MAG, SPPT_LOGIT, SPPT_LSCALE, SPPT_TSCALE, SPPT_INT, SPPT_SFCLIMIT,
SKEB_MAG, SKEB_LSCALE, SKEP_TSCALE, SKEB_INT, SKEBNORM, SKEB_VDOF, USE_ZMTNBLCK, DO_SPP, ISEED_SPP, SPP_VAR_LIST, SPP_MAG_LIST, SPP_LSCALE,
SPP_TSCALE, SPP_SIGTOP1, SPP_SIGTOP2, SPP_STDDEV_CUTOFF, DO_LSM_SPP, LSM_SPP_TSCALE, LSM_SPP_LSCALE, ISEED_LSM_SPP, LSM_SPP_VAR_LIST,
SKEB_MAG, SKEB_LSCALE, SKEP_TSCALE, SKEB_INT, SKEBNORM, SKEB_VDOF, USE_ZMTNBLCK, DO_SPP, SPP_VAR_LIST, SPP_MAG_LIST, SPP_LSCALE,
SPP_TSCALE, SPP_SIGTOP1, SPP_SIGTOP2, SPP_STDDEV_CUTOFF, ISEED_SPP, DO_LSM_SPP, LSM_SPP_TSCALE, LSM_SPP_LSCALE, ISEED_LSM_SPP, LSM_SPP_VAR_LIST,
LSM_SPP_MAG_LIST, HALO_BLEND, PRINT_DIFF_PGR
* - Verification
- OBS_CCPA_APCP01h_FN_TEMPLATE, OBS_CCPA_APCPgt01h_FN_TEMPLATE, OBS_MRMS_REFC_FN_TEMPLATE, OBS_MRMS_RETOP_FN_TEMPLATE,
OBS_NDAS_SFCorUPA_FN_TEMPLATE, OBS_NDAS_SFCorUPA_FN_METPROC_TEMPLATE, VX_FCST_MODEL_NAME, VX_FIELDS, VX_APCP_ACCUMS_HRS, VX_FCST_INPUT_BASEDIR,
- OBS_CCPA_APCP01h_FN_TEMPLATE, OBS_CCPA_APCPgt01h_FN_TEMPLATE, OBS_NOHRSC_ASNOW_FN_TEMPLATE, OBS_MRMS_REFC_FN_TEMPLATE, OBS_MRMS_RETOP_FN_TEMPLATE,
OBS_NDAS_SFCorUPA_FN_TEMPLATE, OBS_NDAS_SFCorUPA_FN_METPROC_TEMPLATE, VX_FCST_MODEL_NAME, VX_FIELDS, VX_APCP_ACCUMS_HRS, VX_ASNOW_ACCUMS_HRS, VX_FCST_INPUT_BASEDIR,
VX_OUTPUT_BASEDIR, VX_NDIGITS_ENSMEM_NAMES, FCST_SUBDIR_TEMPLATE, FCST_FN_TEMPLATE, FCST_FN_METPROC_TEMPLATE, NUM_MISSING_OBS_FILES_MAX, NUM_MISSING_FCST_FILES_MAX
* - cpl_aqm_parm
- CPL_AQM, DO_AQM_DUST, DO_AQM_CANOPY, DO_AQM_PRODUCT, DO_AQM_CHEM_LBCS, DO_AQM_GEFS_LBCS, DO_AQM_SAVE_AIRNOW_HIST, DO_AQM_SAVE_FIRE, DCOMINbio_default,
DCOMINdust_default, DCOMINcanopy_default, DCOMINfire_default, DCOMINchem_lbcs_default, DCOMINgefs_default, DCOMINpt_src_default,
DCOMINairnow_default, COMINbicor, COMOUTbicor, AQM_CONFIG_DIR, AQM_BIO_FILE, AQM_DUST_FILE_PREFIX, AQM_DUST_FILE_SUFFIX, AQM_CANOPY_FILE_PREFIX,
AQM_CANOPY_FILE_SUFFIX, AQM_FIRE_FILE_PREFIX, AQM_FIRE_FILE_SUFFIX, AQM_FIRE_FILE_OFFSET_HRS, AQM_FIRE_ARCHV_DIR, AQM_RC_FIRE_FREQUENCY,
AQM_RC_PRODUCT_FN, AQM_RC_PRODUCT_FREQUENCY, AQM_LBCS_FILES, AQM_GEFS_FILE_PREFIX, AQM_GEFS_FILE_CYC, NEXUS_INPUT_DIR, NEXUS_FIX_DIR,
NEXUS_GRID_FN, NUM_SPLIT_NEXUS: 3NEXUS_GFS_SFC_OFFSET_HRS, NEXUS_GFS_SFC_DIR, NEXUS_GFS_SFC_ARCHV_DIR
NEXUS_GRID_FN, NUM_SPLIT_NEXUS, NEXUS_GFS_SFC_OFFSET_HRS, NEXUS_GFS_SFC_DIR, NEXUS_GFS_SFC_ARCHV_DIR
* - Rocoto
- attrs, cycledefs, entities, log, tasks: taskgroups
- attrs, cycledefs, entities, log, tasks, taskgroups
6 changes: 2 additions & 4 deletions docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ In future shells, you can activate and use this environment with:
source ~/conda/etc/profile.d/conda.sh
conda activate uwtools

See the `workflow-tools respository <https://github.com/ufs-community/workflow-tools>`__ for additional documentation.
See the `workflow-tools repository <https://github.com/ufs-community/workflow-tools>`__ for additional documentation.

Modify a ``wflow_<platform>`` File
``````````````````````````````````````
Expand Down Expand Up @@ -561,8 +561,6 @@ output over the :term:`CONUS`. It generates graphics plots for a number of varia
* Max/Min 2-5 km updraft helicity
* Sea level pressure (SLP)

.. COMMENT: * 500 hPa heights, winds, and vorticity --> seems to be omitted? Why?

This workflow task can produce both plots from a single experiment and difference plots that compare the same cycle from two experiments. When plotting the difference, the two experiments must be on the same domain and available for
the same cycle starting date/time and forecast hours. Other parameters may differ (e.g., the experiments may use different physics suites).

Expand Down Expand Up @@ -702,7 +700,7 @@ Run the following command from the ``ufs-srweather-app/ush`` directory to genera

The last line of output from this script, starting with ``*/1 * * * *`` or ``*/3 * * * *``, can be saved and used later to automatically run portions of the workflow if users have the Rocoto workflow manager installed on their system.

This workflow generation script creates an experiment directory and populates it with all the data needed to run through the workflow. The flowchart in :numref:`Figure %s <WorkflowGeneration>` describes the experiment generation process. The ``generate_FV3LAM_wflow.py``:
This workflow generation script creates an experiment directory and populates it with all the data needed to run through the workflow. The flowchart in :numref:`Figure %s <WorkflowGeneration>` describes the experiment generation process. The ``generate_FV3LAM_wflow.py`` file:
gspetro-NOAA marked this conversation as resolved.
Show resolved Hide resolved

#. Runs the ``setup.py`` script to set the configuration parameters. This script reads three other configuration scripts in order:

Expand Down
24 changes: 21 additions & 3 deletions docs/UsersGuide/source/BuildingRunningTesting/WE2Etests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ The list of fundamental and comprehensive tests can be viewed in the ``ufs-srwea

For convenience, the WE2E tests are currently grouped into the following categories (under ``ufs-srweather-app/tests/WE2E/test_configs/``):

* ``aqm``
This category tests the :term:`AQM` configuration of the SRW App.

* ``custom_grids``
This category tests custom grids aside from those specified in ``ufs-srweather-app/ush/predef_grid_params.yaml``. These tests help ensure a wide range of domain sizes, resolutions, and locations will work as expected. These test files can also serve as examples for how to set your own custom domain.

Expand All @@ -38,15 +41,30 @@ For convenience, the WE2E tests are currently grouped into the following categor
* ``grids_extrn_mdls_suites_nco``
This category of tests ensures that the workflow running in **NCO mode** (i.e., with ``RUN_ENVIR`` set to ``"nco"``) completes successfully for various combinations of predefined grids, physics suites, and input data from different external models. Note that in NCO mode, an operational run environment is used. This involves a specific directory structure and variable names (see :numref:`Section %s <NCOModeParms>`).

* ``ufs_case_studies``
This category tests that the workflow running in community mode completes successfully when running cases derived from the `ufs-case-studies repository <https://github.com/dtcenter/ufs-case-studies>`__.

* ``verification``
This category specifically tests the various combinations of verification capabilities using METPlus.

* ``release_SRW_v1``
This category was reserved for the official "Graduate Student Test" case for the Version 1 SRW code release.

* ``wflow_features``
This category of tests ensures that the workflow completes successfully with particular features/capabilities activated.

.. note::

Users should be aware that some tests assume :term:`HPSS` access.

* ``custom_ESGgrid_Great_Lakes_snow_8km`` and ``MET_verification_only_vx_time_lag`` require HPSS access, as well as ``rstprod`` access on both :term:`RDHPCS` and HPSS.
* On certain machines, *community* and *nco* tests assume HPSS access. If the ``ush/machine/*.yaml`` file contains the following lines, and these paths are different from what is provided in ``TEST_EXTRN_MDL_SOURCE_BASEDIR``, users will need to have HPSS access or modify the tests to point to another data source:
gspetro-NOAA marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: console

data:
ics_lbcs:
FV3GFS:
RAP:
HRRR:

Some tests are duplicated among the above categories via symbolic links, both for legacy reasons (when tests for different capabilities were consolidated) and for convenience when a user would like to run all tests for a specific category (e.g., verification tests).

Running the WE2E Tests
Expand Down
Loading
Loading