Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[uwtools_integration] Integrate run_fcst #271

Open
wants to merge 41 commits into
base: uwtools_integration
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
2d77662
Initial edits to coldstart and creation of run_fcst script
NaureenBharwaniNOAA Oct 11, 2024
4b03af5
Merge branch 'uwtools_integration' of https://github.com/NOAA-GSL/ufs…
NaureenBharwaniNOAA Oct 11, 2024
1c7faec
Updates to run_fcst script
NaureenBharwaniNOAA Oct 11, 2024
12f938c
Update command, fail jsonschema check
NaureenBharwaniNOAA Oct 17, 2024
4249ce2
Updates to config defaults
NaureenBharwaniNOAA Oct 18, 2024
c596c95
Update diag_table and file_copy config keys
NaureenBharwaniNOAA Oct 22, 2024
03e01d1
Config updates
NaureenBharwaniNOAA Oct 22, 2024
16e2ac0
Update config
NaureenBharwaniNOAA Oct 22, 2024
9947701
Update config
NaureenBharwaniNOAA Oct 23, 2024
3e7584e
WIP
christinaholtNOAA Oct 25, 2024
89e9fd1
Set the fix files directly in the fv3 config.
christinaholtNOAA Oct 30, 2024
caec469
Adding a quilting YAML.
christinaholtNOAA Oct 30, 2024
2e4eec1
Light testing fixes. Fixing stochastic namelist settings.
christinaholtNOAA Oct 30, 2024
aca7d70
Cleanup and applying black.
christinaholtNOAA Oct 30, 2024
0b92eb1
Merge pull request #1 from christinaholtNOAA/model_configure_to_yaml
NaureenBharwaniNOAA Oct 31, 2024
18ff149
Updates to config defaults and run script
NaureenBharwaniNOAA Oct 31, 2024
52f3c0f
Merge branch 'integrate-run-fcst' of https://github.com/NaureenBharwa…
NaureenBharwaniNOAA Oct 31, 2024
2c08014
Debugging
NaureenBharwaniNOAA Oct 31, 2024
5cf2b78
Debugging setup.py
NaureenBharwaniNOAA Oct 31, 2024
d22af95
Add fv3nml sfc back
NaureenBharwaniNOAA Oct 31, 2024
0bf8940
Remove call to nonexistent script.
christinaholtNOAA Nov 6, 2024
a4327be
Merge remote-tracking branch 'gsl/uwtools_integration' into integrate…
christinaholtNOAA Nov 6, 2024
5ecc61f
Updates to make_grid
NaureenBharwaniNOAA Nov 7, 2024
63cb2f5
Merge remote-tracking branch 'nb/integrate-run-fcst' into integrate-r…
christinaholtNOAA Nov 7, 2024
4cdabd7
WIP
christinaholtNOAA Nov 7, 2024
67a461c
Cleanup and make xml work.
christinaholtNOAA Nov 12, 2024
4951956
Adding config and tasks for aqm and upp
christinaholtNOAA Nov 13, 2024
7c82f63
Fix for having removed dot_or_underscore
christinaholtNOAA Nov 13, 2024
647a0fd
Merge remote-tracking branch 'gsl/uwtools_integration' into integrate…
christinaholtNOAA Nov 13, 2024
1ad0a41
No errors provisioning a rundir.
christinaholtNOAA Nov 14, 2024
2f0a8c2
Forecast runs.
christinaholtNOAA Nov 19, 2024
ea236bd
Adding grib delivery to the script.
christinaholtNOAA Nov 19, 2024
28c031f
Fix for integration test.
christinaholtNOAA Nov 19, 2024
4535696
Fix diag table templates.
christinaholtNOAA Nov 19, 2024
72c330d
3 of 6 complete.
christinaholtNOAA Nov 21, 2024
9bfc486
Pass 5 of 6
christinaholtNOAA Nov 21, 2024
2466c09
Fundamental tests pass!
christinaholtNOAA Nov 21, 2024
6a7a848
Changes made after self-review.
christinaholtNOAA Nov 21, 2024
973a71f
Update tests for new fcst config structure.
christinaholtNOAA Nov 21, 2024
86ec15f
Fixes needed for updated test config
christinaholtNOAA Nov 21, 2024
5a8a7de
Fix logging message format.
christinaholtNOAA Nov 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
546 changes: 0 additions & 546 deletions parm/FV3.input.yml

This file was deleted.

4 changes: 2 additions & 2 deletions parm/diag_table.FV3_GFS_v15_thompson_mynn_lam3km
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved

#output files
"grid_spec", -1, "months", 1, "days", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table.FV3_GFS_v15p2
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

"grid_spec", -1, "months", 1, "days", "time"
"atmos_4xdaily", 6, "hours", 1, "days", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table.FV3_GFS_v16
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

"grid_spec", -1, "months", 1, "days", "time"
"atmos_4xdaily", 6, "hours", 1, "days", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table.FV3_GFS_v17_p8
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

#output files
"grid_spec", -1, "months", 1, "days", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table.FV3_HRRR
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

"grid_spec", -1, "months", 1, "days", "time"
"atmos_static", -1, "hours", 1, "hours", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table.FV3_RAP
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

"grid_spec", -1, "months", 1, "days", "time"
"atmos_4xdaily", 6, "hours", 1, "days", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table.FV3_RRFS_v1beta
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

"grid_spec", -1, "months", 1, "days", "time"
"atmos_static", -1, "hours", 1, "hours", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table.FV3_WoFS_v0
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

"grid_spec", -1, "months", 1, "days", "time"
"atmos_static", -1, "hours", 1, "hours", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table_aqm.FV3_GFS_v15p2
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

"grid_spec", -1, "months", 1, "days", "time"
#"atmos_4xdaily", 6, "hours", 1, "days", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table_aqm.FV3_GFS_v16
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

"grid_spec", -1, "months", 1, "days", "time"
#"atmos_4xdaily", 6, "hours", 1, "days", "time"
Expand Down
4 changes: 2 additions & 2 deletions parm/diag_table_aqm.FV3_GFS_v17_p8
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{ starttime.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ starttime.strftime("%Y %m %d %H %M %S") }}
{{ cycle.strftime("%Y%m%d.%H") }}Z.{{ cres }}.32bit.non-hydro.regional
{{ cycle.strftime("%Y %m %d %H %M %S") }}

#output files
"grid_spec", -1, "months", 1, "days", "time"
Expand Down
207 changes: 21 additions & 186 deletions parm/fixed_files_mapping.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,200 +2,35 @@ fixed_files:
#
#-----------------------------------------------------------------------
#
# FV3_NML_VARNAME_TO_SFC_CLIMO_FIELD_MAPPING:
# This array is used to set some of the namelist variables in the forecast
# model's namelist file that represent the relative or absolute paths of
# various fixed files (the first column of the array, where columns are
# delineated by the pipe symbol "|") to the full paths to surface climatology
# files (on the native FV3-LAM grid) in the FIXlam directory derived from
# the corresponding surface climatology fields (the second column of the
# array).
# These are the names of the fields generated by the make_sfc_climo
# task.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was a lot of shell-based stuff that didn't work with "flattening" the YAML in the run scripts. The relevant information has been moved to the FV3 YAML.

Two sections remain here. The first describes the list of variables that come out of make_sfc_climo, which the UW Driver could absorb once the output method is instantiated on that class. The second describes the set of required fix files the Thompson mp scheme needs. Perhaps those belong over in ccpp_suites_defaults.yaml? I'll take another peek at that.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those Thompson mp input files eventually involved in a uw fs copy or link action? Mostly just curious how the list is used w.r.t. uwtools.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. They do eventually get added to the files_to_link section of the fv3 UW YAML. Instead of listing the names of CCPP Suites that use Thompson MP, though, SRW calls a function that loads the XML to find out if Thompson is used. I could definitely clean up this section and move it over to CCPP suites to put it closer to its other relevant settings.

#
#-----------------------------------------------------------------------
#
FV3_NML_VARNAME_TO_SFC_CLIMO_FIELD_MAPPING: [
"FNALBC | snowfree_albedo",
"FNALBC2 | facsf",
"FNTG3C | substrate_temperature",
"FNVEGC | vegetation_greenness",
"FNVETC | vegetation_type",
"FNSOTC | soil_type",
"FNVMNC | vegetation_greenness",
"FNVMXC | vegetation_greenness",
"FNSLPC | slope_type",
"FNABSC | maximum_snow_albedo"
]

#
#-----------------------------------------------------------------------
#
# Set the array parameter containing the names of all the fields that the
# TN_MAKE_SFC_CLIMO task generates on the native FV3-LAM grid.
#
#-----------------------------------------------------------------------
#
SFC_CLIMO_FIELDS: [
"facsf",
"maximum_snow_albedo",
"slope_type",
"snowfree_albedo",
"soil_type",
"substrate_temperature",
"vegetation_greenness",
"vegetation_type",
]

#
#-----------------------------------------------------------------------
#
# FNGLAC, ..., FNMSKH:
# Names of (some of the) global data files that are assumed to exist in
# a system directory specified (this directory is machine-dependent;
# the experiment generation scripts will set it and store it in the
# variable FIXgsm). These file names also appear directly in the forecast
# model's input namelist file.
#
#-----------------------------------------------------------------------
#
FNGLAC: &FNGLAC "global_glacier.2x2.grb"
FNMXIC: &FNMXIC "global_maxice.2x2.grb"
FNTSFC: &FNTSFC "RTGSST.1982.2012.monthly.clim.grb"
FNSNOC: &FNSNOC "global_snoclim.1.875.grb"
FNZORC: &FNZORC "igbp"
FNAISC: &FNAISC "CFSR.SEAICE.1982.2012.monthly.clim.grb"
FNSMCC: &FNSMCC "global_soilmgldas.t126.384.190.grb"
FNMSKH: &FNMSKH "seaice_newland.grb"
#
#-----------------------------------------------------------------------
#
# FIXgsm_FILES_TO_COPY_TO_FIXam:
# If not running in NCO mode, this array contains the names of the files
# to copy from the FIXgsm system directory to the FIXam directory under
# the experiment directory. Note that the last element has a dummy value.
# This last element will get reset by the workflow generation scripts to
# the name of the ozone production/loss file to copy from FIXgsm. The
# name of this file depends on the ozone parameterization being used,
# and that in turn depends on the CCPP physics suite specified for the
# experiment. Thus, the CCPP physics suite XML must first be read in to
# determine the ozone parameterizaton and then the name of the ozone
# production/loss file. These steps are carried out elsewhere (in one
# of the workflow generation scripts/functions).
#
#-----------------------------------------------------------------------
#
FIXgsm_FILES_TO_COPY_TO_FIXam: [
*FNGLAC,
*FNMXIC,
*FNTSFC,
*FNSNOC,
*FNAISC,
*FNSMCC,
*FNMSKH,
"global_climaeropac_global.txt",
"fix_co2_proj/global_co2historicaldata_2010.txt",
"fix_co2_proj/global_co2historicaldata_2011.txt",
"fix_co2_proj/global_co2historicaldata_2012.txt",
"fix_co2_proj/global_co2historicaldata_2013.txt",
"fix_co2_proj/global_co2historicaldata_2014.txt",
"fix_co2_proj/global_co2historicaldata_2015.txt",
"fix_co2_proj/global_co2historicaldata_2016.txt",
"fix_co2_proj/global_co2historicaldata_2017.txt",
"fix_co2_proj/global_co2historicaldata_2018.txt",
"fix_co2_proj/global_co2historicaldata_2019.txt",
"fix_co2_proj/global_co2historicaldata_2020.txt",
"fix_co2_proj/global_co2historicaldata_2021.txt",
"global_co2historicaldata_glob.txt",
"co2monthlycyc.txt",
"global_h2o_pltc.f77",
"global_hyblev.l65.txt",
"global_zorclim.1x1.grb",
"global_sfc_emissivity_idx.txt",
"global_tg3clim.2.6x1.5.grb",
"global_solarconstant_noaa_an.txt",
"global_albedo4.1x1.grb",
"geo_em.d01.lat-lon.2.5m.HGT_M.nc",
"HGT.Beljaars_filtered.lat-lon.30s_res.nc",
"replace_with_FIXgsm_ozone_prodloss_filename"
]
#
#-----------------------------------------------------------------------
#
# FV3_NML_VARNAME_TO_FIXam_FILES_MAPPING:
# This array is used to set some of the namelist variables in the forecast
# model's namelist file that represent the relative or absolute paths of
# various fixed files (the first column of the array, where columns are
# delineated by the pipe symbol "|") to the full paths to these files in
# the FIXam directory derived from the corresponding workflow variables
# containing file names (the second column of the array).
#
#-----------------------------------------------------------------------
#
FV3_NML_VARNAME_TO_FIXam_FILES_MAPPING: [
!join_str ["FNGLAC | ",*FNGLAC],
!join_str ["FNMXIC | ",*FNMXIC],
!join_str ["FNTSFC | ",*FNTSFC],
!join_str ["FNSNOC | ",*FNSNOC],
!join_str ["FNAISC | ",*FNAISC],
!join_str ["FNSMCC | ",*FNSMCC],
!join_str ["FNMSKH | ",*FNMSKH]
]

#
#-----------------------------------------------------------------------
#
# CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING:
# This array specifies the mapping to use between the symlinks that need
# to be created in each cycle directory (these are the "files" that FV3
# looks for) and their targets in the FIXam directory. The first column
# of the array specifies the symlink to be created, and the second column
# specifies its target file in FIXam (where columns are delineated by the
# pipe symbol "|").
#
#-----------------------------------------------------------------------
#
CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING: [
"aerosol.dat | global_climaeropac_global.txt",
"co2historicaldata_2010.txt | fix_co2_proj/global_co2historicaldata_2010.txt",
"co2historicaldata_2011.txt | fix_co2_proj/global_co2historicaldata_2011.txt",
"co2historicaldata_2012.txt | fix_co2_proj/global_co2historicaldata_2012.txt",
"co2historicaldata_2013.txt | fix_co2_proj/global_co2historicaldata_2013.txt",
"co2historicaldata_2014.txt | fix_co2_proj/global_co2historicaldata_2014.txt",
"co2historicaldata_2015.txt | fix_co2_proj/global_co2historicaldata_2015.txt",
"co2historicaldata_2016.txt | fix_co2_proj/global_co2historicaldata_2016.txt",
"co2historicaldata_2017.txt | fix_co2_proj/global_co2historicaldata_2017.txt",
"co2historicaldata_2018.txt | fix_co2_proj/global_co2historicaldata_2018.txt",
"co2historicaldata_2019.txt | fix_co2_proj/global_co2historicaldata_2019.txt",
"co2historicaldata_2020.txt | fix_co2_proj/global_co2historicaldata_2020.txt",
"co2historicaldata_2021.txt | fix_co2_proj/global_co2historicaldata_2021.txt",
"co2historicaldata_glob.txt | global_co2historicaldata_glob.txt",
"co2monthlycyc.txt | co2monthlycyc.txt",
"global_h2oprdlos.f77 | global_h2o_pltc.f77",
"global_albedo4.1x1.grb | global_albedo4.1x1.grb",
"global_zorclim.1x1.grb | global_zorclim.1x1.grb",
"global_tg3clim.2.6x1.5.grb | global_tg3clim.2.6x1.5.grb",
"sfc_emissivity_idx.txt | global_sfc_emissivity_idx.txt",
"solarconstant_noaa_an.txt | global_solarconstant_noaa_an.txt",
"global_o3prdlos.f77 | ozprdlos_2015_new_sbuvO3_tclm15_nuchem.f77"
]
SFC_CLIMO_FIELDS:
- facsf
- maximum_snow_albedo
- slope_type
- snowfree_albedo
- soil_type
- substrate_temperature
- vegetation_greenness
- vegetation_type

#
#-----------------------------------------------------------------------
#
# THOMPSON_FIX_FILES
# Thompson microphysics requires additional fix files at the run_fcst step,
# this array is appended to FIXgsm_FILES_TO_COPY_TO_FIXam at the workflow
# generation step. Since all these file names are the same when linked,
# appending CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING is handled in the
# generate_FV3LAM_wflow.py script.
# Thompson microphysics requires additional fix files at the run_fcst
# step. Files named here are added to the fv3.files_to_link section at
# configuration time. They can be found in the FIXam directory.
#
#-----------------------------------------------------------------------
#
THOMPSON_FIX_FILES: [
"CCN_ACTIVATE.BIN",
"freezeH2O.dat",
"qr_acr_qg.dat",
"qr_acr_qs.dat",
"qr_acr_qgV2.dat",
"qr_acr_qsV2.dat",
]

THOMPSON_FIX_FILES:
- CCN_ACTIVATE.BIN
- freezeH2O.dat
- qr_acr_qg.dat
- qr_acr_qs.dat
- qr_acr_qgV2.dat
- qr_acr_qsV2.dat
9 changes: 0 additions & 9 deletions parm/input.nml.FV3
Original file line number Diff line number Diff line change
Expand Up @@ -197,15 +197,6 @@
interp_method = 'conserve_great_circle'
/

&nam_sfcperts
/

&nam_sppperts
/

&nam_stochy
/

christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
&namsfc
fabsl = 99999
faisl = 99999
Expand Down
Loading