UW Tools Test Cases

[j-derrico]

UW Tools Test Cases

External Testing Instructions

For each line in the table, the External Tester listed should:

1. Test the release to confirm the changes work as expected.
2. ON THE ISSUE ITSELF: Document their testing with a comment using the template provided below.
3. ON THIS DISCUSSION: Add a comment linking to their issue comment created above.

Copy/paste the following template into your ISSUE COMMENT:

<h3>Release Acceptance Testing Summary</h3>

Date:
Platform/Location:
Status (PASS/FAIL):
Notes:

The External Tester is set to NA for any issues that do not require external acceptance testing.

atparse_to_jinja2

Description	Tester	Platform	Status (OPEN/PASS/FAIL)	Link to Acceptance Testing Comment
Replaces all instances of @[] in each file type included in weather model tests/parm directory	VH	Hera	PASS
(cont)	VH	Mac	PASS

set_config

Description	Tester	Platform	Status (OPEN/PASS/FAIL)	Link to Acceptance Testing Comment
Load the default config file (yaml) from SRW and fill in all references that are available	VH	Hera	FAIL	UW-306
(cont)	VH	Mac	FAIL
Load a namelist and change multiple values given a YAML file	VH	Hera	PASS
(cont)	VH	Mac	PASS
Load a bash configure file and change multiple values given a YAML	VH	Hera	PASS
(cont)	VH	Mac	PASS
Load a namelist and change values given a bash configure file	VH	Hera	PASS
(cont)	VH	Mac	PASS
Compare two namelists	VH	Hera	PASS
(cont)	VH	Mac	PASS
Compare two YAML files	VH	Hera	PASS
(cont)	VH	Mac	PASS
Compare two bash configure files	VH	Hera	PASS
(cont)	VH	Mac	PASS

Templater

Description	Tester	Platform	Status (OPEN/PASS/FAIL)	Link to Acceptance Testing Comment
Fill in a template from a YAML file	VH	Hera/Mac	PASS
Fill in a template from an INI file	VH	Hera/Mac	FAIL
Have the tool provide the required fields for any Jinja2 template	VH	Hera/Mac	PASS
Source a bash configure file and fill in a template (choose any from the weather model)	VH	Hera/Mac	FAIL	See notes in comment
Try to fill in a template without providing enough information (observe failure)	VH	Hera/Mac	Pass	See notes in comments

[venitahagerty]

Link to comment on testing Templater on Unix:
UW-239 comment

[venitahagerty]

Link to comment on testing Templater on Mac:
UW-237 comment

[venitahagerty]

Release Acceptance Testing Summary - copied from Jira, was a table, reformatting

Date: 3/22/23
Platform/Location: Unix/Hera
Status (PASS/FAIL): Some of each
Notes:

Description Tester. Platform Status (OPEN/PASS/FAIL)
Fill in a template from a YAML file VH Unix/Hera PASS

Fill in a template from an INI file VH Unix/Hera FAIL - issue UW-273

Have the tool provide the required fields for any Jinja2 template VH Unix/Hera PASS

Source a bash configure file and fill in a template (choose any from the weather model) VH Unix/Hera

• When environment vars were set in a bash shell and sourced (echo $var worked afterwards) - FAIL - issue UW-274

• When environment key-value pairs were passed in on the command line - PASS. However, not a lot of error checking seemed to be done (i.e. if user accidentally put spaces around equal signs, results were weird), and the example in the User's Guide has commas between the variables, which incorrectly puts the commas into the values.

Try to fill in a template without providing enough information (observe failure) VH Unix/Hera

• If insufficient variables are filled in - PASS. Will just leave blanks, with no errors, though that could be a problem for whatever program receives the file.

• If a random word is put on the command line (i.e. key without value) - FAIL. Gets a ValueError

It might be good to do more validation of the input and the output.

Also, I can put a config file in the directory with the programs without passing a path, but the input jinja template file must have the path specified even if it's in the same directory. Otherwise it gets a TemplateNotFound error.

[venitahagerty]

Release Acceptance Testing on Mac

Date: 3/22/23
Platform/Location: Mac
Status (PASS/FAIL): Some of each

For these tests, I kept the sample files in a separate directory. All results were still the same as the tests on Unix.

[venitahagerty]

Release Acceptance Testing of set_config.py on Unix/Hera
Date: 3/28/23

Load a namelist and change values given a bash/ini configure file.
5 tests done on Hera in /scratch1/BMC/zrtrr/uw with branch UW-273 with the base a subset of a SRW config file

1. Testing with a base namelist and ini config file with matching headers. No error. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_nml.nml -c ../samples/update_ini_match_hdr.ini -o ../samples/output_nml_ini_match_hdr.nml

../samples/list_nml.nml
&atmos_model_nml
blocksize = 28
chksum_debug = .false.
dycore_only = .false.
avg_max_length = 3600.
ccpp_suite = 'FV3_GFS_v17_coupled_p8'
/

../samples/update_ini_match_hdr.ini
[atmos_model_nml]
chksum_debug = .true.
avg_max_length = 7200.

../samples/output_nml_ini_match_hdr.nml
&atmos_model_nml
blocksize = 28
chksum_debug = '.true.'
dycore_only = .false.
avg_max_length = '7200.'
ccpp_suite = 'FV3_GFS_v17_coupled_p8'
/

2. Notice what happens with the output with the same base namelist file, but a non-matching section name in the ini file. No error. Is this expected behavior?

python src/uwtools/set_config.py -i ../samples/list_nml.nml -c ../samples/update_ini_random_hdr.ini -o ../samples/output_nml_ini_nomatch_hdr.nml

cat ../samples/update_ini_random_hdr.ini
[sec_hdr]
chksum_debug = .true.
avg_max_length = 7200.

cat ../samples/output_nml_ini_nomatch_hdr.nml
&atmos_model_nml
blocksize = 28
chksum_debug = .false.
dycore_only = .false.
avg_max_length = 3600.0
ccpp_suite = 'FV3_GFS_v17_coupled_p8'
/

&sec_hdr
chksum_debug = '.true.'
avg_max_length = '7200.'
/

3. If you run with the same base nml file and without a section name in the ini file, you get an error but still get an output file (though unusual). FAIL?

python src/uwtools/set_config.py -i ../samples/list_nml.nml -c ../samples/update_ini_nohdr.ini -o ../samples/output_nml_ini_nohdr.nml

Traceback (most recent call last):
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/set_config.py", line 179, in
create_config_obj(sys.argv[1:])
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/set_config.py", line 165, in create_config_obj
out_object.dump_file(user_args.outfile)
File "/home/role.rtrr/.local/lib/python3.9/site-packages/uwtools/config.py", line 435, in dump_file
f90nml.Namelist(nml).write(file_name, sort=False)
File "/scratch1/NCEPDEV/nems/role.epic/miniconda3/4.12.0/envs/workflow_tools/lib/python3.9/site-packages/f90nml/namelist.py", line 584, in write
self._writestream(nml_file, sort)
File "/scratch1/NCEPDEV/nems/role.epic/miniconda3/4.12.0/envs/workflow_tools/lib/python3.9/site-packages/f90nml/namelist.py", line 683, in _writestream
self._write_nmlgrp(grp_name, grp_vars, nml_file, sort)
File "/scratch1/NCEPDEV/nems/role.epic/miniconda3/4.12.0/envs/workflow_tools/lib/python3.9/site-packages/f90nml/namelist.py", line 699, in _write_nmlgrp
for v_name, v_val in grp_vars.items():
AttributeError: 'str' object has no attribute 'items'

cat ../samples/update_ini_nohdr.ini
chksum_debug = .true.
avg_max_length = 7200.

cat ../samples/output_nml_ini_nohdr.nml
&atmos_model_nml
blocksize = 28
chksum_debug = .false.
dycore_only = .false.
avg_max_length = 3600.0
ccpp_suite = 'FV3_GFS_v17_coupled_p8'
/

&chksum_debug

4. If you take the section name out of the nml file, too, you still get an error, and less output. FAIL?

python src/uwtools/set_config.py -i ../samples/list_nml_nohdr.nml -c ../samples/update_ini_nohdr.ini -o ../samples/output_nml_ini_nohdr2.nml

Traceback (most recent call last):
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/set_config.py", line 179, in
create_config_obj(sys.argv[1:])
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/set_config.py", line 165, in create_config_obj
out_object.dump_file(user_args.outfile)
File "/home/role.rtrr/.local/lib/python3.9/site-packages/uwtools/config.py", line 435, in dump_file
f90nml.Namelist(nml).write(file_name, sort=False)
File "/scratch1/NCEPDEV/nems/role.epic/miniconda3/4.12.0/envs/workflow_tools/lib/python3.9/site-packages/f90nml/namelist.py", line 584, in write
self._writestream(nml_file, sort)
File "/scratch1/NCEPDEV/nems/role.epic/miniconda3/4.12.0/envs/workflow_tools/lib/python3.9/site-packages/f90nml/namelist.py", line 683, in _writestream
self._write_nmlgrp(grp_name, grp_vars, nml_file, sort)
File "/scratch1/NCEPDEV/nems/role.epic/miniconda3/4.12.0/envs/workflow_tools/lib/python3.9/site-packages/f90nml/namelist.py", line 699, in _write_nmlgrp
for v_name, v_val in grp_vars.items():
AttributeError: 'str' object has no attribute 'items'

cat ../samples/list_nml_nohdr.nml
blocksize = 28
chksum_debug = .false.
dycore_only = .false.
avg_max_length = 3600.
ccpp_suite = 'FV3_GFS_v17_coupled_p8'

cat ../samples/output_nml_ini_nohdr2.nml
&chksum_debug

5. If you run with the base config with no header, and the ini config with the random header, no error, and only values from the ini file in the output. Is this expected behavior?

python src/uwtools/set_config.py -i ../samples/list_nml_nohdr.nml -c ../samples/update_ini_random_hdr.ini -o ../samples/output_nml_ini_nohdr3.nml

cat ../samples/output_nml_ini_nohdr3.nml
&sec_hdr
chksum_debug = '.true.'
avg_max_length = '7200.'
/

[christinaholtNOAA]

@venitahagerty Thanks for running these tests! A bit of feedback on those:

1. YAY!
2. I think that is probably the acceptable behavior. We do want to be able to add a section or key that may not exist. However, I'm curious if it deserves some warning that it's doing that.
3. We likely want to do that write in a try block and cleanup the poorly written file. There may be a chance that checks added to PR UW-229: Add conversion flag to config_parser. #197 will fix this issue, though. It will check the depth of the input "dicts" to make sure that they are consistent with what is required of the output dict.
4. Namelists require a section, key/value structure, so this isn't surprising. I think UW-229: Add conversion flag to config_parser. #197 should also help with this one.
5. That one is less expected. Again, I'm thinking comparing the depths of the two input dicts will catch this case in a more manageable way.

[venitahagerty]

After UW-229, number 3 is still the same.

Number 4 gets a different error, and no output:

python src/uwtools/set_config.py -i ../samples/list_nml_nohdr.nml -c ../samples/update_ini_nohdr.ini -o ../samples/output_nml_ini_nohdr2.nml
Traceback (most recent call last):
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/set_config.py", line 212, in
create_config_obj(sys.argv[1:])
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/set_config.py", line 129, in create_config_obj
input_depth = config_obj.dictionary_depth(config_obj.data)
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/config.py", line 351, in dictionary_depth
return 1 + (max(map(self.dictionary_depth, config_dict.values())))
ValueError: max() arg is an empty sequence

Number 5 gets an error now, and no output:

python src/uwtools/set_config.py -i ../samples/list_nml_nohdr.nml -c ../samples/update_ini_random_hdr.ini -o ../samples/output_nml_ini_nohdr3.nml
Traceback (most recent call last):
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/set_config.py", line 212, in
create_config_obj(sys.argv[1:])
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/set_config.py", line 129, in create_config_obj
input_depth = config_obj.dictionary_depth(config_obj.data)
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/config.py", line 351, in dictionary_depth
return 1 + (max(map(self.dictionary_depth, config_dict.values())))
ValueError: max() arg is an empty sequence

[venitahagerty]

April 24 - did same tests on Mac with develop branch from Friday April 21

All results the same. Users must include headers.

[venitahagerty]

Release Acceptance Testing of set_config.py on Unix/Hera
Date: 3/29/23
Load a namelist and change multiple values given a YAML file
3 tests done on Hera in /scratch1/BMC/zrtrr/uw with branch develop (after UW-273 merge) with the base a subset of a SRW config file

I'm not going to go into as much detail, because I'm sensing a theme here.

1. No section header in yaml config file - error
2. Section headers match between base and config files - updates as expected
3. Section headers don't match? Get both sections separately in output

python src/uwtools/set_config.py -i ../samples/list_nml.nml -c ../samples/update_yaml_match_hdr.yaml -o ../samples/output_nml_yaml_match_hdr.nml

cat ../samples/update_yaml_match_hdr.yaml
atmos_model_nml:
chksum_debug: .true.
avg_max_length: 7200.

cat ../samples/update_yaml_random_hdr.yaml
sec_hdr:
chksum_debug: .true.
avg_max_length: 7200.

The other 2 things, and I don't know whether they matter, are that:

1. The value of chksum_debug in the base and config files had no quotes, but does in the output
2. The base file has a space before the section header, and the following lines have 5 spaces to start.
   The output file has one less space before each line. That means you have to do a diff -w to compare those files.

[christinaholtNOAA]

I think the behavior you outlined in the top section is all on point and acceptable.

We should likely add the piece that SRW has in its implementation that we don't here that guesses at the appropriate data type. Let's open a GitHub issue for that.

TIL diff has a -w flag!!! Very nice.

I don't think the whitespace will matter functionally. I think we could leave that one alone until we get to implementing these tools in SRW to make sure...or when we hear from Ben that the whitespace does count. 😬 Let's leave this on as-is for now. No follow up needed until we know it matters.

[venitahagerty]

April 24 - did same tests on Mac with develop branch from Friday April 21

All results the same. Users must include headers.

[venitahagerty]

I think configparser requires that an INI file have 1 or more section headers

[venitahagerty]

Release Acceptance Testing of set_config.py on Unix/Hera
Date: 4/14/2023
Load a bash configure file and change multiple values given a YAML

4 tests done with an ini file having the same header and key-value pairs as list_nml.nml, and using same config yaml files. Also tested with a namelist extension on output file.

1. Section headers match. Input is ini, config is yaml, output has ini extension. Expected changes made. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_ini.ini -c ../samples/update_yaml_match_hdr.yaml -o ../samples/output_ini_yaml_match_hdr.ini

cat ../samples/list_ini.ini
[atmos_model_nml]
blocksize = 28
chksum_debug = .false.
dycore_only = .false.
avg_max_length = 3600.
ccpp_suite = 'FV3_GFS_v17_coupled_p8'

cat ../samples/output_ini_yaml_match_hdr.ini
[atmos_model_nml]
blocksize = 28
chksum_debug = .true.
dycore_only = .false.
avg_max_length = 7200.0
ccpp_suite = 'FV3_GFS_v17_coupled_p8'

2. Section headers don't match. Input is ini, config is yaml, output has ini extension. Output has 2 sections, as expected. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_ini.ini -c ../samples/update_yaml_random_hdr.yaml -o ../samples/output_ini_yaml_random_hdr.ini

cat ../samples/output_ini_yaml_random_hdr.ini
[atmos_model_nml]
blocksize = 28
chksum_debug = .false.
dycore_only = .false.
avg_max_length = 3600.
ccpp_suite = 'FV3_GFS_v17_coupled_p8'

[sec_hdr]
chksum_debug = .true.
avg_max_length = 7200.0

3. Section headers match. Input is ini, config is yaml, output has nml extension. Got a namelist output file. Expected changes made. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_ini.ini -c ../samples/update_yaml_match_hdr.yaml -o ../samples/output_ini_match_hdr.nml

cat ../samples/output_ini_match_hdr.nml
&atmos_model_nml
blocksize = '28'
chksum_debug = '.true.'
dycore_only = '.false.'
avg_max_length = 7200.0
ccpp_suite = "'FV3_GFS_v17_coupled_p8'"
/

4. Section headers don't match. Input is ini, config is yaml, output has nml extension. Got a namelist output file. Output has 2 sections, as expected. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_ini.ini -c ../samples/update_yaml_random_hdr.yaml -o ../samples/output_ini_random_hdr.nml

cat ../samples/output_ini_random_hdr.nml
&atmos_model_nml
blocksize = '28'
chksum_debug = '.false.'
dycore_only = '.false.'
avg_max_length = '3600.'
ccpp_suite = "'FV3_GFS_v17_coupled_p8'"
/

&sec_hdr
chksum_debug = '.true.'
avg_max_length = 7200.0
/

[venitahagerty]

April 24 - did same tests on Mac with develop branch from Friday April 21

All results the same.

[venitahagerty]

Release Acceptance Testing of set_config.py on Unix/Hera
Date: 4/14/2023
Compare two namelists

1. Two namelist files with the same entries, with 2 having different values. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_nml.nml -c ../samples/output_ini_match_hdr.nml --compare

- /scratch1/BMC/zrtrr/uw/samples/list_nml.nml
+ /scratch1/BMC/zrtrr/uw/samples/output_ini_match_hdr.nml

atmos_model_nml: blocksize: - 28 + 28
atmos_model_nml: chksum_debug: - False + .true.
atmos_model_nml: dycore_only: - False + .false.
atmos_model_nml: avg_max_length: - 3600.0 + 7200.0
atmos_model_nml: ccpp_suite: - FV3_GFS_v17_coupled_p8 + 'FV3_GFS_v17_coupled_p8'

cat ../samples/list_nml.nml
&atmos_model_nml
blocksize = 28
chksum_debug = .false.
dycore_only = .false.
avg_max_length = 3600.
ccpp_suite = 'FV3_GFS_v17_coupled_p8'
/

cat ../samples/output_ini_match_hdr.nml
&atmos_model_nml
blocksize = '28'
chksum_debug = '.true.'
dycore_only = '.false.'
avg_max_length = 7200.0
ccpp_suite = "'FV3_GFS_v17_coupled_p8'"
/

2. Two namelist files with the same entries in one section, and one having an additional section. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_nml.nml -c ../samples/output_ini_random_hdr.nml --compare

- /scratch1/BMC/zrtrr/uw/samples/list_nml.nml
+ /scratch1/BMC/zrtrr/uw/samples/output_ini_random_hdr.nml

atmos_model_nml: blocksize: - 28 + 28
atmos_model_nml: chksum_debug: - False + .false.
atmos_model_nml: dycore_only: - False + .false.
atmos_model_nml: avg_max_length: - 3600.0 + 3600.
atmos_model_nml: ccpp_suite: - FV3_GFS_v17_coupled_p8 + 'FV3_GFS_v17_coupled_p8'
sec_hdr: chksum_debug: - None + .true.
sec_hdr: avg_max_length: - None + 7200.0

cat ../samples/output_ini_random_hdr.nml
&atmos_model_nml
blocksize = '28'
chksum_debug = '.false.'
dycore_only = '.false.'
avg_max_length = '3600.'
ccpp_suite = "'FV3_GFS_v17_coupled_p8'"
/

&sec_hdr
chksum_debug = '.true.'
avg_max_length = 7200.0
/

[venitahagerty]

April 24 - did same tests on Mac with develop branch from Friday April 21

All results the same.

[venitahagerty]

Release Acceptance Testing of set_config.py on Unix/Hera
Date: 4/14/2023
Compare two YAML files

1. Two YAML files, with the same header. Two key-value pairs are reversed between the two files. If only differences should be printed: SUCCESS

python src/uwtools/set_config.py -i ../samples/update_yaml_match_hdr.yaml -c ../samples/update_yaml_match_rev_hdr.yaml --compare

- /scratch1/BMC/zrtrr/uw/samples/update_yaml_match_hdr.yaml
+ /scratch1/BMC/zrtrr/uw/samples/update_yaml_match_rev_hdr.yaml

atmos_model_nml: blocksize: - None + 76

cat ../samples/update_yaml_match_hdr.yaml
atmos_model_nml:
chksum_debug: .true.
avg_max_length: 7200.

cat ../samples/update_yaml_match_rev_hdr.yaml
atmos_model_nml:
avg_max_length: 7200.
chksum_debug: .true.
blocksize: 76

2. Two YAML files, with different headers. Same key-value pairs, in same order. Output as expected. SUCCESS

python src/uwtools/set_config.py -i ../samples/update_yaml_match_hdr.yaml -c ../samples/update_yaml_random_hdr.yaml --compare

- /scratch1/BMC/zrtrr/uw/samples/update_yaml_match_hdr.yaml
+ /scratch1/BMC/zrtrr/uw/samples/update_yaml_random_hdr.yaml

atmos_model_nml: chksum_debug: - .true. + None
atmos_model_nml: avg_max_length: - 7200.0 + None
sec_hdr: chksum_debug: - None + .true.
sec_hdr: avg_max_length: - None + 7200.0

cat ../samples/update_yaml_random_hdr.yaml
sec_hdr:
chksum_debug: .true.
avg_max_length: 7200.

[venitahagerty]

April 24 - did same tests on Mac with develop branch from Friday April 21

All results the same.

[venitahagerty]

Release Acceptance Testing of set_config.py on Unix/Hera
Date: 4/17/2023
Compare two bash configure files

1. Two bash (ini) files, with the same header. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_ini.ini -c ../samples/update_ini_match_hdr.ini --compare

- /scratch1/BMC/zrtrr/uw/samples/list_ini.ini
+ /scratch1/BMC/zrtrr/uw/samples/update_ini_match_hdr.ini

atmos_model_nml: blocksize: - 28 + None
atmos_model_nml: chksum_debug: - .false. + .true.
atmos_model_nml: dycore_only: - .false. + None
atmos_model_nml: avg_max_length: - 3600. + 7200.
atmos_model_nml: ccpp_suite: - 'FV3_GFS_v17_coupled_p8' + None

cat ../samples/list_ini.ini
[atmos_model_nml]
blocksize = 28
chksum_debug = .false.
dycore_only = .false.
avg_max_length = 3600.
ccpp_suite = 'FV3_GFS_v17_coupled_p8'

cat ../samples/update_ini_match_hdr.ini
[atmos_model_nml]
chksum_debug = .true.
avg_max_length = 7200.

2. Two bash (ini) files, with different headers. SUCCESS

python src/uwtools/set_config.py -i ../samples/list_ini.ini -c ../samples/update_ini_random_hdr.ini --compare

- /scratch1/BMC/zrtrr/uw/samples/list_ini.ini
+ /scratch1/BMC/zrtrr/uw/samples/update_ini_random_hdr.ini

atmos_model_nml: blocksize: - 28 + None
atmos_model_nml: chksum_debug: - .false. + None
atmos_model_nml: dycore_only: - .false. + None
atmos_model_nml: avg_max_length: - 3600. + None
atmos_model_nml: ccpp_suite: - 'FV3_GFS_v17_coupled_p8' + None
sec_hdr: chksum_debug: - None + .true.
sec_hdr: avg_max_length: - None + 7200.

cat ../samples/update_ini_random_hdr.ini
[sec_hdr]
chksum_debug = .true.
avg_max_length = 7200.

[venitahagerty]

April 24 - did same tests on Mac with develop branch from Friday April 21

All results the same.

[venitahagerty]

Release Acceptance Testing of atparse_to_jinja2.py on Unix/Hera
Date: 4/17/2023
Replaces all instances of @[] in each file type included in weather model tests/parm directory

Jumping back to atparse_to_jinja2.py. Read in file with 9 instances of @[var] and replaced all instances with {{var}}. SUCCESS

python src/uwtools/atparse_to_jinja2.py -i ../samples/regional.nml.IN -o ../samples/regional_out_jinja.nml

diff ../samples/regional.nml.IN ../samples/regional_out_jinja.nml
6c6
<      ccpp_suite = '@[CCPP_SUITE]'
---
>      ccpp_suite = '{{CCPP_SUITE}}'
30c30
<        layout = @[INPES],@[JNPES]
---
>        layout = {{INPES}},{{JNPES}}
39c39
<        make_nh = @[MAKE_NH]
---
>        make_nh = {{MAKE_NH}}
64c64
<        na_init = @[NA_INIT]
---
>        na_init = {{NA_INIT}}
76c76
<        external_ic = @[EXTERNAL_IC]
---
>        external_ic = {{EXTERNAL_IC}}
79,80c79,80
<        nggps_ic = @[NGGPS_IC]
<        mountain = @[MOUNTAIN]
---
>        nggps_ic = {{NGGPS_IC}}
>        mountain = {{MOUNTAIN}}
95c95
<        warm_start = @[WARM_START]
---
>        warm_start = {{WARM_START}}
109c109
<        write_restart_with_bcs = @[WRITE_RESTART_WITH_BCS]
---
>        write_restart_with_bcs = {{WRITE_RESTART_WITH_BCS}}

[venitahagerty]

April 24 - did same tests on Mac with develop branch from Friday April 21

All results the same.

[venitahagerty]

Release Acceptance Testing of sset_config.py on Unix/Hera
Date: 4/21/2023
Load the default config file (yaml) from SRW and fill in all references that are available

• Testing is being done in /scratch1/BMC/zrtrr/uw
• Cloned the develop branch this morning.
• Input, config, and out files are in subdirectory: samples
• Using a config_default.yaml file from SRW on Hera.
• Ran values_needed with set_config.py on config_default.yaml to get list of unset values
• Grepped various yaml files in /scratch1/BMC/zrtrr/rrfs_ci/autoci/pr/1264903189/20230412185015/ufs-srweather-app/ush and its subirectory machine to get realistic (though may not be matching) values, put into .samples/update_config_defaults.yaml
• Ran values_needed with updated file to see if all values were set.
• These are all very long files.

python workflow-tools/scripts/set_config.py -i samples/config_defaults.yaml -c samples/update_config_defaults.yaml -o samples/output_config_defaults.yaml

python workflow-tools/scripts/set_config.py -i samples/output_config_defaults.yaml --values_needed > samples/output_updated_config.yaml

First 4 issues:

1. Problems with an unusual looking value:

workflow:
WORKFLOW_ID: !nowtimestamp ''

When run this way, program errors out. Then took out the punctuation, and it set the simplified value.

Traceback (most recent call last):
File "/scratch1/BMC/zrtrr/uw/workflow-tools/scripts/set_config.py", line 212, in
create_config_obj(sys.argv[1:])
File "/scratch1/BMC/zrtrr/uw/workflow-tools/scripts/set_config.py", line 124, in create_config_obj
user_config_obj = getattr(config,
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/config.py", line 380, in init
self.update(self._load())
File "/scratch1/BMC/zrtrr/uw/workflow-tools/src/uwtools/config.py", line 394, in _load
cfg = yaml.load(file_name, Loader=loader)
File "/home/role.rtrr/.local/lib/python3.9/site-packages/yaml/init.py", line 81, in load
return loader.get_single_data()
File "/home/role.rtrr/.local/lib/python3.9/site-packages/yaml/constructor.py", line 51, in get_single_data
return self.construct_document(node)
File "/home/role.rtrr/.local/lib/python3.9/site-packages/yaml/constructor.py", line 60, in construct_document
for dummy in generator:
File "/home/role.rtrr/.local/lib/python3.9/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
value = self.construct_mapping(node)
File "/home/role.rtrr/.local/lib/python3.9/site-packages/yaml/constructor.py", line 218, in construct_mapping
return super().construct_mapping(node, deep=deep)
File "/home/role.rtrr/.local/lib/python3.9/site-packages/yaml/constructor.py", line 143, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/home/role.rtrr/.local/lib/python3.9/site-packages/yaml/constructor.py", line 100, in construct_object
data = constructor(self, node)
File "/home/role.rtrr/.local/lib/python3.9/site-packages/yaml/constructor.py", line 427, in construct_undefined
raise ConstructorError(None, None,
yaml.constructor.ConstructorError: could not determine a constructor for the tag '!nowtimestamp'
in "/scratch1/BMC/zrtrr/uw/samples/update_config_defaults.yaml", line 45, column 16

2-4. Problems with three longer values that have single ticks, double ticks, percents, and curly braces. Set to blank, shown as not set by values_needed on updated file

platform:
TEST_VX_FCST_INPUT_BASEDIR: '{{ "/scratch1/NCEPDEV/nems/role.epic/UFS_SRW_data/develop/output_data/fcst_" }}{{ "ens" if (global.NUM_ENS_MEMBERS > 0) else "det" }}{{ "/{{workflow.PREDEF_GRID_NAME}}/${CDATE}/postprd" }}{% raw %}{% endraw %}'

global:
ENSMEM_NAMES: '{% for m in range(NUM_ENS_MEMBERS) %}{{ "mem%03d, " % m }}{% endfor %}'
FV3_NML_ENSMEM_FPS: '{% for mem in ENSMEM_NAMES %}{{ [EXPTDIR, "%s_%s" % FV3_NML_FN, mem]|path_join }}{% endfor %}'

[venitahagerty]

April 24 - did same tests on Mac with develop branch from Friday April 21

All results the same.

[christinaholtNOAA]

This is good, I think. I do have questions/coments, though.

1. I know that we will need a way to register some PyYAML constructors. I'm curious if you see any obvious solutions to that. I mean, we can keep the ones needed by SRW in UW (as a stop gap), but it would be nice to have a hook to allow each app register their own when they need it. Maybe that's a development we can push off until implementation time, though. This test case just highlights that the feature is needed. I think SRW has a couple others, defined in the ush/python_utils/config_parser.py file.

2-4.
In the platform example, I'm not seeing the problem leap out. Do you mind leading me to it?

In the global example, I see that there is Jinja2 filter (|path_join) that will cause the same types of problems as the PyYAML constructors. We will eventually want a solution for those.

What other problems am I missing here?

[venitahagerty]

I'm just guessing, but of the many values being set, only 3 contain a percent sign (%) - the 3 that are not getting set.

platform.TEST_VX_FCST_INPUT_BASEDIR
global.ENSMEM_NAMES
global.FV3_NML_ENSMEM_FPS

So I'm guessing that the code is not handling the percent signs correctly/defensively.

I have not worked with PyYAML constructors.