Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce memory required by MERRA2 option; fix diag_tables for P7 tests; adds C384 P7 tests to Cheyenne.intel #866

Closed
wants to merge 107 commits into from

Conversation

DeniseWorthen
Copy link
Collaborator

@DeniseWorthen DeniseWorthen commented Oct 10, 2021

PR Checklist

  • Ths PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.

  • This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR

  • An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
    are specified below.

  • If new or updated input data is required by this PR, it is clearly stated in the text of the PR.

Instructions: All subsequent sections of text should be filled in as appropriate.

The information provided below allows the code managers to understand the changes relevant to this PR, whether those changes are in the ufs-weather-model repository or in a subcomponent repository. Ufs-weather-model code managers will use the information provided to add any applicable labels, assign reviewers and place it in the Commit Queue. Once the PR is in the Commit Queue, it is the PR owner's responsiblity to keep the PR up-to-date with the develop branch of ufs-weather-model.

Description

  • Updates the ingest of MERRA2 data to reduce the memory footprint of the ingested data.
  • Adds MERRA2 fields to the forecast files for both Coupled and Standalone P7 tests. Removes fields which are not also present in diag_table_gfsv16
  • Adds C384 P7 tests (cpld_control_c384_p7 and cpld_bmark_p7) tests to Cheyenne.intel

Issue(s) addressed

Testing

Testing on hera.intel and hera.gnu show that all P7 tests (coupled and standalone) fail comparison only because of the changes to the forecast files associated with the updated diag_table. All common fields in the forecast files are B4B and all restart files are B4B.

For hera.gnu, the failed tests are:

control_debug_p7 029 failed in check_result
cpld_debug_p7 033 failed in check_result
control_p7 010 failed in check_result
cpld_control_p7 032 failed in check_result

For hera.intel, the failed tests are:

control_debug_p7 063 failed in check_result
control_p7 026 failed in check_result
cpld_control_p7 001 failed in check_result
control_decomp_p7 028 failed in check_result
cpld_decomp_p7 004 failed in check_result
cpld_2threads_p7 003 failed in check_result
control_2threads_p7 029 failed in check_result
cpld_debug_p7 011 failed in check_result
cpld_control_wave_p7 005 failed in check_result
cpld_bmark_p7 010 failed in check_result
cpld_control_c384_p7 008 failed in check_result
cpld_control_c192_p7 006 failed in check_result

RegressionTests_hera.gnu.log
RegressionTests_hera.intel.log

Comparing the sfcf006.nc file in the cpld_control_c384_p7 test against the current baseline using the comparison tool cprnc gives the result:

SUMMARY of cprnc:
 A total number of    158 fields were compared
          of which      0 had non-zero differences
               and      0 had differences in fill patterns
               and      0 had different dimension sizes
 A total number of      0 fields could not be analyzed
 A total number of      6 time-varying fields on file 1 were not found on file 2.
 A total number of      0 time-constant fields on file 1 were not found on file 2.
 A total number of      0 time-varying fields on file 2 were not found on file 1.
 A total number of      0 time-constant fields on file 2 were not found on file 1.
  diff_test: the two files DIFFER only in their field lists
  • hera.intel
  • hera.gnu
  • orion.intel
  • cheyenne.intel
  • cheyenne.gnu
  • gaea.intel
  • jet.intel
  • wcoss_cray
  • wcoss_dell_p3
  • CI

Dependencies

DeniseWorthen and others added 30 commits March 27, 2021 12:30
This reverts commit 7b826d4.
* turn cpld_control_c384_p7 and cpld_bmark_p7 tests on cheyenne
* using updated diag_table, confirm that all coupled tests fail
because of changed fields in forecast files; all other compared
files are B4B
* use variable NEW_DIAGTABLE to add merra2 output
Copy link
Collaborator

@climbfuji climbfuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for getting the C384 P7 tests going on Cheyenne!

@DeniseWorthen DeniseWorthen added Baseline Updates Current baselines will be updated. Waiting for Reviews The PR is waiting for reviews from associated component PR's. labels Oct 11, 2021
@junwang-noaa
Copy link
Collaborator

@DeniseWorthen Is the diag_table for cpld_bmark_p7 correct? Since it has merra2 update, I assume the diag_table will contain the 6 AOD fields too.

@DeniseWorthen
Copy link
Collaborator Author

@junwang-noaa All the coupled tests use the same file, diag_table_template. This is identical to diag_table_gfsv16_merra2 but with the addition of the MOM6 output. The standalone P7 tests will now also use diag_table_gfsv16_merra2.

@DeniseWorthen
Copy link
Collaborator Author

Closed. Code will be committed in PR #831

junwang-noaa pushed a commit that referenced this pull request Oct 29, 2021
…ression tests, set ECF_TRIES to 2, update Cheyenne job submission scripts; contains "Reduce memory required by MERRA2 option; fix diag_tables for P7 tests; adds C384 P7 tests to Cheyenne.intel" (#866), contains "Update CMEPS" (#775) (#831)

- Remove unused/broken HAFS-HWRF regression tests that used Ferrier-Aligo MP, and updates the old HAFS-HWRF regression tests to use Noah MP instead of Noah-WRFv4. 
- An update in rt.sh that uses the automatic rerun capability for ecflow
- Code updates to reduce memory required by MERRA2 option; fix diag_tables for P7 tests; adds C384 P7 tests to Cheyenne.intel
- CMEPS update
- Update Cheyenne job submission scripts due to a recent change on Cheyenne

Co-authored-by: denise.worthen <[email protected]>
Co-authored-by: Dusan Jovic <[email protected]>
@DeniseWorthen DeniseWorthen deleted the feature/aero4 branch February 4, 2022 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Baseline Updates Current baselines will be updated. Waiting for Reviews The PR is waiting for reviews from associated component PR's.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

diag_tables for P7 tests do not include merra2 variables C384 P7 coupled tests fail on cheyenne
3 participants