Update Intel/impi version to Intel 2021/2022 #1221

junwang-noaa · 2022-05-16T13:37:48Z

PR Checklist

This PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.
This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR
An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
are specified below.
Results for one or more of the regression tests change and the reasons for the changes are understood and explained below.
New or updated input data is required by this PR. If checked, please work with the code managers to update input data sets on all platforms.

Instructions: All subsequent sections of text should be filled in as appropriate.

The information provided below allows the code managers to understand the changes relevant to this PR, whether those changes are in the ufs-weather-model repository or in a subcomponent repository. Ufs-weather-model code managers will use the information provided to add any applicable labels, assign reviewers and place it in the Commit Queue. Once the PR is in the Commit Queue, it is the PR owner's responsiblity to keep the PR up-to-date with the develop branch of ufs-weather-model.

Description

In this PR, the Intel compiler will be updated to Intel 2021/2022 for hera, orion, jet, gaea, and Cheyenne. The wcoss dell/cray will go down in two months and additional disk space cleanup is required for installing the hpc-stack with the new Intel compiler, so the intel compiler is not updated on these two platforms.

Additional changes are:

The environment variable I_MPI_DAPL_UD is turned off and all the GOCART tests passed.
png library is changed to libpng in ufs_common/ufs_common_debug.

The branch was tested on hera, orion, jet and gaea. Butterfly tests were conducted on orion/hera. The results are summarized in the issue #1113.

Issue(s) addressed

Link the issues to be closed with this PR, whether in this repository, or in another repository.
(Remember, issues must always be created before starting work on a PR branch!)

fixes Update intel/impi compiler to 2022 #1113

Testing

How were these changes tested? What compilers / HPCs was it tested with? Are the changes covered by regression tests? (If not, why? Do new tests need to be added?) Have regression tests and unit tests (utests) been run? On which platforms and with which compilers? (Note that unit tests can only be run on tier-1 platforms)

Dependencies

If testing this branch requires non-default branches in other repositories, list them. Those branches should have matching names (ideally).

Do PRs in upstream repositories need to be merged first?
If so add the "waiting for other repos" label and list the upstream PRs

waiting on noaa-emc/nems/pull/<pr_number>
waiting on noaa-emc/fv3atm/pull/<pr_number>

modulefiles/ufs_common

modulefiles/ufs_gaea.intel

…6.37)

jkbk2004 · 2022-05-16T15:08:57Z

@junwang-noaa I am testing cpld_control_p8 on cheyenne. It seems results changes. Maybe I need butterfly test for validation.

junwang-noaa · 2022-05-16T15:11:58Z

@jkbk2004 Updating Intel/impi version will change the results. But the library updates (g2 (3.4.5), Jasper (2.0.25), PIO (2.5.3), libpng (1.6.37)) will not change results. Please let me know from which libraries that you found result change.

Intel2022: update Cheyenne hpc-stack locations

modulefiles/ufs_cheyenne.intel

on-behalf-of @ufs-community <[email protected]>

BrianCurtis-NOAA · 2022-05-20T15:37:47Z

I don't see hera.gnu and I don't see a label that would have started it after the previous failure. Is this being worked on?

jkbk2004 · 2022-05-20T15:39:34Z

@MinsukJi-NOAA @BrianCurtis-NOAA do you want me to run RT on hera.gnu to confirm? so that we can merge.

jkbk2004 · 2022-05-20T15:40:48Z

@BrianCurtis-NOAA @MinsukJi-NOAA hera.gnu should work since this pr has not much to do with gnu.

BrianCurtis-NOAA · 2022-05-20T16:00:28Z

@MinsukJi-NOAA Would you rather copy over the GNU baselines from the previous date, since i assume they won't change and add hera-gnu-RT, or just add the hera-gnu-BL?

BrianCurtis-NOAA · 2022-05-20T18:08:45Z

Automated RT Failure Notification
Machine: hera
Compiler: gnu
Job: RT
[RT] Repo location: /scratch1/NCEPDEV/nems/emc.nemspara/autort/pr/937515199/20220520180014/ufs-weather-model
Please make changes and add the following label back: hera-gnu-RT

BrianCurtis-NOAA · 2022-05-20T18:11:33Z

Automated RT Failure Notification Machine: hera Compiler: gnu Job: RT [RT] Repo location: /scratch1/NCEPDEV/nems/emc.nemspara/autort/pr/937515199/20220520180014/ufs-weather-model Please make changes and add the following label back: hera-gnu-RT

@MinsukJi-NOAA

'cat: /scratch1/NCEPDEV/nems/emc.nemspara/autort/pr/937515199/20220520180014/ufs-weather-model/tests/log_hera.gnu/compile_*_time.log: No such file or directory', "++ echo 'rt.sh error on line 858'", 'rt.sh error on line 858'

BrianCurtis-NOAA · 2022-05-20T18:14:06Z

Project: stmp1 
		Directory: /scratch2/NCEPDEV/stmp1 DiskInUse=420724 GB, Quota=400000 GB, Files=27611254, FileQUota=80000000
		Directory: /scratch1/NCEPDEV/stmp2 DiskInUse=418850 GB, Quota=400000 GB, Files=40878708, FileQUota=80000000
		Directory: /scratch2/NCEPDEV/stmp3 DiskInUse=420724 GB, Quota=400000 GB, Files=27611254, FileQUota=80000000
		Directory: /scratch1/NCEPDEV/stmp4 DiskInUse=418850 GB, Quota=400000 GB, Files=40878708, FileQUota=80000000

We're full, maybe thats it?

MinsukJi-NOAA · 2022-05-20T18:18:49Z

@jkbk2004 @BrianCurtis-NOAA This is Hera.gnu compile error message.

Lmod has detected the following error: These module(s) or extension(s) exist
but cannot be loaded as requested: "libpng/1.6.37"
   Try: "module spider libpng/1.6.37" to see how to load the module(s).

BrianCurtis-NOAA · 2022-05-20T18:22:51Z

@MinsukJi-NOAA

It loaded fine when i did it manually. I wonder why it had issue? Maybe something temp on Hera?

MinsukJi-NOAA · 2022-05-20T18:34:42Z

@MinsukJi-NOAA

It loaded fine when i did it manually. I wonder why it had issue? Maybe something temp on Hera?

@BrianCurtis-NOAA I am not able to manually load libpng

[Minsuk.Ji@hecflow01 tests]$ module use ../modulefiles/
[Minsuk.Ji@hecflow01 tests]$ pwd
/scratch1/NCEPDEV/stmp4/Minsuk.Ji/PR1221/tests
[Minsuk.Ji@hecflow01 tests]$ module use /scratch1/NCEPDEV/nems/emc.nemspara/soft/modulefiles
[Minsuk.Ji@hecflow01 tests]$ module load miniconda3/3.7.3
[Minsuk.Ji@hecflow01 tests]$ 
[Minsuk.Ji@hecflow01 tests]$ module use /contrib/sutils/modulefiles
[Minsuk.Ji@hecflow01 tests]$ module load sutils
[Minsuk.Ji@hecflow01 tests]$ 
[Minsuk.Ji@hecflow01 tests]$ module load cmake/3.20.1
[Minsuk.Ji@hecflow01 tests]$ 
[Minsuk.Ji@hecflow01 tests]$ module use /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/modulefiles/stack
[Minsuk.Ji@hecflow01 tests]$ 
[Minsuk.Ji@hecflow01 tests]$ module load hpc/1.1.0
[Minsuk.Ji@hecflow01 tests]$ 
[Minsuk.Ji@hecflow01 tests]$ module load hpc-gnu/9.2.0
[Minsuk.Ji@hecflow01 tests]$ module load hpc-mpich/3.3.2
[Minsuk.Ji@hecflow01 tests]$ module load ufs_common
Lmod has detected the following error:  These module(s) or extension(s) exist but cannot be loaded as requested: "libpng/1.6.37"
   Try: "module spider libpng/1.6.37" to see how to load the module(s).

jkbk2004 · 2022-05-20T18:46:54Z

@BrianCurtis-NOAA @MinsukJi-NOAA @junwang-noaa seems like no libpng with gnu there. we have to follow up with nceplibs team.
------------------ /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/modulefiles/compiler/gnu/9.2.0 -------------------
bacio/2.4.1 g2c/1.6.2 grib_util/1.2.4 (D) png/1.6.35 w3nco/2.4.1

jkbk2004 · 2022-05-20T18:51:15Z

@BrianCurtis-NOAA @MinsukJi-NOAA I sent a message on hpc-stack slack and asked Kyle and Hang to install one.

jkbk2004 · 2022-05-20T19:04:12Z

@MinsukJi-NOAA @BrianCurtis-NOAA I checked Hang just installed libpng/1.6.37. Its available on hera now. So nice to see his quick installation.

MinsukJi-NOAA · 2022-05-20T19:06:01Z

@MinsukJi-NOAA @BrianCurtis-NOAA I checked Hang just installed libpng/1.6.37. Its available on hera now. So nice to see his quick installation.

Thanks so much @jkbk2004! hera.gnu RT is being rerun.

on-behalf-of @ufs-community <[email protected]>

BrianCurtis-NOAA · 2022-05-21T03:03:18Z

Automated RT Failure Notification
Machine: gaea
Compiler: intel
Job: BL
[BL] Repo location: /lustre/f2/pdata/ncep/emc.nemspara/autort/pr/937515199/20220520033006/ufs-weather-model
[BL] Baseline creation and move successful
[RT] Repo location: /lustre/f2/pdata/ncep/emc.nemspara/autort/pr/937515199/20220520104410/ufs-weather-model
[RT] Error: Test cpld_restart_c384_p8 011 failed in run_test failed
Please make changes and add the following label back: gaea-intel-BL

junwang-noaa · 2022-05-21T20:40:17Z

@MinsukJi-NOAA @jkbk2004 @BrianCurtis-NOAA Thank you very much for helping running RT.

junwang-noaa added 6 commits March 15, 2022 15:10

update intel compiler to intel2022

ae6d0ff

update to develop branch

052607e

Merge remote-tracking branch 'upstream/develop' into intel2022

cb2ee6f

update libpng

600b784

update intel compiler on ufs_hera.intel_debug

55c7745

Merge remote-tracking branch 'upstream/develop' into intel2022

91ff7b9

junwang-noaa requested a review from DusanJovic-NOAA May 16, 2022 13:40

DusanJovic-NOAA reviewed May 16, 2022

View reviewed changes

modulefiles/ufs_common Outdated Show resolved Hide resolved

DusanJovic-NOAA reviewed May 16, 2022

View reviewed changes

modulefiles/ufs_gaea.intel Outdated Show resolved Hide resolved

junwang-noaa added 2 commits May 16, 2022 13:54

remove I_MPI_DAPL_UD fot Intel 2022/2021

52ab37f

update libraries g2 (3.4.5), Jasper (2.0.25), PIO (2.5.3), libpng (1.…

3dae554

…6.37)

jkbk2004 and others added 5 commits May 16, 2022 16:42

updated gnu/cheyenne hpc-stack location

c10ab6d

Update ufs_cheyenne.gnu_debug

4f506f4

Update ufs_cheyenne.intel

d0c46da

Update ufs_cheyenne.intel_debug

5b25383

Merge pull request #16 from jkbk2004/intel2022

b450910

Intel2022: update Cheyenne hpc-stack locations

DusanJovic-NOAA reviewed May 19, 2022

View reviewed changes

modulefiles/ufs_cheyenne.intel Show resolved Hide resolved

DusanJovic-NOAA approved these changes May 19, 2022

View reviewed changes

Merge remote-tracking branch 'upstream/develop' into intel2022

ccb942e

junwang-noaa requested a review from MinsukJi-NOAA May 20, 2022 02:36

junwang-noaa added Baseline Updates Current baselines will be updated. hera-gnu-BL labels May 20, 2022

BrianCurtis-NOAA added 2 commits May 20, 2022 08:49

[AutoRT] cheyenne.intel Job Completed.

f92aef1

on-behalf-of @ufs-community <[email protected]>

[AutoRT] hera.intel Job Completed.

5f49221

on-behalf-of @ufs-community <[email protected]>

Add wcoss dell RT log: all tests passed

c08b248

MinsukJi-NOAA added the hera-gnu-RT label May 20, 2022

BrianCurtis-NOAA removed the hera-gnu-RT label May 20, 2022

MinsukJi-NOAA added the hera-gnu-RT label May 20, 2022

BrianCurtis-NOAA removed the hera-gnu-RT label May 20, 2022

[AutoRT] hera.gnu Job Completed.

a9a7593

on-behalf-of @ufs-community <[email protected]>

Add gaea RT log: all tests passed

637292a

jkbk2004 approved these changes May 21, 2022

View reviewed changes

junwang-noaa merged commit 2f05121 into ufs-community:develop May 22, 2022

This was referenced May 23, 2022

Update NCEP regression tests to use newer intel NOAA-EMC/WW3#718

Closed

Update wave related settings and code for P8c NOAA-EMC/global-workflow#736

Closed

junwang-noaa deleted the intel2022 branch February 8, 2023 13:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Intel/impi version to Intel 2021/2022 #1221

Update Intel/impi version to Intel 2021/2022 #1221

junwang-noaa commented May 16, 2022 •

edited by MinsukJi-NOAA

Loading

jkbk2004 commented May 16, 2022

junwang-noaa commented May 16, 2022

BrianCurtis-NOAA commented May 20, 2022

jkbk2004 commented May 20, 2022

jkbk2004 commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022

MinsukJi-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022 •

edited

Loading

MinsukJi-NOAA commented May 20, 2022 •

edited

Loading

jkbk2004 commented May 20, 2022

jkbk2004 commented May 20, 2022

jkbk2004 commented May 20, 2022

MinsukJi-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 21, 2022

junwang-noaa commented May 21, 2022

Update Intel/impi version to Intel 2021/2022 #1221

Update Intel/impi version to Intel 2021/2022 #1221

Conversation

junwang-noaa commented May 16, 2022 • edited by MinsukJi-NOAA Loading

PR Checklist

Instructions: All subsequent sections of text should be filled in as appropriate.

Description

Issue(s) addressed

Testing

Dependencies

jkbk2004 commented May 16, 2022

junwang-noaa commented May 16, 2022

BrianCurtis-NOAA commented May 20, 2022

jkbk2004 commented May 20, 2022

jkbk2004 commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022

MinsukJi-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 20, 2022 • edited Loading

MinsukJi-NOAA commented May 20, 2022 • edited Loading

jkbk2004 commented May 20, 2022

jkbk2004 commented May 20, 2022

jkbk2004 commented May 20, 2022

MinsukJi-NOAA commented May 20, 2022

BrianCurtis-NOAA commented May 21, 2022

junwang-noaa commented May 21, 2022

junwang-noaa commented May 16, 2022 •

edited by MinsukJi-NOAA

Loading

BrianCurtis-NOAA commented May 20, 2022 •

edited

Loading

MinsukJi-NOAA commented May 20, 2022 •

edited

Loading