Workarounds for MPICH

[mathomp4]

Description

This PR tracks various workarounds needed for MPICH as it does not support an interface in use mpi (that it should, see pmodels/mpich#6691) and a bug in how it handles ierror (fixed in repo, see pmodels/mpich#6693 and pmodels/mpich#6694). The latter will probably be in MPICH 4.1.3, but that might be a while off.

Note that this also requires updates to pFlogger which has the same non-support (Goddard-Fortran-Ecosystem/pFlogger#100) which is in develop there and will be released soon.

These were all pointed out to me by @climbfuji. I'm efforting to see if I can build a working MPICH stack on discover. It's been...a challenge.

Related Issue

Motivation and Context

Let's MAPL (and hopefully GEOS) build with MPICH. And makes the life of @climbfuji better. (Though we might need to backport this stuff to a previous release cycle if it all works)

How Has This Been Tested?

Ran with Intel Fortran/Intel MPI on discover and zero-diff. 🎉

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Trivial change (affects only documentation or cleanup)

Checklist:

• I have tested this change with a run of GEOSgcm (if non-trivial)
• I have added one of the required labels (0 diff, 0 diff trivial, 0 diff structural, non 0-diff)
• I have updated the CHANGELOG.md accordingly following the style of Keep a Changelog

[mathomp4]

Well, this seems to be safe. Zero-diff in testing.

[mathomp4]

@climbfuji When you all use MAPL + MPICH, are you using our Oserver? If not, this might be enough for you (assuming you don't use pflogger as well).

[mathomp4]

Note to @GEOS-ESM/mapl-team: I tested this on discover and zero-diff.

[climbfuji]

@mathomp4 I was successful with backporting the workaround to 2.35.2 and 2.40.3. There are other issues with mpich4 with the ufs-weather-model that prevent us from testing if those backports are good or not. But I can share them with you if you wish.

[mathomp4]

@mathomp4 I was successful with backporting the workaround to 2.35.2 and 2.40.3. There are other issues with mpich4 with the ufs-weather-model that prevent us from testing if those backports are good or not. But I can share them with you if you wish.

@climbfuji Are the new updates in MAPL? If so, please do pass them along so I can incorporate them here.

Also, would you like us to make new releases of MAPL 2.35 and 2.40 with the changes?

[climbfuji]

@mathomp4 I was successful with backporting the workaround to 2.35.2 and 2.40.3. There are other issues with mpich4 with the ufs-weather-model that prevent us from testing if those backports are good or not. But I can share them with you if you wish.

@climbfuji Are the new updates in MAPL? If so, please do pass them along so I can incorporate them here.

Also, would you like us to make new releases of MAPL 2.35 and 2.40 with the changes?

No, they are patches that we run on those two versions in spack (our fork). I can create the PRs for that so that you can look at it).

[mathomp4]

No, they are patches that we run on those two versions in spack (our fork). I can create the PRs for that so that you can look at it).

Oh...okay. Well, it should be a pretty easy release if needed. But if this one is good, then we can work on getting out MAPL 2.42 for future work.

[tclune]

More checks - makes cmake initial step slower. :-(