Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin matplotlib and numpy to 3.7.4 and 1.23.5 #1283

Merged

Conversation

DavidHuber-NOAA
Copy link
Collaborator

@DavidHuber-NOAA DavidHuber-NOAA commented Sep 3, 2024

Summary

During testing of an early 1.8.0 candidate, it was found that compiling numpy v1.25.x and matplotlib v3.8.x together with Intel caused matplotlib to fail to generate basic plots that included tick marks. Reverting the versions to 1.23.5 and 3.7.4 resolved the issue.

Testing

@climbfuji was able to generate basic plots using this combination.
I concretized the unified-dev environment on Hera.

Applications affected

Verif-global and most any other plotting package.

Systems affected

All

Dependencies

N/A

Issue(s) addressed

#1276

Checklist

  • This PR addresses one issue/problem/enhancement, or has a very good reason for not doing so.
  • These changes have been tested on the affected systems and applications.

@climbfuji climbfuji changed the base branch from develop to release/1.8.0 September 3, 2024 18:08
Copy link
Collaborator

@climbfuji climbfuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DavidHuber-NOAA There are several entries in the site configs that have py-numpy set to @:1.25 unfortunately (ignore the mpich matches). All of these will need to be changed as well.

[dom@blackpearl spack-stack-1.8.0]$ grep -Rie 1.25 configs
configs/common/packages.yaml:    - '@:1.25'
configs/sites/tier1/acorn/packages_intel.yaml:      - '@:1.25 ^openblas'
configs/sites/tier1/orion/packages_intel.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/hercules/packages_intel.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/noaa-gcloud/packages_intel.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/noaa-aws/packages_intel.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/noaa-azure/packages_intel.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/s4/packages.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/gaea-c6/packages.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/hera/packages_intel.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/aws-pcluster/packages_intel.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/derecho/packages_intel.yaml:      mpi:: [[email protected]]
configs/sites/tier1/derecho/packages_intel.yaml:    - spec: [email protected]%[email protected] +wrappers
configs/sites/tier1/derecho/packages_intel.yaml:      - cray-mpich/8.1.25
configs/sites/tier1/derecho/packages_intel.yaml:    - '@:1.25 ^openblas'
configs/sites/tier1/derecho/packages_gcc.yaml:      mpi:: [[email protected]]
configs/sites/tier1/derecho/packages_gcc.yaml:    - spec: [email protected]%[email protected] +wrappers
configs/sites/tier1/derecho/packages_gcc.yaml:      - cray-mpich/8.1.25
configs/sites/tier1/nautilus/packages_aocc.yaml:    - '@:1.25 ^intel-oneapi-mkl'
configs/sites/tier1/nautilus/packages_gcc.yaml:    - '@:1.25 ^intel-oneapi-mkl'
configs/sites/tier1/gaea-c5/packages.yaml:      mpi:: [[email protected]]
configs/sites/tier1/gaea-c5/packages.yaml:    - spec: [email protected]%[email protected]~wrappers
configs/sites/tier1/gaea-c5/packages.yaml:      prefix: /opt/cray/pe/mpich/8.1.25/ofi/intel/19.0
configs/sites/tier1/gaea-c5/packages.yaml:      - cray-mpich/8.1.25
configs/sites/tier1/gaea-c5/packages.yaml:    - '@:1.25 ^openblas'
configs/containers/specs/jedi-ci.yaml:    [email protected],

configs/common/packages.yaml Show resolved Hide resolved
configs/common/packages.yaml Show resolved Hide resolved
@DavidHuber-NOAA
Copy link
Collaborator Author

@climbfuji Alright, I have updated all of the site configs as well.

@AlexanderRichert-NOAA
Copy link
Collaborator

@DavidHuber-NOAA numpy 1.23.x and 1.25.x conflicts with Intel 19-- Will this be a problem on Acorn? I was planning to restrict the version on Acorn in such a way that it will use 1.24.4.

@climbfuji
Copy link
Collaborator

@DavidHuber-NOAA numpy 1.23.x and 1.25.x conflicts with Intel 19-- Will this be a problem on Acorn? I was planning to restrict the version on Acorn in such a way that it will use 1.24.4.

The problem was that plotting didn't work with Intel (newer Intel compilers, though) and numpy 1.24/1.25 + matplotlib. Can you check for the problem described in #1276 on Acorn with your compiler and numpy versions?

@AlexanderRichert-NOAA
Copy link
Collaborator

It seems to run okay with [email protected] ^[email protected] %[email protected], specifically, it saves test.png without any errors.

@climbfuji
Copy link
Collaborator

It seems to run okay with [email protected] ^[email protected] %[email protected], specifically, it saves test.png without any errors.

Ah well. Then let's pin numpy to 1.24.4 on acorn and 1.23.5 everywhere else. We need to update our inline comments around numpy to remind us that any changes for numpy in configs/common/packages.yaml will likely require changes in a number of other places.

Copy link
Collaborator

@climbfuji climbfuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am approving this in anticipation of a follow-up PR specifically for Acorn (probably by @AlexanderRichert-NOAA) that changes py-numpy for that platform only (and probably only for Intel 19?) to 1.24.4.

@climbfuji climbfuji merged commit e689737 into JCSDA:release/1.8.0 Sep 4, 2024
8 checks passed
@DavidHuber-NOAA DavidHuber-NOAA mentioned this pull request Sep 4, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants