Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin numpy<2.0.0 #178

Merged
merged 4 commits into from
Jul 1, 2024
Merged

Pin numpy<2.0.0 #178

merged 4 commits into from
Jul 1, 2024

Conversation

angus-g
Copy link
Collaborator

@angus-g angus-g commented Jul 1, 2024

I guess it's nice that we have type information encoded here now too. This does introduce a bit of an incompatibility in the testing environment, where numpy 2.0.0 is required, but it's not an issue for actual running.

Famous last words: it is an issue for running, at the moment. We will probably just have to restrict ourselves to numpy < 2.0.0 for the moment.

@angus-g
Copy link
Collaborator Author

angus-g commented Jul 1, 2024

I think the new error we're seeing is an incompatibility between xESMF and numpy 2.0.0 (maybe):

ValueError: cannot reshape array of size 161438054400 into shape (np.int32(249), np.int32(140), np.int32(2412), np.int32(1920))

32-bit ints are not big enough to represent the products of the dimensions, and I wonder if the overflow there is causing it to fail a check somewhere:

>>> import numpy as np
>>> np.int32(249)*np.int32(140)*np.int32(2412)*np.int32(1920)
<stdin>:1: RuntimeWarning: overflow encountered in scalar multiply
-1770702848

The fix for the moment might be to just pin numpy to <2.0.0.

There are still a few incompatibilities with numpy 2.0.0, so we'll pin
the previous version for the moment. In particular, xESMF regridders
can't handle large grids with a number of points that overflows a
32-bit integer.
@angus-g angus-g changed the title Update doctest for numpy 2.0.0 Pin numpy<2.0.0 Jul 1, 2024
@ashjbarnes
Copy link
Collaborator

Nice! Ok should we merge this right away and rebase all active PRs back to main?

@navidcy
Copy link
Contributor

navidcy commented Jul 1, 2024

FYI: Previously I tried to add 3.11 to the CI and failed; see #105

@angus-g
Copy link
Collaborator Author

angus-g commented Jul 1, 2024

How odd that it's the only one to hang!

@navidcy
Copy link
Contributor

navidcy commented Jul 1, 2024

I'm happy to just remove the 3.11 tbh...

For some reason it seems to hang indefinitely. We'll just disable it
for the moment (3.12 works, anyway).
@navidcy navidcy merged commit 578590f into main Jul 1, 2024
5 checks passed
@navidcy navidcy deleted the angus-g/numpy-2 branch July 1, 2024 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants