-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for numpy >= 2.x.x #378
Comments
The only problematic dependency seems to be bitshuffle - changing the numpy dependency to allow 2.0.0 leads to the following error in HSDS CI:
I was able to replicate this error on my machine while trying to bump the numpy version requirement. That said, I'm not actually sure what's causing it. I created a fork of bitshuffle that uses numpy 2.0.0 and its own tests pass just fine. When I extracted the bitshuffle unit tests from HSDS and ran them on their own, they also passed just fine. This seems to be some particular quirk of how HSDS uses or imports bitshuffle. Having HSDS use the updated fork of bitshuffle still leads to this error. |
Running HSDS locally, the bitshuffle fork seems to work. I've made a PR to the bitshuffle repo, but it doesn't seem to be actively maintained. We may need to find another external library to use for this. |
You could try docker exec'ing to the HSDS container and see if the bitshuffle unit tests work. If not, it's probably an issue with how the HSDS docker image is being built. |
The reason Docker CI fails is that each Docker contains needs to install its own dependencies from pip, and the main branch of bitshuffle that it installs isn't compatible with numpy 2.0. Since the repo for bitshuffle seems dead, our best bet may be publish our own fork of bitshuffle to PyPI and then update HSDS's requirements |
No description provided.
The text was updated successfully, but these errors were encountered: