Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken easyconfigs using Rust (from before Cargo easyblocks) #20718

Open
Micket opened this issue Jun 4, 2024 · 4 comments · Fixed by #20864
Open

Broken easyconfigs using Rust (from before Cargo easyblocks) #20718

Micket opened this issue Jun 4, 2024 · 4 comments · Fixed by #20864
Milestone

Comments

@Micket
Copy link
Contributor

Micket commented Jun 4, 2024

So, anything using rust that doesn't use the cargo*.py easyblocks most certainly already doesn't work, or won't work in the near future.

Retroactively fixing them is hard; the cargo lock and such does not account for compiler versions, and finding the old package list or just a ton of tedious unrewarding work that I'm certainly not willing to do.

We should clean all of these out, replace with newer versions if needed/possible.

List that I find are:

  • p/polars/polars-0.15.6-foss-2022a.eb would also affect infercnvpy-0.4.2-foss-2022a.eb
  • a/alevin-fry/alevin-fry-0.6.0-GCCcore-10.3.0.eb
  • a/alevin-fry/alevin-fry-0.4.3-GCCcore-11.2.0.eb
  • l/Longshot/Longshot-0.4.3-GCCcore-10.2.0.eb
  • l/librsvg/librsvg-2.51.2-GCCcore-10.3.0.eb (unsure, but they do use the Rust as a build dep)
  • l/librsvg/librsvg-2.52.8-GCCcore-11.2.0.eb
  • l/librsvg/librsvg-2.55.1-GCCcore-11.3.0.eb
  • l/librsvg/librsvg-2.58.0-GCCcore-13.2.0.eb
  • l/libavif/libavif-0.9.0-foss-2020b.eb (unsure, these are really strange, using Rust as a runtime dep. For rav1e codec i assume? But why runtime dep...)
  • l/libavif/libavif-0.11.1-foss-2022a.eb
  • l/libavif/libavif-0.11.1-foss-2021a.eb
  • l/libavif/libavif-1.0.4-foss-2023a.eb
  • l/libavif/libavif-0.11.1-GCCcore-11.3.0.eb
  • g/gsutil/gsutil-5.10-GCCcore-11.2.0.eb
  • f/fastparquet/fastparquet-0.8.0-foss-2021b.eb
  • f/fastparquet/fastparquet-0.7.2-foss-2021a.eb
  • s/Safetensors/Safetensors-0.3.1-foss-2022a.eb
  • s/Safetensors/Safetensors-0.3.1-foss-2022a-CUDA-11.7.0.eb
  • t/tokenizers/tokenizers-0.12.1-GCCcore-10.3.0.eb
  • t/Transformers/Transformers-4.30.2-foss-2022b.eb
  • t/Transformers/Transformers-4.24.0-foss-2022a.eb
  • t/Transformers/Transformers-4.29.2-foss-2022a.eb
@branfosj branfosj added the EasyBuild-5.0 EasyBuild 5.0 label Jun 5, 2024
@boegel boegel added this to the 5.0 milestone Jun 5, 2024
@Micket
Copy link
Contributor Author

Micket commented Jun 5, 2024

Doesn't really solve the issue with Rust and cargo easyblock, but there is at least now that fixes that strange runtime compiler dep: #20747

Both librsvg and libavif are troublesome, we'll have to think of another solution there, because I don't want to make some a bunch of extra easyblocks like "ConfigureMakeCargo" etc. Yuck. All they really need is the

  1. crates support for unpacking the extra sources
  2. a couple of environment variables which tells cargo how to build.
    Not that easy to add to ConfigureMake/CMakeMake type blocks though...

@Micket
Copy link
Contributor Author

Micket commented Jun 5, 2024

I asked @lcniel to look into making a newer and imprved gsutil without custom cryptography + Rust.

@Micket
Copy link
Contributor Author

Micket commented Jun 19, 2024

#20864 Fixing latest Transformers here.
Maybe want to fix the rest of the Safetensors and Transformers too.

@Micket
Copy link
Contributor Author

Micket commented Jun 20, 2024

i think github accidentally picked up on "partly fixes xxxx"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants