graalvm-ce: 22.3.1 -> 21.0.0 #257433

thiagokokada · 2023-09-26T12:31:47Z

Description of changes

This initially may look like a downgrade, but this is caused by how upstream is tagging versions.

Before they would have the GraalVM having its own version (e.g. 22.3.1), and each version would support multiple JVM versions (e.g. 11, 17, 19). Now each release supports only one JVM version (e.g.: 21), and they track the same version as the JVM.

They also changed packaging, making all sub-products (e.g.: GraalPy, GraalRuby, etc.) standalone, so they do not depend in GraalVM anymore and have their own version. Thanks to this change, we will need to repackage everything.

Fix: #257292

Things done

thiagokokada · 2023-09-26T18:23:43Z

pkgs/development/compilers/graalvm/community-edition/graalpy/hashes.nix

+  "graalpy" = {
+    "aarch64-linux" = {
+      sha256 = "0n0zz86h7jsqgdiyj6vj7qw57ny40jpmfvylyxq70riy86a4zp67";
+      url = "https://github.com/oracle/graalpython/releases/download/graal-23.1.0/graalpy-community-23.1.0-linux-aarch64.tar.gz";


I am using the non-JVM variants here, e.g.: graalpy-community-jvm. Those versions includes the libraries to interface with JVM, that I imagine enables usage of mixed languages with interop (https://www.graalvm.org/latest/reference-manual/js/JavaInteroperability/).

Not sure who is using this though, and this complicates the packaging, so for now I am going to package the standalone versions instead, and if there is someone that needs those libraries please open an issue.

I'm a bit late to this, but I thought I would add:

For Truffle polyglot projects, if I was going to write one, I would be using something like mvn2nix and I think I would be able to fetch everything necessary from Maven instead of needing them in nixpkgs.

(So, packaging only the standalone versions seems fine.)

thiagokokada · 2023-09-26T18:34:12Z

Result of nixpkgs-review pr 257433 run on x86_64-linux 1

16 packages built:

babashka
babashka-unwrapped
bbin
clj-kondo
clojure-lsp
dapl-native
dbqn-native
graalvm-ce
graalvmCEPackages.graaljs
graalvmCEPackages.graalnodejs
graalvmCEPackages.graalpy
graalvmCEPackages.truffleruby
jet
neil
vscode-extensions.betterthantomorrow.calva
zprint

This initially may look like a downgrade, but this is caused by how upstream is tagging versions. Before they would have the GraalVM having its own version (e.g. 22.3.1), and each version would support multiple JVM versions (e.g. 11, 17, 19). Now each release supports only one JVM version (e.g.: 21), and they track the same version as the JVM. They also changed packaging, making all sub-products (e.g.: GraalPy, GraalRuby, etc.) standalone, so they do not depend in GraalVM anymore and have their own version. Thanks to this change, we will need to repackage everything. To simplify, this commit will remove all sub-products and only care about the GraalVM/Native Image (that is back to GraalVM itself) part. Other commits will re-added each sub-product. Fix (partial): #257292

Exposing this derivation so it is easier to test and use.

thiagokokada · 2023-09-27T21:57:21Z

@ofborg build babashka babashka-unwrapped bbin clj-kondo clojure-lsp dapl-native dbqn-native graalvm-ce graalvmCEPackages.graaljs graalvmCEPackages.graalnodejs graalvmCEPackages.graalpy graalvmCEPackages.truffleruby jet neil vscode-extensions.betterthantomorrow.calva zprint

Finally fix the remaining issues in macOS thanks to the help of @reckenrode. This PR should be ready for review/merge now.

thiagokokada · 2023-09-28T11:40:17Z

Can I get a re-review @ericdallo ?

Also pinging @glittershark @jlesquembre @DerGuteMoritz @bennyandresen @babariviere @hlolli for review.

DerGuteMoritz · 2023-09-28T12:42:51Z

@thiagokokada I'm afraid I'm not familiar enough with graalvm itself to meaningfully review your changes. But I totally would like to commend your effort, thanks a lot!! 🥳

AndrewKvalheim · 2023-12-01T01:15:08Z

nixos/doc/manual/release-notes/rl-2311.section.md

@@ -155,6 +155,10 @@

 - PHP now defaults to PHP 8.2, updated from 8.1.

+- GraalVM has been updated to the latest version, and this brings significant changes. Upstream don't release multiple versions targeting different JVMs anymore, so now we only have one GraalVM derivation (`graalvm-ce`). While at first glance the version may seem a downgrade (22.3.1 -> 21.0.0), the major version is now following the JVM it targets (so this latest version targets JVM 21). Also some products like `llvm-installable-svm` and `native-image-svm` were incorporate to the main GraalVM derivation, so they're included by default.
+
+- GraalPy (`graalCEPackages.graalpy`), TruffleRuby (`graalCEPackages.truffleruby`), GraalJS (`graalCEPackages.graaljs`) and GraalNodeJS (`grallCEPackages.graalnodejs`) are now indepedent from the main GraalVM derivation.


graalCEPackages → graalvmCEPackages

nixos-discourse · 2023-12-01T18:02:03Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-do-i-migrate-this-expression-to-nixos-23-11/36314/1

nixos-discourse · 2023-12-01T21:23:05Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-do-i-migrate-this-expression-to-nixos-23-11/36314/2

rafaelrc7 · 2024-07-20T15:00:34Z

@thiagokokada Sorry for the pinging on an old PR, but I don't know if this was supposed to be an issue...

This PR removed specific versions of graalvm (graalvm17-ce for example) and it says that "upstream dropped support to different JDK versions for each GraalVM release". However, this does not seem to be the case. The release calendar (https://www.graalvm.org/release-calendar/) for instance states that graalvm17 and graalvm21 will be updated at least until 2026 and the download page (https://www.graalvm.org/downloads/) lets you download the JDK 17, 21 and 22 versions. Did I misunderstand this?

I'm also asking about this, because I am trying to package pkl (#286658), we recently managed to fix the gradle build. However, this package also supports a native build using graalvm, that is much faster. However, it depends specifically on graalvm17. Because of the removal of graalvm17 I am not being able to build the native image through nixpkgs. Assuming I did not misunderstand anything, would it be possible to readd the still supported past graalvm versions? (currenty jdk 17 and 21)

thiagokokada · 2024-07-20T16:23:52Z

@thiagokokada Sorry for the pinging on an old PR, but I don't know if this was supposed to be an issue...

This PR removed specific versions of graalvm (graalvm17-ce for example) and it says that "upstream dropped support to different JDK versions for each GraalVM release". However, this does not seem to be the case. The release calendar (https://www.graalvm.org/release-calendar/) for instance states that graalvm17 and graalvm21 will be updated at least until 2026 and the download page (https://www.graalvm.org/downloads/) lets you download the JDK 17, 21 and 22 versions. Did I misunderstand this?

I'm also asking about this, because I am trying to package pkl (#286658), we recently managed to fix the gradle build. However, this package also supports a native build using graalvm, that is much faster. However, it depends specifically on graalvm17. Because of the removal of graalvm17 I am not being able to build the native image through nixpkgs. Assuming I did not misunderstand anything, would it be possible to readd the still supported past graalvm versions? (currenty jdk 17 and 21)

Yes, you misunderstood this. Oracle GraalVM is still supported for LTS versions, but GraalVM Community Edition is only supported in one major JVM version. You can take a look at the link you sent that shows a table comparing Oracle vs Community Edition versions.

Also, AFAIK we can't distribute binaries built with Oracle GraalVM, since the license is proprietary. We can still add them as proprietary, but the resulting binaries will not be built in Hydra.

BTW, I have zero interest in adding support for Oracle GraalVM, but if someone wants to do so it is fine, I can review it.

rafaelrc7 · 2024-07-20T16:32:16Z

Yes, you misunderstood this. Oracle GraalVM is still supported for LTS versions, but GraalVM Community Edition is only supported in one major JVM version.

Ah, I understand it now, thanks for the explanation!

BTW, I have zero interest in adding support for Oracle GraalVM, but if someone wants to do so it is fine, I can review it.

It seems someone is working on it (#321026)

Thanks for the answers.

ofborg bot added 8.has: clean-up 8.has: package (new) This PR adds a new package labels Sep 26, 2023

ofborg bot requested review from glittershark, DerGuteMoritz, bennyandresen, babariviere, jlesquembre, ericdallo and hlolli September 26, 2023 17:57

ofborg bot added 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Sep 26, 2023

thiagokokada commented Sep 26, 2023

View reviewed changes

thiagokokada marked this pull request as ready for review September 26, 2023 18:37

This comment was marked as outdated.

Sign in to view

thiagokokada added 9 commits September 27, 2023 10:25

graalvmCEPackages.graalpy: init at 23.1.0

681f8bf

graalvmCEProducts: generate version inside update.sh script

fef3db6

graalvmCEPackages.truffleruby: init at 23.1.0

f52add0

graalvmCEPackages.graaljs: init at 23.1.0

ba9a3fe

graalvmCEPackages.graalnodejs: init at 23.1.0

8374f4d

graalvmCEPackages: reorganize each product in its own subdirectory

3e9ecc8

graalvmCEPackages: normalize package platforms in update.sh script

76f51c0

graalvmCEPackages.buildGraalvmProduct: fix build in darwin

8bf7e83

This comment was marked as outdated.

Sign in to view

graalvmCEPackages.buildGraalvm: mark as broken in darwin

55d5224

This comment was marked as outdated.

Sign in to view

graalvmCEPackages: refactor

9249918

This comment was marked as outdated.

Sign in to view

ericdallo approved these changes Sep 27, 2023

View reviewed changes

delroth added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Sep 27, 2023

ofborg bot requested a review from ericdallo September 27, 2023 16:23

delroth removed the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Sep 27, 2023

thiagokokada added 3 commits September 27, 2023 22:18

graalvmCEPackages.buildGraalvm: fix native-image compilation

61b9105

graalvmCEPackages.buildGraalvmProduct: add Foundation to buildInputs

7457b3c

graalVMPackages.graalvm-ce-musl: init at 21.0.0

cfeeda0

Exposing this derivation so it is easier to test and use.

release-notes: add GraalVM changes

8c01dd9

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation 8.has: changelog labels Sep 27, 2023

thiagokokada mentioned this pull request Sep 28, 2023

GraalVM 23.0.0's native-image is cleaning the environment before calling C compiler oracle/graal#7502

Closed

graalvmCEPackages: some fixes to update.sh script

6a87192

wegank approved these changes Sep 28, 2023

View reviewed changes

delroth added 12.approvals: 2 This PR was reviewed and approved by two reputable people 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in the package labels Sep 28, 2023

thiagokokada merged commit 88c5afe into NixOS:master Sep 29, 2023
25 checks passed

thiagokokada deleted the graalvm-bump branch September 29, 2023 10:12

jlesquembre mentioned this pull request Oct 5, 2023

mkGraalBin.nix { graalvm ? graalvmCEPackages.graalvm19-ce } jlesquembre/clj-nix#94

Closed

cdmistman mentioned this pull request Nov 27, 2023

everything is unstable replit/nixmodules#184

Merged

1 task

AndrewKvalheim reviewed Dec 1, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

graalvm-ce: 22.3.1 -> 21.0.0 #257433

graalvm-ce: 22.3.1 -> 21.0.0 #257433

thiagokokada commented Sep 26, 2023 •

edited

Loading

thiagokokada Sep 26, 2023

solson Oct 7, 2023

thiagokokada commented Sep 26, 2023

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

thiagokokada commented Sep 27, 2023

thiagokokada commented Sep 28, 2023

DerGuteMoritz commented Sep 28, 2023

AndrewKvalheim Dec 1, 2023

nixos-discourse commented Dec 1, 2023

nixos-discourse commented Dec 1, 2023

rafaelrc7 commented Jul 20, 2024 •

edited

Loading

thiagokokada commented Jul 20, 2024

rafaelrc7 commented Jul 20, 2024

graalvm-ce: 22.3.1 -> 21.0.0 #257433

graalvm-ce: 22.3.1 -> 21.0.0 #257433

Conversation

thiagokokada commented Sep 26, 2023 • edited Loading

Description of changes

Things done

thiagokokada Sep 26, 2023

Choose a reason for hiding this comment

solson Oct 7, 2023

Choose a reason for hiding this comment

thiagokokada commented Sep 26, 2023

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

thiagokokada commented Sep 27, 2023

thiagokokada commented Sep 28, 2023

DerGuteMoritz commented Sep 28, 2023

AndrewKvalheim Dec 1, 2023

Choose a reason for hiding this comment

nixos-discourse commented Dec 1, 2023

nixos-discourse commented Dec 1, 2023

rafaelrc7 commented Jul 20, 2024 • edited Loading

thiagokokada commented Jul 20, 2024

rafaelrc7 commented Jul 20, 2024

thiagokokada commented Sep 26, 2023 •

edited

Loading

rafaelrc7 commented Jul 20, 2024 •

edited

Loading