haskell.compiler.ghc*: fix cross-built native GHC #243619

AlexandreTunstall · 2023-07-15T09:32:26Z

Description of changes

This PR fixes all versions of GHC older than 9.6 to allow cross-compiling GHC itself (build ≠ host = target). In addition to that, it adds an overridable option to force building an unregisterised version.

Changes have also been made to 9.6 and newer, but they are merely tentative because Hadrian cannot currently cross-compile GHC.

Due to bugs in cc-wrapper and binutils-wrapper, cross-compiling GHC still doesn't produce binaries usable without emulation or further fixes, but this will at the very least allow users to port GHC to platforms that Nixpkgs doesn't support natively.

Things done

ghost · 2023-07-16T21:12:54Z

Hi, thanks for writing this PR!

This is a really really big PR (not your fault; ghc still uses copy-paste-tweak style instead of switch-on-version style) in monolithic single-commit form which makes it very hard to review; would you mind breaking it up into a separate commit for each mechanical change (like "search and replace FOO with BAR" or "replace libffi with targetPackages.libffi")? Then we can focus our review effort on the remaining commits with non-mechanical changes, which should be short and focused.

Also, have you tried building anything that uses Template Haskell? Last time I tried to get build,host=x86_64-linux target=aarch64-linux to work using qemu-user for Template Haskell, I hit a known GHC bug in their custom linker that caused segfaults. I can dig up the GHC bug link if you like.

Unfortunately Template Haskell is widely used in the Haskell ecosystem; there are a couple of pervasive dependencies (like Vty) that use it, so everything that uses those needs it.

sternenseemann · 2023-07-16T21:24:19Z

Also, have you tried building anything that uses Template Haskell? Last time I tried to get build,host=x86_64-linux target=aarch64-linux to work using qemu-user for Template Haskell, I hit a known GHC bug in their custom linker that caused segfaults. I can dig up the GHC bug link if you like.

That is kind of orthogonal to this PR which gets build != host, host == target to work properly (I recall that building with build, host, target distinct is impossible). -fexternal-interpreter is something we should still tackle, but it makes more sense to do so in a different context, I'd say.

ghost · 2023-07-16T22:07:06Z

That is kind of orthogonal to this PR which gets build != host, host == target to work properly

Ah, so, first line of the PR should probably be adjusted to:

This PR fixes all versions of GHC older than 9.6 to allow ~~both building GHC as a cross-compiler and~~ cross-compiling GHC itself.

AlexandreTunstall · 2023-07-16T22:09:42Z

would you mind breaking it up into a separate commit for each mechanical change (like "search and replace FOO with BAR" or "replace libffi with targetPackages.libffi")?

@amjoseph-nixpkgs I assume by this you mean split it up by change (e.g. apply X change to each version) and not split it up by version (e.g. fix 9.4.5), right? Sure.

Also, have you tried building anything that uses Template Haskell?

I have encountered a few errors that mentioned -fexternal-interpreter while trying to use a riscv64 cross-compiled GHC to bootstrap GHC 9.6.2 natively, but so far they've all been in test suites (so I disabled checks to get around them). My goal with this isn't to build a fully-functional GHC, but just something to bootstrap GHC on otherwise unsupported platforms. As far as I'm aware, building GHC does not require TH support.

I haven't tried using the cross-compilers to build any TH-using code. As sternenseemann mentioned, this is more about fixing cross-compiling GHC itself. I assume Nixpkgs already supports building GHC as a cross-compiler, but if it doesn't, this PR also makes that possible.

ghost · 2023-07-16T22:25:31Z

@amjoseph-nixpkgs I assume by this you mean split it up by change (e.g. apply X change to each version) and not split it up by version (e.g. fix 9.4.5), right? Sure.

Correct!

My goal with this isn't to build a fully-functional GHC, but just something to bootstrap GHC on otherwise unsupported platforms.

Ah, thanks, now I get it! Yes this will be very useful, because the ancient GHC-supplied binaries we've been using to bootstrap on powerpc64le have ceased working. So I'd like to use this there too. Awesome!

AlexandreTunstall · 2023-07-17T03:25:09Z

pkgsCross.aarch64-multiplatform.pkgsBuildHost.haskell.compiler.native-bignum.ghc945 seems to build fine on Nixpkgs already, so I've amended the PR description.

AlexandreTunstall · 2023-07-23T09:37:31Z

@amjoseph-nixpkgs I have split this into 11 separate (hopefully atomic) commits.

I've also spent much of my week trying to compile GHC 9.6.2 natively on RISC-V using an unregisterised GHC 9.4.5 cross-compiled using these changes, and I've figured out what it takes.

pkgs.haskell.compiler.ghc962.override (old: rec {
  bootPkgs = pkgs.haskell.packages.ghc945.override {
    buildHaskellPackages = bootPkgs;

    ghc = let
      passthru = {
        targetPrefix = "";
        enableShared = false;
        hasHaddock = false;

        llvmPackages = pkgs.llvmPackages_12;

        haskellCompilerName = "ghc-9.4.5";
      };
    in passthru // {
      version = "9.4.5";

      outPath = builtins.storePath boot/ghc;

      inherit passthru;

      meta = {
        license = lib.licenses.bsd3;
        platforms = [ "riscv64-linux" ];
      };
    };

    overrides = self: super: {
      mkDerivation = args: super.mkDerivation ({
        enableLibraryProfiling = false;
      } // args);

      # These test suites don't compile with boot GHC
      alex = pkgs.haskell.lib.compose.dontCheck super.alex;
      data-array-byte = pkgs.haskell.lib.compose.dontCheck super.data-array-byte;
      doctest = pkgs.haskell.lib.compose.dontCheck super.doctest;
      hashable = pkgs.haskell.lib.compose.dontCheck super.hashable;
      optparse-applicative = pkgs.haskell.lib.compose.dontCheck super.optparse-applicative;
      QuickCheck = pkgs.haskell.lib.compose.dontCheck super.QuickCheck;
      temporary = pkgs.haskell.lib.compose.dontCheck super.temporary;
      vector = pkgs.haskell.lib.compose.dontCheck super.vector;
    };
  };
})

And I needed to patch Nixpkgs so that Hadrian wouldn't try to use RTS flags only supported by the threaded runtime.
I couldn't work out how to do this with override/overrideAttrs.

diff --git a/pkgs/development/tools/haskell/hadrian/default.nix b/pkgs/development/tools/haskell/hadrian/default.nix
index 5911c34982b..da4d194e220 100644
--- a/pkgs/development/tools/haskell/hadrian/default.nix
+++ b/pkgs/development/tools/haskell/hadrian/default.nix
@@ -29,6 +29,8 @@ mkDerivation {
     # Additionally we need to recompile it on every change of UserSettings.hs.
     # See https://gitlab.haskell.org/ghc/ghc/-/merge_requests/1190
     "-O0"
+    # Don't use threaded-only RTS flags at runtime
+    "-f-threaded"
   ];
   isLibrary = false;
   isExecutable = true;

I'd like to make it possible to use a non-threaded GHC to build GHC, but I'm not sure how. Should it be as above? Should it be an overridable option to GHC which is propagated to Hadrian? Should it be a flag on GHC derivations similar to hasHaddock? Anyway, that's a change for another PR.

AlexandreTunstall · 2023-07-23T09:46:57Z

Is it possible to have Hydra cross-compile GHC for all platforms?

It could potentially simplify bootstrapping on unsupported platforms by substituting the derivation instead of having to manually build it on a supported platform, copy the closure to the host, and write some rather elaborate code to convince Nixpkgs to use it as a boot compiler.

In fact, we could even configure Nixpkgs to do so by default on unsupported platforms, so that users can just nix-shell -p ghc and get a working compiler.

sternenseemann · 2023-08-09T10:24:13Z

It could potentially simplify bootstrapping on unsupported platforms by substituting the derivation instead of having to manually build it on a supported platform, copy the closure to the host, and write some rather elaborate code to convince Nixpkgs to use it as a boot compiler.

We can via release-cross.nix. Relying on substitution would be bad form here, rather we should maintain a derivation that downloads a known good build artefact using an ordinary fetcher, fixes runtime dependencies with patchelf etc. Not sure yet if we need to upload something to tarballs.nixos.org for this or we can fetch from the normal binary cache for this.

sternenseemann

Sorry I took so long to get back to you! This looks great, I'll be sure to play around with it a bit as well. It is also somehow simpler than I expected, e.g. it is not necessary to set any make variables telling it to build stage 2?

All suggestions are only noted once, since the changes are duplicated (very neatly, I must say) over the expressions.

I normally dislike tentative changes that don't work yet, but I guess the changes to the hadrian expression are alright as the retain some measure of consistency with the old expressions!

Also since this PR was opened, 9.4.6.nix was added, so we'll need to remember to take care of that at the end.