Fixes for macOS

[ncough]

Partial fixes for getting builds to work on macOS. Many fixes are quite hacky, hopefully someone who knows what they are doing can fix them properly.

• Explicitly link abseil into gtirb, as a dependency of protobuf. Not clear why this isn't done automatically.
• Missing #include <map> in gtirb-pprinter.
• Disable concurrent souffle builds.
• Explicitly provide an implementation of protoc to basil, due to issues downloading and caching it.
• Fix a linking issue with ocamlPackages.z3.
• Adjust build paths for python-gtirb

[katrinafyi]

This is excellent! Thanks for your work. I hope Nix wasn't too frustrating :)

[ncough]

Nix has been mostly pain free. A bit confused on by basil build if you get a moment, it passes on my machine but fails on ci. Seems the ci doesn't run the preConfigure step.

[katrinafyi]

Basil build is fixed on my computer by 302dd59. This happened because the configure phase was not being run in the dependency derivation, which mkSbtDerivation runs to download all dependencies (evidenced by the basil-sbt-dependencies.tar.zst> prefix in front of the log messages, instead of basil>). Within this derivation, configuring is explicitly disabled.

Propagating the replacement command makes it work. I suspect some caching was at play to make it succeed locally.

[katrinafyi]

Re protobuf abseil linking errors: https://www.github.com/protocolbuffers/protobuf/issues/12637

[ncough]

From what I see locally, the only remaining issue is the determinism fix for ddisasm not working on macOS. If this isn't used, I suspect it would be far easier to disable the test.

[katrinafyi]

It looks really good now! Just one thing left, we should make the ddisasm test use a "nolibc" variant of the compiler so it doesn't spin cross-compiling libc. I can look into this if you want.

Edit: Or simply vendor an aarch64 ELF binary (somewhere not in the git repo) - the test is not about the cross compiler.

[ncough]

Thanks! I'll leave it to you. I was wondering if it would be possible to have Nix provide a full cross compiling environment on macOS anyway, as full tests aren't practical without it.

[katrinafyi]

todo notes for myself

[katrinafyi]

@ncough real quick about using Nix as a cross-compile environment, this is definitely possible. but the main problem is the Nix-provided compilers are not hermetic - they will look for and may or may not find libraries on your system. It is especially dicey if the Nix compilers call a non-Nix tool which it finds through PATH.

The solutions I can conceptualise are:

• run all compilations within a Nix sandbox - clunky, slow, difficult to integrate, adds Nix as a runtime dependency, and bloats Nix store on disk with compilation results
• use Nix compiler within a chroot / Docker - probably easiest. some trouble transporting files across the boundary
• somehow manually isolate the Nix compiler, e.g. by clearing PATH in a wrapper script - hard
• just use non-Nix compilers from OS packages - give up, very easy but obviously non-portable and non reproducible

[katrinafyi]

In advancing the nixpkgs version for arm-tv, it seems that ocaml is broken on aarch64-darwin https://www.github.com/NixOS/nixpkgs/issues/350394. This seems to be fixed now https://hydra.nixos.org/build/276865224