python: propagate makeWrapper arguments

[rnhmjoj]

Motivation for this change

Python libraries that have runtime dependencies, like Qt and GTK
libraries, are currently not working. These are not included in
the interpreter wrapper and no mechanism exist to do so automatically.

This commit adds a nix-support/make-wrapper-args file that stores
the makeWrapper arguments of a python derivation (created with
buildPythonPackage). The arguments are loaded from python.buildEnv
when creating the Python environment and passed to the makeWrapper
invocation.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change
• Tested compilation of all pkgs that depend on this change: obviously not.
• Tested the wrapping is working for pythonPackages.matplotlib with Qt
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

---

This should fix issues #25351 #80147 #39637

[FRidh]

I think this is a good idea but want the bash code that's now in the postFixup somewhere else.

My idea is that the various wrapper scripts write to this make-wrapper-args file, until we have declarative wrappers. The consumer of that file should maybe be able to choose what parts to take based on an identifier, though I think we can do without. Let's wait for more feedback.

[FRidh]

In mk-python-derivation a hook needs to do this; the function itself should remain small.

@worldofpeace @jtojnar any common approach that can be taken here? I want declarative wrappers in JSON with structured attributes, but we're not there. Let's discuss this in the main thread.

[rnhmjoj]

If the various hook were to export their arguments in a common array, be it makeWrapperArgs or something else it would be great.

[rnhmjoj]

Ok, I moved the shell code to a dedicated hook. I tested again with matplotlib and it seems good. I used a clone of buildPythonPackage to avoid a mass rebuild, so I haven't tested much else.

[rnhmjoj]

Rebased, fixed a conflict and removed the unnecessary escaping.

[FRidh]

I have some changes lined up regarding this (https://github.com/FRidh/nixpkgs/pull/new/makewrapperargshook), but most importantly now, this PR breaks python3.tests. Every interpreter has a tests attribute containing some environment tests.

[rnhmjoj]

@FRidh can you push your changes here? I couldn't use git am to pull them in because the hash of the ancestor changed while rebasing.

[FRidh]

My apologies for not following up with this. I understand the need for this feature, but I am not happy with the eventual approach.
I am of the opinion the specific hooks should propagate a generic hook that does the final wrapping, and to get there that takes more effort. I started this off in #102949. Once that is in, it would be trivial to add this feature.

Declarative wrappers is just so much more work, that is definitely not something to wait for.

[rnhmjoj]

No problem, I very well knew this approach wasn't the ultimate solution. I hoped it could be used as a stop gap to quickly fix python packages with Qt dependencies, but it seems there isn't much of an interest: there are only a couple open issues in Nixpkgs.

Thank you for starting the work on unifying the wrapper hooks. There was much talk about this situation but nothing material had been proposed.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/low-effort-and-high-impact-tasks/21095/41