Possible strategy to resolve the neverending CoreFoundation purity woes on Darwin

[copumpkin]

This is a placeholder until I finish writing more details, but I want to refer to it from code.

The short answer is some rpath hackery in the pure CoreFoundation and some careful hooks.

[copumpkin]

I've started work on this, but it's a little tricky

[aflatter]

Thanks 🙏 for working on this!

[copumpkin]

Here's the strategy, since I'm having free time issues:

1. Adjust our pure CF library's install name explicitly to say @rpath/CoreFoundation. This means that anything linking against it (regardless of what path it lives at) will actually refer to @rpath/CoreFoundation rather than the path we pass in.
2. Adjust our ld-wrapper to automatically add an -rpath ${CF} to everything we link. This should ensure that everything works as it does today, given part 1
3. Make the behavior in part 2 conditional on a stdenv variable that the frameworks derivations would set as a build hook, so ld-wrapper could optionally add an -rpath pointing at /System/Library/Frameworks/CoreFoundation.framework if that variable is set

This means that any time you pull in frameworks.X, a build hook would set something like RPATH_SYSTEM_COREFOUNDATION, which would tell ld-wrapper to change the rpath to the impure CF. That would mean that even libraries built much earlier in the dependency hierarchy would get retroactively linked at runtime against the system CF, despite being built against the pure one.

Another way of describing the strategy is to use the @rpath as a late-binding mechanism for CF specifically. And possibly SystemConfiguration.framework and anything else that might need it.

Does that make sense?

[periklis]

After some thinking i cannot think of any linking issues and why this shouldn't work. Since every part of our compile/link toolchain is a wrapper we can instruct it to do what we want. The only issue i would like to point is, what is the impact if some builds with our older pure CF headers but the newer linked CF has different symbols. Do you known any list of BC breaks in the apple docs for CF?

[0xABAB]

A complete proposal of a design should reference Apple documentation for every assertion made.

So, a strategy I would recommend is:

• Write an experiment on your own machine
• Announce outcome of experiment
• Write some tests to make your experiment reproducible for others, so they don't need to take your word for it and it can be used to see if Apple breaks certain contracts
• Based on the experiments, write a complete proposal for a design, which could have additional insights/optimizations
• Implement based on the design a production ready solution

I think some split of these tasks would be good, in order to avoid a two year long issue.

[Ericson2314]

Another strategy I was thinking was with to exploit the cross compilation stuff so that build-time dependencies (normally executables to be run) might be install_name_tool'd or otherwise made impure, while run-time-dependencies (normally libraries) are kept pure.

Now, code gen for build time, runtime executable dependencies, etc etc complicate things. But extra conditions can get around all that.

Forgive me if this solves a non-problem, but this this or a variation would allow transitive library deps that ultimately need the impure one to not necessitate an impure build process.

[LnL7]

Implemented in #27598

[copumpkin]

❤️