Windows does not allow symlinks by default, resulting in some packages unable to be unpacked correctly

[IntegratedQuantum]

Zig Version

zig-windows-x86_64-0.12.0-dev.983+78f2ae7f2

Steps to Reproduce and Observed Behavior

Download my project on windows:
https://github.com/PixelGuys/Cubyz/tree/0c2d309f2af7e66fdb99bb5dac2a2237635151aa
(Sorry for just dumping the entire project here. I don't own windows, so sadly I couldn't create a minimal reproducible. So I only have a report from a user who stumbled over the error)
Run it with zig build run or run.bat(this one automatically downloads the right zig version)
Observe the following error:

error: unable to unpack tarball
   .url = "https://github.com/harfbuzz/harfbuzz/archive/refs/tags/8.2.2.tar.gz",
note: unable to create symlink from 'README' to 'README.md': AccessDenied

Expected Behavior

Zig should support packages that contain symlinks. It works fine on Linux by the way.

If it is impossible to create symlinks on windows without permissions, then zig should at least use a workaround, like replacing the symlink file with a copy of the original or something like that.

[der-teufel-programming]

Doesn't reproduce for me using 0.12.0-dev.1118+d8f7c7929

[IntegratedQuantum]

Weird. Maybe it only happens with certain user permissions or something like that?

[der-teufel-programming]

Could they try using a newer version of Zig? There were some symlink and tar related PRs merged recently

[squeek502]

https://learn.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/create-symbolic-links

[andrewrk]

Zig does support symlinks on Windows, and the package manager supports them correctly as well. The problem is the OS does not allow them without a setting being toggled.

In this use case, https://github.com/harfbuzz/harfbuzz/archive/refs/tags/8.2.2.tar.gz is a "pristine upstream source" tarball, meaning that the contents are controlled by the harfbuzz project. Those symlinks are present, and they may have semantic meaning, which is why they are represented in the package hash. According to the zig build system, they cannot be replaced with a workaround, because that might mean something different to the package, and it would change the package hash.

Here are two ways for the situation to be resolved:

1. The harfbuzz zig package could be provided via the fork strategy instead of the pristine upstream source tarball strategy. In this strategy, one forks harfbuzz, deletes the symlinks and other unnecessary files, adds build.zig to it, and then that is the package that you depend on.
2. Zig toolchain could gain more features to control which files from a fetched package are intended to be included in the package hash. This could then exclude the symlinks, which would change the package hash, and make it work fine on Windows. This option depends on package fetching: elide errors from unpacking that relate to excluded file paths #17460 and also depends on a new feature of file filtering being added to build.zig.zon files, which I'm not promising will happen.

[Corendos]

https://learn.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/create-symbolic-links

Just to add something on top of that, a quick fix if someone encounters this issue is to enable Developer Mode. I had the AccessDenied issue on a Windows machine and enabling it solved the issue.

[jacwil]

Putting aside harfbuzz and the current implementation in the Zig build system... I suspect it would still be a problem even if the hash was calculated before the package was extracted to ensure the package matched what the maintainer expects: if the files are needed then skipping them would break the Windows build, while copying would imply the maintainer should supply a max depth to limit infinite loops.

Symlinks are not enabled on Windows by default because it is a security issue for the platform. Specifically, app developers and even the OS devs often do not have symlinks in mind. Giving users (or yourself) the permission to create symlinks can open up your dev machine to some rare attacks per the warning in the docs.

I am developing and building on a Windows machine and can actively reproduce this issue with libxml2. Specifically when consuming zig-build-libxml2:

.dependencies = .{
        .libxml2 = .{
            .url = "https://github.com/ianprime0509/zig-build-libxml2/archive/4d1b7db156b0e7a19127c652adefdf27799770ab.tar.gz",
            .hash = "122011b13203141cc965cfe6b070ffb5a8835eb906bb2cfd4650dbc17574e6e36fd5",
        },
    },

Fetch Packages [4/3] libxml2... C:\Users\Jack\AppData\Local\zig\p\122011b13203141cc965cfe6b070ffb5a8835eb906bb2cfd4650dbc17574e6e36fd5\build.zig.zon:16:20: error: unable to unpack tarball
            .url = "https://gitlab.gnome.org/GNOME/libxml2/-/archive/v2.11.5/libxml2-v2.11.5.tar.gz",
                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: unable to create symlink from 'test/relaxng/ambig_name-class.rng' to 'tutorA.rng': AccessDenied
note: unable to create symlink from 'test/relaxng/ambig_name-class2.rng' to 'ambig_name-class.xml': AccessDenied

Edit: I'm running a recent master branch build, 0.12.0-dev.2208+4debd4338.

[silbinarywolf]

Throwing this idea out there and acknowledging it's probably bad / would possibly lead to annoying/bad quirks/errors but:
What if for Windows, Zig could track the symlinks internally itself without creating them and then anytime you use a LazyPath, Zig could redirect the paths that match on known symlink to the actual folder.

[AtikoSpeed]

https://learn.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/create-symbolic-links

Just to add something on top of that, a quick fix if someone encounters this issue is to enable Developer Mode. I had the AccessDenied issue on a Windows machine and enabling it solved the issue.

thank you so much i love u

[andrewrk]

Proposed solution: #22350

[der-teufel-programming]

Proposed solution: #22350
Looks good