ci: publish the minimal SDK in a continuous release

[dscho]

Currently, when asking the setup-git-for-windows-sdk Action for the minimal SDK, it first looks at the latest successful ci-artifacts run to determine the commit, and then performs a shallow, partial & sparse checkout of the git-sdk-64 repository at that commit.

This is relatively slow, taking typically something around 25 seconds for the actual clone, compared to typically around 3 seconds to restore it from a cached version. Therefore, the Action tries to cache it, even if it sometimes adds quite an overhead (for example, here something spent 2 minutes after the cache was saved before the workflow was allowed to continue).

Not only is this taking quite some time, diminishing the developer experience, but it is also unnecessary, seeing as the ci-artifacts run already provides the minimal SDK in a ready-to-use form and uploads it as a workflow artifact.

So why not use that workflow artifact? A couple of reasons:

• It would appear as if workflow artifacts are somehow compressed as zip archives on the fly when trying to download them manually,
• While not possible at the time setup-for-windows-sdk was designed, it now seems possible to use actions/download-artifact to download from other repositories, but:
• Workflow run artifacts obey a limited retention policy, which is undesirable here, and finally:
• You can only download workflow artifacts when authenticated, while clones can be performed anonymously; There are likely use cases where the minimal SDK is needed outside of GitHub Actions (for example in other CI systems).

I once looked into publishing the minimal SDK continuously, first considering GitHub Packages. But they have rather strict retention rules and we definitely do not need to retain anything but the latest working minimal-sdk artifact. (And I don't want to waste resources unnecessarily, even if someone else pays for them.)

Then I had looked into uploading that artifact to Azure Blobs, but had to pull the plug in one big hurry because it threatened to blow my Azure budget.

So eventually I gave up on that plan and instead fell back to the "partial, shallow, sparse clone & then cache" strategy, which is in place to this day.

However, people who are more creative than I am figured out ways to publish artifacts continuously. I just had not found those solutions yet at the time. In the MSYS2 project, for example, they have a srcinfo artifact that is updated continuously, and initially they used eine/tip to publish it in a GitHub release that is kept up to date by replacing the assets as new revisions come in. Nowadays they use the same strategy, albeit manually via gh release upload --clobber.

This here PR uses a slight variation of the same strategy: Since --clobber basically deletes the asset before uploading a new version, there is a time window where the asset is not available. Since the .tar.gz file is around 95MB, uploading it can take a while, in particular given the vagaries of any network operation. For that reason, in this PR the upload is done to a temporary name, then the old asset is deleted, and finally the new asset is renamed to the old name. This does not avoid the window of a missing asset altogether, but at least it makes it a lot smaller.

Finally, I took this opportunity to generate and publish .zip and .7z.exe variants, the latter being a self-extracting 7-Zip archive that does not require any special tools to extract. Granted, since a native BSD tar is available in C:\Windows\system32\tar.exe on pretty much any non-ancient Windows version, and that tar.exe can extract both .tar.gz and .zip archives without problems, that seems to be lesser of an issue. But still, there's things like Windows Nano Server that may not have that BSD tar.

tl;dr with this PR, the ci-artifacts workflow runs not only verify that the minimal SDK is able to compile a working git.exe, but then also publishes the minimal SDK in various variants in a git-sdk-64 GitHub release called ci-artifacts whose tag will point to the corresponding commit.

[dscho]

For the record, here is the ci-artifacts release in my personal fork that was done with the code presented here (plus a small commit on top to convince the workflow to run and publish in a fork).

[dennisameling]

Wow, your PR description is truly outstanding! Thank you so much for picking this up.

This change looks incredibly promising and exciting to me - I wouldn't be surprised if this reduces the runtime of the setup-git-for-windows-sdk action from minutes (without cache) to mere seconds. Looking forward to testing it there!!