[Do not merge] Test migration to Actions demonstration PR

[BillyONeal]

We are looking to move everything to Actions to:

• Address compliance burden the vcpkg team currently has to keep dev.azure.com/vcpkg alive.
• Make it so that GitHub contributors don't need to click 5 times to get their build failure logs.
• Make it easier to generate console output that makes for a better PR experience for contributors in the future.

Known limitations:

• GitHub Actions has no equivalent to the test run telemetry originally contributed by @dg0yt .
• GitHub Actions makes temporary edits to test e.g. a subset of triplets more difficult.

[dg0yt]

Make it so that GitHub contributors don't need to click 5 times to get their build failure logs.

It would be great not just for vcpkg if it was easy enough to report the AZP asset download link from the pipeline job so that it can be integrated in PR conversation.

Make it easier to generate console output that makes for a better PR experience for contributors in the future.

Personally I don't see an improvement with GHA output. IMO it would be more relevant to make vcpkg tool and the port scripts highlight actual problems, including messaging out of project mode (currently ending in logs).

[dg0yt]

GitHub Actions has no equivalent to the test run telemetry originally contributed by @dg0yt .

FTR I actually use this to assess if ports are build at all or in particular configurations. Recent case: gazebo, cf. #30862 (comment).

[BillyONeal]

It would be great not just for vcpkg if it was easy enough to report the AZP asset download link from the pipeline job so that it can be integrated in PR conversation.

There are so many 'if only' jokes I can put here

Personally I don't see an improvement with GHA output.

Noted, thank you!

IMO it would be more relevant to make vcpkg tool and the port scripts highlight actual problems, including messaging out of project mode (currently ending in logs).

The problem is that right now problems don't show up on GitHub which makes arguing to improve it hard...

[BillyONeal]

@Osyotr @m-kuhn @autoantwort Tagging you three since you 👍 'd above. What are your thoughts? dg0yt's feedback is having me question if we should do this.

[m-kuhn]

I widely use gha, I hope this will improve the evolution of this stack in general. You already caught my interest with the pull_request_target to upload binary caches from pr's (still wondering about the security impact). My 👍 was motivated by this alignment with my usual tools and potential synergies. But also by the promise to make logs more accessible (not only but especially on a phone, the current way to access the logs is not ideal).

I assume you get around the usual 6 hours limit by using custom runners.

Sidenote, I also plan creating a standalone gh action or two, e.g. to create a report about port updates in manifests, cmp qgis/QGIS#59288 (comment) )

[dg0yt]

FTR I didn't meant to veto. But I'm just surprised that MS is unable or unwilling to do more for AZP.

Improving output is an issue for local usage, too.

[BillyONeal]

FTR I didn't meant to veto. But I'm just surprised that MS is unable or unwilling to do more for AZP.

Didn't interpret it as a veto. It's more "the main reason for this change is to improve UX for contributors, so if important contributors don't see it as helpful, and we are giving up useful functionality to do it, maybe we shouldn't"

Maybe the AZP folks could change things about the situation, but I assume if they cared a lot about this scenario that deep links would already work.

Improving output is an issue for local usage, too.

Agreed!

[Osyotr]

What are your thoughts?

Pros:
+ easy to access artifacts
Cons:
- Things you've mentioned in "known limitations" section (although I only use triplet filtering from time to time)
No change:
- Searching in logs is still awkward

[dg0yt]

• Things you've mentioned in "known limitations" section (although I only use triplet filtering from time to time)

The triplet filtering has two faces. One is editing the default (you and me). The other is the AZP interactive UI (AZP project owners). I would assume that "editing the default" should be possible with GHA, too.
(I know the AZP UI because I ported that feature from homegrown superbuild 😄)

[BillyONeal]

The triplet filtering has two faces. One is editing the default (you and me). The other is the AZP interactive UI (AZP project owners). I would assume that "editing the default" should be possible with GHA, too.

Unfortunately not. pull-request-target gets the .github directory from the target, not the source, branch. If we use pull-request, GitHub doesn't let us get to the caches.

[BillyONeal]

@JavierMatosD @ras0219-msft @AugP @vicroms and I discussed this today. Given that people who were not Billy did not find it blindingly more obvious how to find the failure logs, and the performance issues with linking from the checks page, in addition to the limitations above, we have decided to stick with Azure DevOps for now.

[m-kuhn]

GitHub doesn't let us get to the caches.

Doesn't let us write caches -- reading still works

[talregev]

@BillyONeal I like the idea that it will be going to github action. most of the open source project do use github action as their main ci with vcpkg, and it will be more close to them and find bugs quickly.

Will you make more effort on this direction? Thank you in advance.

[BillyONeal]

Doesn't let us write caches -- reading still works

Unfortunately not. Maximum access is 'none'. https://docs.github.com/en/actions/security-for-github-actions/security-guides/automatic-token-authentication#permissions-for-the-github_token

most of the open source project do use github action as their main ci with vcpkg, and it will be more close to them and find bugs quickly.

Most failures we have seen from GitHub Actions are 'something the default runners did broke us', but this isn't using the default runners, so that doesn't matter.

Will you make more effort on this direction? Thank you in advance.

Not anytime soon.

[m-kuhn]

Doesn't let us write caches -- reading still works

Unfortunately not. Maximum access is 'none'. https://docs.github.com/en/actions/security-for-github-actions/security-guides/automatic-token-authentication#permissions-for-the-github_token

Works for me 🤷‍♂️

Maybe we talk about something different? I use nuget caching in ghcr.

[BillyONeal]

Works for me 🤷‍♂️

Maybe we talk about something different? I use nuget caching in ghcr.

Our caches here use the Azure Blob Storage backend, so they need to be able to run azure/login@v2, which needs id-token: write.

I don't believe GitHub Container Registry has the data lifecycle management bits we need, nor support for the ~10TB of space that we need.

[talregev]

You can still run on GitHub action and think on external storage like in azure/aws storage.

[BillyONeal]

You can still run on GitHub action and think on external storage like in azure/aws storage.

That is exactly what we are doing. Doing that requires getting auth into the pipeline, which requires id-token: write, which is forbidden on the pull_request: trigger.