Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test results are attributed to the wrong workflow #181

Closed
TWiStErRob opened this issue Oct 31, 2021 · 6 comments
Closed

Test results are attributed to the wrong workflow #181

TWiStErRob opened this issue Oct 31, 2021 · 6 comments

Comments

@TWiStErRob
Copy link

TWiStErRob commented Oct 31, 2021

My workflow setup: https://github.com/TWiStErRob/net.twisterrob.healthcheck/tree/master/.github/workflows

  • CI uses tests-run.yml (All and Smoke tests)
  • Schedule uses tests-run.yml (cinema, regex, www, uw tests)
  • tests-run.yml uses EnricoMi/publish-unit-test-result-action@v1

Which means that each result should be under the corresponding workflow that uses test-run.yml, but all of them are under CI:
TWiStErRob/net.twisterrob.healthcheck#9
image

or earlier in another commit it was all under Scheduled (since then I renamed the jobs):
image

@EnricoMi
Copy link
Owner

This is a known limitation of the GitHub Check Status API. GitHub attributes the Check Status to an arbitrary workflow, not the one that is running the action that calls the Check Status API. This is annoying, I know. See #12.

@TWiStErRob
Copy link
Author

Wow, that's quite a non-response from GitHub. Thanks, hope there will be something happening. I'll submit a support request too to get it on their radar more.
For ref: https://github.community/t/specify-check-suite-when-creating-a-checkrun/118380/16

@TWiStErRob
Copy link
Author

Trimmed down version of their response, sounds more promising than is visible at github.community:

We have had a number of users write in about this and the team is currently working on improving this behavior.
Currently, there isn't any public issue available for tracking. We have internal discussions taking place with our developers to form a solution to this problem. I have added your report to it for tracking.
While I do not have an ETA for exactly when we'll see changes made, I can assure you there's been much activity on this issue recently.
We usually announce new features as they become available on the ChangeLog page of our Blog. You can keep an eye on these for when it becomes available.

@EnricoMi EnricoMi pinned this issue Nov 1, 2021
@TWiStErRob
Copy link
Author

I guess this is the same issue: but with timings days apart.

I have a failed scheduled nightly workflow run
https://github.com/TWiStErRob/net.twisterrob.healthcheck/actions/runs/1415459852

With a failed test in a job
https://github.com/TWiStErRob/net.twisterrob.healthcheck/runs/4089035369

Which reports its annotations to the CI run of the last commit on master.
https://github.com/TWiStErRob/net.twisterrob.healthcheck/runs/4089052737

The consequence of this is that I can't see the failed test info on the scheduled run until I click around and find the last master commit's annotations.

@EnricoMi EnricoMi unpinned this issue Nov 13, 2021
@remcowesterhoud
Copy link

I'm running into this issue now. Did anyone ever find a solution for this? It seems GitHub still didn't do anything about this limitation.

@EnricoMi
Copy link
Owner

EnricoMi commented Jun 5, 2023

The best workaround that community has come up with is https://github.com/orgs/community/discussions/24616#discussioncomment-5607870.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants