Several workflows but only one produces test results

[kpturner]

I have three workflows. All three produce unit test results and all three try to publish them with this action. Only one of them actually publishes anything. The others publish nothing despite showing no errors. Similarly, when all three workflows run as checks as the result of a push, only one set of results are published, and against a completely random workflow that doesn't even use this action. Am I missing something specific that needs to be added to the inputs?

[kpturner]

logs_1724.zip

No errors and no results published

[EnricoMi]

Do all three actions use the same value for check_name? Then they will overwrite each other's results. Use different values there to distinguish between the three workflows and consecutive runs of each of the workflows.

Are your workflows in a public repo so I can take a closer look?

The random workflow is a known issue with GitHubs AP (#12) and out of our control.

[kpturner]

Oh wow! Didn't expect such a quick response. None of them specify check_name so they all use the default. I will give that a go thanks. I am afraid they are not public.

I found that other issue too. How annoying :(

[kpturner]

So I used a unique check_name but it didn't seem to help. I am testing it by starting all three with workflow_dispatch. The workflows run concurrently on a load balanced EC2 instance (maybe that is the issue?). They each upload their unit tests results as artifacts. Then a separate job starts on ubuntu-latest that downloads the artifacts and publishes them.
The location and names of the artifacts are unique for each workflow.

I only get results published for the first one workflow I start. The other two never produce anything.

If I change the order that I start them in, then the first one works always, even though it didn't work when it was run second or third.

[kpturner]

I tried running two workflows and they both ran concurrently, both uploaded uniquely names artifacts, downloaded them, ran this action but produced no output. Very weird.

[kpturner]

I have now added the github.run_id to the check_name and I am now getting results for two out of three.

[EnricoMi]

Another thing to try is setting the log level for all of the three actions as follows:

with:
  log_level: DEBUG
  root_log_level: DEBUG

Then lets compare the log output of all three actions (the log zip you added before).

If that does not provide any insights, could you provide the workflow yaml files so I can reproduce it in a public repo? You said they start due to a workflow_dispatch, all on the same workflow_dispatch event, or do you trigger them individually (three times)?

[kpturner]

Triggered three times (individually).

I will try this now....

[kpturner]

Worked
logs_1737.zip
Worked
logs_1738.zip
Says it worked but can't find results anywhere
logs_1739.zip

[EnricoMi]

Your third action's check_name is check_name: FDEL Unit Test Results$ {{ github.run_id }}, I think it should be check_name: FDEL Unit Test Results ${{ github.run_id }}.

[EnricoMi]

Do you run the workflow_dispatch on master or a branch?

[EnricoMi]

I have tried to reproduce this in my playground repo EnricoMi/python. There are two manual trigger workflows: https://github.com/EnricoMi/python/tree/master/.github/workflows

I triggered them to run on master concurrently. You can navigate to the results as follows:

1. Go to the commit history of master: https://github.com/EnricoMi/python/commits/master
2. Click on the ❌, scroll down to the result tests and click on Details:
   
3. This brings you to the one test result, which may be at a random workflow but all actions' output should be at the same workflow. You can also see all workflows that ran on that commit when you follow that link. If you click on any Details link that is not a test result, you end up in a specific workflow, not all of them. That is GitHub magic.

My two manual workflows have individual check_name values but fixed across runs (no github.run_id). I would suspect you to find all action results in the same way in your repo.

[kpturner]

That doesn't really work for me I am afraid - even if it did it is not a very intuitive way to find the test results. There is no relationship between a manual workflow dispatch and a commit. For example, I could run a manual workflow dispatch on a branch that has no commits on it at all.

So when I look through the commit history on this branch there is no route to see test results to click on:

The only route I know of to seeing test results of a manual workflow is to navigate to that workflow run. Here are my three that you have logs for

No results here. This was the last one to finish. Where would I look? It is not attached to any workflow and is not relevant to any commit

This one shows two results of three

This one has results that show on the previous run

So of two runs, I cannot find any results for the one that took the longest to finish.

[EnricoMi]

Thanks for the images, this is very helpful to see what you see. It is interesting that you do not have any check results behind the commits, see the ✔️ behind the "committed 3 days ago":

An alternative route is the following:

1. click on the commit sha:
   
2. click on the ✔️ before the commit message:
   
   

If all this does not work for you, try the following route:

1. Go to your actions
2. Click on a workflow_dispatch action run
3. Click on the Publish Unit Tests Results job
   
4. Remove the ?check_suite_focus=true from the URL
   
5. You should see all workflows that ran on the same commit as your action

The results should be listed under one of these workflows.

[kpturner]

Blimey - that is sure convoluted. I did find them that way. Not sure it is the answer my team will want though :D

I am thinking I might try just putting the three runs in a composite action and publish all the results just once.

[EnricoMi]

You can have the build and test job of all three workflows in one workflow as separate jobs and have a single publish job. That is probably what you meant with composite, but "composite action" means something different in GitHub.

I am glad you found your results. The workflow, runs and check model of GitHub is frustratingly complex. Well, convoluted is really the right term here.

We are all set then?

[kpturner]

Yes I know what a composite action is :)

I created a single private composite action (that does a build/test/upload) that accepts an input for the thing it is going to operate on. So instead of three separate workflows I just have one that calls the composite action for each of the components - then has one single publication job at the end for the whole lot. It is not ideal, because it is aggregating all the results together. I probably need to give the XML output unique names to work around that (at the moment the XML files are the same name in different folders).

It isn't a great solution but I think we can close this if there is no other way around it.

[kpturner]

That is a better compromise :)

[EnricoMi]

You can have three "builds and tests" jobs and have the publish job depend on all three. This way the three jobs run in parallel.

Anyway, glad we could solve the where-are-the-results mystery.