Cannot read annotations for a previous attempt of a workflow

[timmc-edx]

Select Topic Area

Product Feedback

Body

Summary

The API endpoint https://github.com/openedx/edx-platform/actions/runs/#####/attempts/# only allows accessing annotations for the most recent attempt, no matter which attempt you select. This appears to be a bug in the API and does not match the UI.

Details

I'm trying to get statistics on how often we're seeing a certain failure in GitHub Actions runs. I should be able to use the Check Runs API to get the annotations for the workflow run. The problem is that people often re-run their CI checks when they see this failure, so the most recent attempt of the workflow is a success, and the API apparently only allows you to see the last check run output. (The web UI allows you to see all of the check runs.)

Here's an example:

• https://github.com/openedx/edx-platform/actions/runs/5476653975/attempts/1 shows the target failure. Job python-3.8,django-pinned,lms-5 has the annotation "The self-hosted runner: edx-platform-openedx-ci-runner-deployment-7xdl7-twcfc lost communication with the server." If you switch to attempt # 2 using the UI, you'll see that it succeeded.
• When I ask for https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/1 and .../attempts/2, the results differ in only a few places. Here are the important differences between the attempts:

  @@ -12,1 +12,1 @@
  -  "conclusion": "failure",
  +  "conclusion": "success",
  @@ -295,1 +295,1 @@
  -  "run_attempt": 1,
  +  "run_attempt": 2,
  @@ -321,1 +321,1 @@
  -  "logs_url": "https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/1/logs",
  +  "logs_url": "https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/2/logs",
  @@ -326 +326 @@
  -  "previous_attempt_url": null,
  +  "previous_attempt_url": "https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/1",

  Most notably, they share a check_suite_url, which only shows the successful checks from the second attempt. (They do have different log URLs, but the logs don't show this kind of failure.)
• Asking for all check runs of the commit (https://api.github.com/repos/openedx/edx-platform/commits/ab3d322bc9/check-runs) again shows this most recent check suite, but not the first attempt.

(This is a re-post of https://github.com/orgs/community/discussions/103026 into "Product Feedback" rather than "Questions"—it was originally a "how do I do this" question, but given the lack of responses I think the answer is "you can't".)

[timmc-edx]

Can you show an example of that workaround in use? As far as I can tell, there is no such workaround.

[Carlostoledof]

Help

[alvicsam]

Hey @timmc-edx I'm currently researching the same issue and maybe this information might help.

It's possible to download logs from any attempt like this:

curl -L -v -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/1/logs --output logs.zip

Then inside of the logs.zip you can find the file with the text that you see in the annotation of the current attempt. The filename should be something like 0_<name_of_the_failed_job>.txt in the root of the archive.

[timmc-edx]

That only appears to show me the logs for a job, not the annotations for the workflow. Were you looking into the same kind of "lost communication with the server" error annotation, or something else?

[alvicsam]

Not exactly, but I was also looking for an error (The runner has received a shutdown signal.) that is written in annotation and I could find the text of the error in those logs.

[timmc-edx]

Ah, OK. In my case it's just a complete connection drop, so there's nothing in the logs -- it's only recorded in the annotations. (And you can only get annotations for the most recent attempt.)