v4 fails to upload: "Failed to properly create commit"

[jmgrady]

After updating codecov/codecov-action to v4 in our GitHub workflows, the coverage reports fail to upload. The upload fails because of a connection error or timeout. The GitHub Action output is:

Run codecov/codecov-action@f30e4959ba63075080d4f7f90cacc18d9f3fafd7
  with:
    token: ***
    fail_ci_if_error: true
    files: coverage.cobertura.xml
    flags: backend
    name: Backend
    verbose: true
==> linux OS detected
https://cli.codecov.io/latest/linux/codecov.SHA[2](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:2)56SUM
Received SHA256SUM 10[3](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:3)bfefcc56f76[4](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:4)73179e600b96eb81[5](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:5)0b0f349ad9483[6](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:6)b0f63f03ffac469ad[7](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:7)  codecov
Received SHA256SUM signature -----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEJwNOf9uFDgu[8](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:8)LGL/gGuyiu13mGkFAmW8+QUACgkQgGuyiu13
mGll6A/8CVDQ3LQaqpgQEIekCJUNKU/+tpF1EGpuUsMi7DZZ0RA6cmazUJFUlZK2
GS7Ptcwly+SYTT+a61SzeVlewPLa47XZIFyC+7tgSO6XgUNlJ+KTevyeL6GG0Qs1
YaciuJv8IHAqaaYT4[9](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:10)iKpWE99OME1VXY3fIm4uEHlc4pgnmLv+FNdxnit4AYLGlw
2JHzdDMd5aHlnYYIyYJ9UbM8fFVWgddL1venoYl59NKc5gXjH1/+yPPWz2R+4f22
jZfofI04aEJxNhGinfV5Vykb9asyfMupmLUweArgTIF3wzIEoYo/pK0nVgRBiROE
1hiaH5lti8brA2NF+pzp4+xFEyCU1m/mgN/rj1VRkDs0CW4S86eYWVbfuHEb0ymQ
o7Oe/rST6IjqM72B8eleEdT1DKdeX8DYSXnKvR2J1POyoPMMt/HUoCxphiNOq6Ei
416xOCgqWwOEGbeZ1pxp4Eovf6fffbd2F9MUcJTzgWocOqLh3lZB/EX3G9eLKsyf
WLW3s3NXzajS9j4zEPZxGPmnqAadfm9dzffwokFZnqMTJ7HB4qXH87BPgwPbq8R+
y/1Hv7hHEbJJhbE1fitwHFMg4gfUP6q39VtrfooQSBRYYDzrstowO/L2xsr+AkwW
EQGY70I4SY4XPKKzs8tF3Jch7Oa+xgeNMM1qSGyb2Vjn0KKSDMk=
=jEIQ
-----END PGP SIGNATURE-----

==> Running version latest
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: /home/runner/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1

gpg: Signature made Fri Feb  2 14:15:33 2024 UTC
gpg:                using RSA key 27034E7FDB850E0BBC2C62FF806BB28AED779869
gpg: Good signature from "Codecov Uploader (Codecov Uploader Verification Key) <[email protected]>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 2703 4E7F DB85 0E0B BC2C  62FF 806B B28A ED77 9869

==> Running version v0.4.6
==> Running command '/home/runner/work/_actions/codecov/codecov-action/f30e4959ba63075080d4f7f90cacc18d9f3fafd7/dist/codecov -v create-commit'
/home/runner/work/_actions/codecov/codecov-action/f30e4959ba63075080d4f7f90cacc18d9f3fafd7/dist/codecov -v create-commit -C 3ae8c57c7413fde413ca1f6c254dfcc18b29b6ce
==> Uploader SHASUM verified ([10](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:11)3bfefcc56f76473179e600b96eb8150b0f349ad94836b0f63f03ffac469ad7  codecov)
info - 2024-02-06 19:24:32,215 -- ci service found: github-actions
debug - 2024-02-06 19:24:32,215 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-02-06 19:24:32,216 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-02-06 19:24:32,219 -- Loading config from /home/runner/work/TheCombine/TheCombine/codecov.yml
debug - 2024-02-06 19:24:32,227 -- Starting create commit process --- {"commit_sha": "3ae8c57c7413fde413ca1f6c254dfcc18b29b6ce", "parent_sha": null, "pr": "2941", "branch": "codecov-v4", "slug": "sillsdev/TheCombine", "token": "9******************", "service": "github", "enterprise_url": null}
warning - 2024-02-06 19:24:33,252 -- Request failed. Retrying --- {"retry": 0}
warning - 2024-02-06 19:24:34,756 -- Request failed. Retrying --- {"retry": 1}
warning - 2024-02-06 19:24:36,772 -- Request failed. Retrying --- {"retry": 2}
Traceback (most recent call last):
  File "codecov_cli/main.py", line 81, in <module>
  File "codecov_cli/main.py", line 77, in run
  File "click/core.py", line [11](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:12)57, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "click/decorators.py", line 33, in new_func
  File "codecov_cli/commands/commit.py", line 64, in create_commit
  File "codecov_cli/services/commit/__init__.py", line 28, in create_commit_logic
  File "codecov_cli/services/commit/__init__.py", line 64, in send_commit_data
  File "codecov_cli/helpers/request.py", line 65, in wrapper
Exception: Request failed after too many retries
[21[12](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:13)] Failed to execute script 'main' due to unhandled exception!
Error: Codecov: Failed to properly create commit: The process '/home/runner/work/_actions/codecov/codecov-action/f30e4959ba63075080d4f7f90cacc[18](https://github.com/sillsdev/TheCombine/actions/runs/7804991065/job/21288230413?pr=2941#step:6:19)d9f3fafd7/dist/codecov' failed with exit code 1

[gmazzo]

Same here, Codecov still states this is not required for public projects, but bumping to v4 fails to publish it

[dereuromark]

Same here
https://github.com/cakephp/localized/actions/runs/7790301113
Used to work fine with v3

[bkoelman]

I'm getting this as well.

[thomasrockhu-codecov]

I believe this is due to this incident. @dereuromark are you still getting the issue?

[dereuromark]

Yes, reran the workflow just now, same thing
https://github.com/cakephp/localized/actions/runs/7806912790

[Luthaf]

Same issue here, using codecov v4 and a PR from a fork: https://github.com/lab-cosmo/metatensor/actions/runs/7817731428/job/21326326361

EDIT: seems fixed after a restart, might have been transient. Here is the old log for reference:

==> Uploader SHASUM verified (103bfefcc56f76473179e600b96eb8150b0f349ad94836b0f63f03ffac469ad7  codecov)
info - 2024-02-07 16:21:44,309 -- ci service found: github-actions
info - 2024-02-07 16:21:44,504 -- The PR is happening in a forked repo. Using tokenless upload.
info - 2024-02-07 16:21:44,733 -- Process Commit creating complete
error - 2024-02-07 16:21:44,733 -- Commit creating failed: {"error": "Server Error (500)"}
Traceback (most recent call last):
  File "codecov_cli/main.py", line 81, in <module>
  File "codecov_cli/main.py", line 77, in run
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "click/decorators.py", line 33, in new_func
  File "codecov_cli/commands/commit.py", line 64, in create_commit
  File "codecov_cli/services/commit/__init__.py", line 39, in create_commit_logic
  File "codecov_cli/helpers/request.py", line 133, in log_warnings_and_errors_if_any
NameError: name 'exit' is not defined
[4757] Failed to execute script 'main' due to unhandled exception!
Error: Codecov: Failed to properly create commit: The process '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov' failed with exit code 1

[thomasrockhu-codecov]

@dereuromark you need to add the Codecov token

Error: Codecov token not found. Please provide Codecov token with -t flag.

Instructions to do so are here

[gdalle]

Getting the same server error on all my repos

Example: https://github.com/gdalle/DifferentiationInterface.jl/actions/runs/7818651755/job/21329350528

[dereuromark]

you need to add the Codecov token

We never had to enter any tokens for the last decade.
Since those are all open source repos.
And I sure dont want to add them for 30+ repos I manage.. There must be a better way.

Note that v3 is working just fine
I would expect cakephp/localized@d8125f3 to be the only change needed to continue working with v4.

[Czaki]

And I sure dont want to add them for 30+ repos I manage.. There must be a better way.

Remember that secrets are not passed to workflow triggered by PR from fork.

[dereuromark]

Remember that secrets are not passed to workflow triggered by PR from fork.

Both PR and main branch CI worked so far with v3 afaik

[Czaki]

Yes. I point a problem with v4. Where token will not help for typical OSS.

[rohan-at-sentry]

Hi @Czaki @dereuromark something I'd like to clarify here

On requiring a token
This is primarily a performance need. Codecov works by making an API call to GitHub to confirm that the repo and commit are the correct values. Making this call for thousands of repositories causes our GitHub token to hit the limit causing the issues that many of you may have seen - see codecov/feedback#126 as an example

On impacting contribution flows

We're aware that for open source contributors, the fork->commit-> PR workflow (which is by far the most common) would be impacted if we enforced token usage aggresively, so currently we DO NOT require forked repos to setup a token. You can read more on our blog (look for the section called Future of tokenless

On adding a single token for multiple repos

This usecase is served by using Codecov's GLOBAL UPLOAD TOKEN. Here's how to set that up (docs)

I hope this helps, please don't hesitate to reach out if you have challenges with this.

[Czaki]

@rohan-at-sentry
This is PR to update codecov to version 4 in my repository 4DNucleome/PartSeg#1066
As you can see, it is open source, and it is failing. This repository has already set up the codecov token and PR is done from the same repository (dependabot creates a branch from the same repository in my case). And even on PR from the same repo, the secret is not passed.

I try to play with workflow_run trigger. Like this:

on:
  workflow_run:
    workflows: [Tests]
    types:
      - completed

But it ends with codecov failing to determine PR/commit.

If you could fix codecov action to work in workflow triggered by workflow_run, then it is possible to provide tokens for the uploader. And provide extensible instructions on how to upload codevov results using action/upload@v4 and download the results in the next workflow (triggered by workflow_run) to have a properly working upload.

[gdalle]

This usecase is served by using Codecov's GLOBAL UPLOAD TOKEN. Here's how to set that up (docs)

Is there a way to have a global token for a user, and not just an org?

[rohan-at-sentry]

@Czaki I suspect that the token is not being set correctly in that example you shared.

Is it possible for you to try the following and see if it works

name: Upload coverage reports to Codecov
      uses: codecov/codecov-action@v4
      with:
        verbose: true
      env:
        CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

[rohan-at-sentry]

@gdalle

Is there a way to have a global token for a user, and not just an org?

Not currently... Can you describe your usecase? Maybe there's a way we can already help now

[dereuromark]

Not currently... Can you describe your usecase? Maybe there's a way we can already help now

In my case I have 30+ repos in my github user account (personal), is there a way to set up the env token once across all of them? I only saw sth like that for an org.

Sorry to chime in.

[Czaki]

@rohan-at-sentry Why use env variable instead of pass in as a parameter?

[rohan-at-sentry]

Oh I think I misunderstood the original question.

@dereuromark - I believe if you navigate to https://app.codecov.io/account/gh/<your_GH_username> , and click the settings tab, you should find the ability to set the Global Upload Token

@gdalle if your usecase is similar to above, then you can try this as well

[dereuromark]

I believe if you navigate to https://app.codecov.io/account/gh/<your_GH_username> , and click the settings tab, you should find the ability to set the Global Upload Token

Well, I generated a token there, not setting one
But then I still need to put it somewhere, from my understanding this would have to be on github side if you want it to work with

- uses: codecov/codecov-action@v4
  with:
    token: ${{ secrets.CODECOV_TOKEN }}

in all 30+ repos of my github account and CI

[Czaki]

@rohan-at-sentry After adding verbose, the upload passed. But I'm not sure if it is a proper solution.

[giovanni-guidini]

@Czaki if you are referring to 4DNucleome/PartSeg#1066 when you say adding verbose fixed it,
(that you mentioned in this comment)

I don't think it was the verbose option that made the upload run. I think it was the fact that it was your user (or pre-commit ci app) triggered the workflow that happen to have the verbose option in it.

Apparently dependabot can't access repo secrets (see here)

so the action didn't have the token to upload when dependabot tried to run it. Because that PR comes from the same repo, it needs a token (and so it failed).

On the other hand, when your user (or pre-commit ci app) triggered the action would have access to the token, and the upload would then run.

[dereuromark]

Apart from my personal account, which cannot be fixed for now, I also tried the org approach
Here, I set a global secret and it seems to pull it into the repos and so far it also seems to work
But the final lines of the v4 action give a bit mixed feedback:
https://github.com/php-collective/decimal-object/actions/runs/7836624847/job/21384669898#step:6:50

==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov do-upload'
/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov do-upload -C 0e00248c6eefc7cfcbf5b8d59662058d4f96a46e
info - 2024-02-08 22:12:04,446 -- ci service found: github-actions
warning - 2024-02-08 22:12:04,449 -- No config file could be found. Ignoring config.
warning - 2024-02-08 22:12:04,455 -- xcrun is not installed or can't be found.
warning - 2024-02-08 22:12:04,496 -- No gcov data found.
warning - 2024-02-08 22:12:04,496 -- coverage.py is not installed or can't be found.
info - 2024-02-08 22:12:04,506 -- Found 1 coverage files to upload
info - 2024-02-08 22:12:04,506 -- > /home/runner/work/decimal-object/decimal-object/coverage.xml
info - 2024-02-08 22:12:04,616 -- Process Upload complete
error - 2024-02-08 22:12:04,617 -- Upload failed: {"detail":"You do not have permission to perform this action."}

The last message sounds like it actually still failed?
And apparently, inside codecov backend, the commit never arrived.
Previous PRs in the old action worked.

[Czaki]

it looks working now. I will provide it more time for test.

[thomasrockhu-codecov]

@jmgrady can you confirm this is working for you now?

For everyone else, please note that this thread is getting a little bit crazy. There are a few root causes that we have since patched. If you are still experiencing this issue, please open a new issue so that we can track.

[jmgrady]

@thomasrockhu-codecov
I apologize for not replying earlier. I missed your question earlier but have now fixed my GitHub config.

I am still seeing the problem. I am using codecov/[email protected]:

      - name: Upload coverage report
        uses: codecov/[email protected]
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          fail_ci_if_error: true
          files: coverage.cobertura.xml
          flags: backend
          name: Backend

The failure message matches the one in my original post.

[AlexSkrypnyk]

Use env in GHA to pass the value from secrets:

      - name: Upload coverage report to Codecov
        uses: codecov/codecov-action@v4
        with:
          directory: /tmp/.scaffold-coverage-html
          fail_ci_if_error: true
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}