Update codecov config to support commenting in PR

[henrybear327]

• add missing directory fixing go.etcd.io/etcd/etcdutl/v3/::etcdutl/
• update codecov config to support commenting in PR

Reference:

• Worrying trend with test coverage #18131
• https://docs.codecov.com/docs/unexpected-coverage-changes#reasons-for-indirect-changes

Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

[henrybear327]

@serathius I was checking other project's codecov configuration file and the official documentation, we should consider putting codecov token in Github Secret (maybe we need to regenerate a new token first, but we need to have access to the codecov account, which I don't know who to ping).

I can't do this on my own since I don't have Github Repo Setting access, so this part I might need your help to look into this!

Thanks

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.83%. Comparing base (5790774) to head (6615b5e).

❗ Current head 6615b5e differs from pull request most recent head 1579c57

Please upload reports for the commit 1579c57 to get more accurate results.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

see 22 files with indirect coverage changes

@@            Coverage Diff             @@
##             main   #18143      +/-   ##
==========================================
- Coverage   69.08%   68.83%   -0.25%     
==========================================
  Files         416      416              
  Lines       35127    35127              
==========================================
- Hits        24266    24181      -85     
- Misses       9475     9544      +69     
- Partials     1386     1402      +16     

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5790774...1579c57. Read the comment docs.

[jmhbnz]

/retest

[jmhbnz]

LGTM - Thanks @henrybear327, the preview above in this pr is looking good.

[henrybear327]

LGTM - Thanks @henrybear327, the preview above in this pr is looking good.

Thanks for reviewing @jmhbnz! :)

Currently, the Codecov comment is set to update the existing code coverage comment when a new commit is pushed (default behavior), instead of deleting the existing comment and creating a new one.

Using the default behavior means that the code coverage information will likely be at the top of the comment, so PRs with long discussion threads will require some scrolling to view the code coverage information. Deleting the old one and adding a new comment with each commit will generate more email notifications though.

In order to avoid email spamming, I think it's better that we use the default behavior.

What do you think? :)

[henrybear327]

/retest

/cc @serathius @ahrtr @wenjiaswe

[ahrtr]

see 20 files with indirect coverage changes

Thanks @henrybear327 for the PR. But I am curious why the coverage decreased for many go files even this PR doesn't change any go source code?

[jmhbnz]

In order to avoid email spamming, I think it's better that we use the default behavior.

Agreed. Let's keep it to one coverage comment per pr that is always up to date.

[henrybear327]

see 20 files with indirect coverage changes

Thanks @henrybear327 for the PR. But I am curious why the coverage decreased for many go files even this PR doesn't change any go source code?

@ahrtr I suspect it's due to rebasing. I will look into this!

[wenjiaswe]

/lgtm thank you @henrybear327 !

[henrybear327]

see 20 files with indirect coverage changes

Thanks @henrybear327 for the PR. But I am curious why the coverage decreased for many go files even this PR doesn't change any go source code?

@ahrtr I suspect it's due to rebasing. I will look into this!

This is due to indirect changes [1].

I suspect that during test coverage execution, we have some of the tests written in a way that is not deterministic. Because when I check the "indirect changes" of PRs from this list, hit/miss differences come from things like select cases (e.g. error handling, timeout, etc.), cache eviction, etc.

So our test coverage percentage will have a small variation across runs, as you can see here.

Do you think this makes sense @ahrtr?

Reference:
[1] https://docs.codecov.com/docs/unexpected-coverage-changes#reasons-for-indirect-changes

[ahrtr]

[1] https://docs.codecov.com/docs/unexpected-coverage-changes#reasons-for-indirect-changes

Thanks for the clarification, which basically makes sense. Can you pls add a comment to include the link for reference?

[henrybear327]

[1] https://docs.codecov.com/docs/unexpected-coverage-changes#reasons-for-indirect-changes

Thanks for the clarification, which basically makes sense. Can you pls add a comment to include the link for reference?

Thanks @ahrtr! I have

• updated the PR description
• updated the commit message
• added link to the discussion comment in the issue

Not sure if this is what you expected :)

[ahrtr]

Sorry for the confusion, I was saying to add a comment into the repo, e.g. above the cov_pass

etcd/scripts/test.sh

Line 263 in 5790774

function cov_pass {

[henrybear327]

https://docs.codecov.com/docs/unexpected-coverage-changes#reasons-for-indirect-changes

No worries :)

Done as requested! Thanks

[ahrtr]

LGTM

Thanks @henrybear327