Skip to content
This repository has been archived by the owner on Dec 1, 2023. It is now read-only.

Automate disabling flaky tests #60

Open
galargh opened this issue Nov 14, 2022 · 1 comment
Open

Automate disabling flaky tests #60

galargh opened this issue Nov 14, 2022 · 1 comment

Comments

@galargh
Copy link
Contributor

galargh commented Nov 14, 2022

Flaky test lowers make CI unpredictable. You have to review the CI run and compare it with other runs to actually accept a PR (for example with bitswap PR).

  • Identify, fix, or disable flaky test.

https://engineering.fb.com/2020/12/10/developer-tools/probabilistic-flakiness/

Things we can do now:

Monitor CI run and identify flaky tests for go-bitswap (the one case I, Laurent, knows about)

Monitor CI runs and identify flaky tests for project xxxx

Things that would help:

Reach out to maintainers and raise the discussion about disabling flaky tests entirely

Things we can do later

Automate Flakyness measure & report

Related PRs

ipfs/boxo#86

@galargh
Copy link
Contributor Author

galargh commented Nov 14, 2022

This should build up and be linked to Marco's work on flaky tests in go-libp2p.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Status: 🥺 Backlog
Development

No branches or pull requests

1 participant