Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jepsen tests to CI workflow #2774

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open

Conversation

pcholakov
Copy link
Contributor

@pcholakov pcholakov commented Feb 24, 2025

A long time coming but finally ready to merge - adds a step to run the https://github.com/restatedev/jepsen suite (and specifically, the set-vo and set-mds workloads) against open PRs.

@pcholakov pcholakov force-pushed the chore/integrate-jepsen-checks branch from 56b3194 to 2e79431 Compare February 24, 2025 08:18
Copy link

github-actions bot commented Feb 24, 2025

Test Results

  7 files  ±0    7 suites  ±0   3m 19s ⏱️ -2s
 45 tests ±0   44 ✅ ±0  1 💤 ±0  0 ❌ ±0 
174 runs  ±0  171 ✅ ±0  3 💤 ±0  0 ❌ ±0 

Results for commit 8116cc0. ± Comparison against base commit 622f4b0.

♻️ This comment has been updated with latest results.

@pcholakov pcholakov force-pushed the chore/integrate-jepsen-checks branch from 1eac129 to 923afd0 Compare February 24, 2025 11:38
@pcholakov pcholakov changed the title Add Jepsen tests to CI workflow (main repo branch) Add Jepsen tests to CI workflow Feb 24, 2025
@pcholakov pcholakov added the jepsen-tests PRs run Jepsen tests in CI label Feb 25, 2025
@pcholakov pcholakov marked this pull request as ready for review February 26, 2025 13:32
Comment on lines +99 to +100
# additional features added for CI validation builds only
features: metadata-api
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only enabled on the internal docker artifact we attach to the workflow run - same one that gets used by the SDK tests.

Comment on lines +236 to +237
jepsen:
needs: docker
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test currently runs after docker, just like the SDK tests. At the moment it takes ~3 min but we can tune it to run for shorter or longer. The downside is that there's a single Jepsen workers cluster backing it so we need to run multiple PRs sequentially. As long as there isn't a big backlog of PRs, it shouldn't add any more time to the overall PR checks duration.

id-token: write # NB: can obtain OIDC tokens on behalf of this repository!
steps:
- uses: restatedev/jepsen/.github/actions/run-tests@reusable
continue-on-error: true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect this will be flaky for a while - I believe this will make it report error but not block PR merging. But, I may be wrong about how this behaves :-)

@pcholakov pcholakov force-pushed the chore/integrate-jepsen-checks branch from 9737d1c to 8116cc0 Compare February 27, 2025 04:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
jepsen-tests PRs run Jepsen tests in CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant