Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OLD] Run tier tests on Testing Farm #340

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ansasaki
Copy link
Contributor

@ansasaki ansasaki commented Mar 14, 2023

Add a reusable workflow that schedule tests on testing farm using the CLI. If the specified plan is not found, the test is ignored and passes.

The tier tests are executed sequentially using the reusable workflow. The tests from the tier1.yaml plan are executed on a single distro, currently Fedora 37.

If the 'tier' input is set as 'tier2', the tests from the tier2.yaml plan are executed in multiple distros, currently Fedora 37 and 38.

To avoid the need for every contributor to have their own Testing Farm token, the following 2-step trick is implemented.

  1. A workflow is triggered when a PR is opened. This workflow will not actually run the tests as the repository secrets are not accessible. It will only store the PR info as an artifact.
  2. A workflow is triggered when the first workflow is completed. This will actually run the tests as it will have access to the secrets, but it will use the workflow code from the repository (i.e. code already merged). This means that PRs that changes workflows cannot be tested this way (will only be effective after merging). Since this second workflow is not tied with a PR (as it was triggered not by the "pull_request" event, but by the finish of the first workflow), the results of the tests will not be shown on the PR that originated the tests. To make that connection, we need to implement the "check" API from github, to show the test results on the PR.

@ansasaki ansasaki force-pushed the test_tiers branch 17 times, most recently from 09a1426 to 45d7f5e Compare March 14, 2023 17:43
@ansasaki ansasaki force-pushed the test_tiers branch 13 times, most recently from fadde24 to 098e139 Compare March 24, 2023 17:08
Add a reusable workflow that schedule tests on testing farm using the
CLI. If the specified plan is not found, the test is ignored and passes.

The tier tests are executed sequentially using the reusable workflow.
The tests from the tier1.yaml plan are executed on a single distro,
currently Fedora 37.

If the 'tier' input is set as 'tier2', the tests from the tier2.yaml
plan are executed in multiple distros, currently Fedora 37 and 38.

The tests are triggered when a pull request is opened or reopened.

Signed-off-by: Anderson Toshiyuki Sasaki <[email protected]>
@ansasaki
Copy link
Contributor Author

ansasaki commented Jun 1, 2023

To see what happens, I merged this PR in my own repo: https://github.com/ansasaki/keylime-tests/

Check what happens when a PR is opened: ansasaki#1

Then, after the PR is registered successfully, it will trigger the internal workflow: https://github.com/ansasaki/keylime-tests/actions/runs/5142302701

Currently, the "check" API is not implemented to link the second workflow with the PR that originated the run.

@kkaarreell kkaarreell changed the title Run tier tests on Testing Farm [OLD] Run tier tests on Testing Farm Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant