Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(test) O3-2525: Add new GH workflow to run E2E on release PRs #754

Merged
merged 9 commits into from
Oct 27, 2023

Conversation

jayasanka-sack
Copy link
Member

@jayasanka-sack jayasanka-sack commented Oct 25, 2023

Ticket: https://issues.openmrs.org/browse/O3-2525

🎃 Test PR: jayasanka-sack#4

This GitHub Actions workflow, named "Run E2E Tests on Release PRs," serves as Quality Gate #4 in the O3 Release QA pipeline.

Screenshot 2023-10-24 at 18 46 49

The workflow is designed to automate the end-to-end (E2E) testing process for release pull requests (PRs)
that opened in the format described in here: How to Release the O3 RefApp

The workflow is conditional and only runs if the pull request title starts with "(release)".

Below is an overview of the key components of the workflow:
Screenshot 2023-10-24 at 18 13 19

Job: build

This job prepares the test environment, extracts version numbers, builds and runs Docker containers, and uploads
artifacts for use in subsequent E2E testing jobs.

Checking out to the release commit

The reason for the step:

- name: Checkout to the release commit
  run: |
  release_commit_sha=$(git log --oneline | awk 'NR==3 {print $1}')
  git checkout $release_commit_sha

is to ensure that the workflow checks out to the specific release commit associated with the pull request. This is
necessary because the pull request contain both release commits and revert commits, and the goal is to specifically
target the release commit for further processing.

Here's an explanation of what this step does:

  1. It uses git log --oneline to retrieve a list of recent commits in the repository.
  2. The awk 'NR==3 {print $1}' part is used to extract the SHA hash of the third most recent commit, which is the
    release commit we want to target. This is done by specifying NR==3, which means selecting the third line (commit)
    in the log.
    The reason to use the 3rd commit instead of 2nd is that a GitHub Action workflow operates within a context where
    there's an additional merge commit on the head of the pull request.
  3. Finally, the workflow checks out to this release commit by using git checkout $release_commit_sha.

By checking out to the specific release commit, it ensures that the subsequent steps of the workflow are based on the
state of the code associated with the release, and not on the revert commit present in the pull request.

End-to-End Test Jobs

The workflow includes several end-to-end test jobs, each corresponding to a specific components of O3 (frontend
mono-repos). These jobs are structured similarly and are listed below:

  • run-patient-management-e2e-tests
  • run-patient-chart-e2e-tests
  • run-form-builder-e2e-tests
  • run-esm-core-e2e-tests
  • run-cohort-builder-e2e-tests

In each "End-to-End Test Job," the workflow first checks out the repository associated with a specific OpenMRS
component. It then downloads Docker images from a previous "build" job, loads these images, and starts an OpenMRS instance.

Why Check Out to the Tags?

The workflow checks out a specific tagged version of the mono-repo, the tag is imported from the previous "build" job. This is necessary because the goal is to perform end-to-end tests on the codebase that corresponds to a
particular release version, rather than the code at the head of the repository.

Screenshot 2023-10-24 at 17 24 38

@jayasanka-sack jayasanka-sack changed the title Add new GH workflow to run E2E on release PRs (test) O3-2525: Add new GH workflow to run E2E on release PRs Oct 25, 2023
Copy link
Member

@ibacher ibacher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jayasanka-sack! This is excellent! A few minor nits, but this LGTM.

.github/workflows/e2e-on-release.yml Outdated Show resolved Hide resolved
e2e_test_support_files/extract_tag_numbers.sh Outdated Show resolved Hide resolved
Comment on lines +41 to +46
- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: e2e_release_env_images
path: e2e_release_env_images.tar.gz
retention-days: 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd probably add a "finally" job that just removes the artifact. That's better hygiene, I think.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleting artifact seems tricky. While there are marketplace options available for the same, they are used by only a few users.

.github/workflows/e2e-on-release.yml Outdated Show resolved Hide resolved
e2e_test_support_files/docker-compose-build.yml Outdated Show resolved Hide resolved
e2e_test_support_files/docker-compose.yml Show resolved Hide resolved
e2e_test_support_files/commit_and_export_images.sh Outdated Show resolved Hide resolved
@jayasanka-sack
Copy link
Member Author

Thank you for your review, @ibacher . The issue with using HEAD~2 is that it actually points to the second most recent commit on the main branch. This is the reason I included depth:0, as without it, the PR branch's commits wouldn't be fetched. I'd appreciate your thoughts on this matter.

This is how it's represented in the GitHub workflow context.

Screenshot 2023-10-25 at 11 11 00

@ibacher
Copy link
Member

ibacher commented Oct 25, 2023

@jayasanka-sack Alright... How about something a little cleverer HEAD^{/\(release)}. You're right that I forgot that Actions will always be working on a merge commit.

- Remove unused ports
- Refactored workflow to use mktemp for temporary directory
- Simplified compression step
- Updated Docker image loading process
@jayasanka-sack jayasanka-sack marked this pull request as draft October 26, 2023 04:40
@jayasanka-sack
Copy link
Member Author

git checkout HEAD^{/\(release\)} checkouts to whatever commit that contains release. Any idea?

Btw, how about git checkout $(git log --grep="^(release)" --format="%H" | head -1)?

@jayasanka-sack jayasanka-sack marked this pull request as ready for review October 27, 2023 00:51
@ibacher
Copy link
Member

ibacher commented Oct 27, 2023

checkouts to whatever commit that contains release. Any idea?

Right... specifically it should checkout the nearest commit to HEAD that contains (release). You could also add the ^ like in the grep to force it be at the start. You may need to quote the HEAD^{/\(release\)} like 'HEAD^{/\(release\)}' for it to work (like with the grep).

@ibacher ibacher closed this Oct 27, 2023
@ibacher ibacher reopened this Oct 27, 2023
@jayasanka-sack
Copy link
Member Author

That worked. Thanks, @ibacher ! Could you please review the PR again?

@ibacher ibacher merged commit ad1202e into openmrs:main Oct 27, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants