chore: Added capability of running ITs using ok-to-test #38355
Conversation
WalkthroughThis pull request introduces enhancements to the CI/CD workflow and testing infrastructure across multiple files. The changes focus on improving test execution flexibility, workflow automation, and artifact management. The modifications span GitHub Actions workflows, Maven configuration, and test utility classes, with a primary goal of providing more granular control over test runs and streamlining the integration testing process. Changes
Sequence DiagramsequenceDiagram
participant PA as PR Automation Workflow
participant ST as Tag Parser Script
participant SB as Server Build Workflow
PA->>ST: Parse PR Tags
ST-->>PA: Return Tag Response
PA->>SB: Trigger Build with Parameters
SB->>SB: Conditional Test Execution
SB-->>PA: Build and Test Results
Possibly Related PRs
Suggested Labels
Suggested Reviewers
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
github-actions
bot
added
the
skip-changelog
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
app/server/appsmith-server/src/test/utils/com/appsmith/server/git/GitServerInitializerExtension.java (1)
40-40: Ensure null-safety on the injectedArtifactService.While Spring typically injects this service correctly, consider verifying that it is not null before use, especially for test scenarios.
.github/workflows/server-build.yml (1)
223-231: Remove trailing whitespace on line 223.The test execution logic is correct, but there's a formatting issue.
- +🧰 Tools
🪛 yamllint (1.35.1)
[error] 223-223: trailing spaces
(trailing-spaces)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
.github/workflows/pr-automation.yml(1 hunks).github/workflows/scripts/test-tag-parser.js(1 hunks).github/workflows/server-build.yml(3 hunks)app/server/appsmith-server/src/test/it/com/appsmith/server/git/templates/contexts/GitContext.java(1 hunks)app/server/appsmith-server/src/test/utils/com/appsmith/server/git/GitServerInitializerExtension.java(3 hunks)app/server/pom.xml(3 hunks)
🧰 Additional context used
🪛 actionlint (1.7.4)
.github/workflows/pr-automation.yml
139-139: property "its" is not defined in object type {matrix: string; spec: string; tags: string}
(expression)
143-143: input "its" is typed as string by reusable workflow "./.github/workflows/server-build.yml". bool value cannot be assigned
(expression)
144-144: input "is-pg-build" is typed as string by reusable workflow "./.github/workflows/server-build.yml". bool value cannot be assigned
(expression)
🪛 yamllint (1.35.1)
.github/workflows/server-build.yml
[error] 223-223: trailing spaces
(trailing-spaces)
🔇 Additional comments (10)
app/server/appsmith-server/src/test/it/com/appsmith/server/git/templates/contexts/GitContext.java (1)
28-28: Storing the artifactType in the context store looks good.
This addition helps keep the artifact type readily accessible for downstream test extensions and setups.
app/server/appsmith-server/src/test/utils/com/appsmith/server/git/GitServerInitializerExtension.java (3)
4-4: Importing ArtifactService is appropriate.
Replacing ApplicationService with ArtifactService aligns with the move toward a dedicated artifact management approach.
59-59: Make sure the artifact type is present in the store.
If artifactType is missing or null, the subsequent SSH key generation may fail. Consider adding a fallback or appropriate error handling.
67-67: Verify the error handling of the createOrUpdateSshKeyPair call.
This call returns a reactive type. Ensure any exceptions are properly handled, e.g., logging or cleanup if the call fails.
app/server/pom.xml (4)
114-130: Unit test configuration is well-structured.
The maven-surefire-plugin is properly configured for unit tests with appropriate skip conditions and dependencies.
131-148: Integration test configuration is properly set up.
The maven-failsafe-plugin configuration correctly separates integration tests and includes necessary system properties.
46-48: LGTM! Test skip properties are well-structured.
The properties follow Maven conventions and provide granular control over test execution.
Let's verify these properties are used consistently across the project:
✅ Verification successful
Properties are correctly configured and used in the Maven build
The verification shows that the skip properties are used consistently within the app/server/pom.xml:
skipTestsis correctly used as the master switchskipITsandskipUTsare properly referenced in their respective build configurations- No conflicting or inconsistent usage found in other pom files
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for usage of skip properties in pom files
rg -U "skip(ITs|UTs|Tests)" -g "pom.xml"
Length of output: 356
80-103: Test source directories are properly configured.
The build-helper-maven-plugin configuration correctly adds multiple test source directories.
Let's verify the test directory structure exists:
.github/workflows/server-build.yml (2)
8-28: Workflow inputs are well-defined and documented.
The input parameters are clearly described with appropriate defaults.
Also applies to: 42-46
238-238: LGTM! Maven command correctly uses 'verify' phase.
The change to 'mvn verify' is appropriate for running both unit and integration tests.
| const okToTestPattern = body.match(/^(\/ok-to-test) tags="(.+?)"( it=true)?/m); | ||
|
|
||
| if (okToTestPattern?.[1]) { | ||
| var response = {}; | ||
| const tagsMatch = okToTestPattern?.[2]; | ||
| if (tagsMatch) { | ||
| if (tagsMatch === "@tag.All") { | ||
| response = { tags: tagsMatch }; | ||
| } else { | ||
| const parts = tagsMatch.split(/\s*,\s*/); | ||
| for (const part of parts) { | ||
| if (!allTags.includes(part)) { | ||
| throw new Error("Unknown tag: " + part); | ||
| } | ||
| } | ||
| response = { tags: tagsMatch }; | ||
| } | ||
| } | ||
| return { tags: strictMatch }; | ||
| const itsMatch = okToTestPattern?.[2]; | ||
| if (itsMatch) { | ||
| response = { ...response, its: 'true' }; | ||
| } |
There was a problem hiding this comment.
Fix capturing group usage for it=true.
Line 49 reuses okToTestPattern?.[2], but this group already captures tags rather than the optional it=true segment. Update line 49 to capture group 3 (e.g. okToTestPattern?.[3]). Also ensure you provide an output in the job if you intend to use its downstream.
Below is an example diff to capture it=true correctly:
- const itsMatch = okToTestPattern?.[2];
+ const itsMatch = okToTestPattern?.[3];
if (itsMatch) {
response = { ...response, its: 'true' };
}Additionally, if you want to pass its out of this step, define:
core.setOutput("tags", parseResult.tags ?? "");
core.setOutput("spec", parseResult.spec ?? "");
+core.setOutput("its", parseResult.its ?? "");Committable suggestion skipped: line range outside the PR's diff.
|
|
||
| perform-it: | ||
| needs: [ parse-tags ] | ||
| if: success() && needs.parse-tags.outputs.its == 'true' | ||
| uses: ./.github/workflows/server-build.yml | ||
| secrets: inherit | ||
| with: | ||
| its: 'true' | ||
| is-pg-build: ${{ github.event.pull_request.base.ref == 'pg' }} |
There was a problem hiding this comment.
Address mismatch with its output and input types.
According to static analysis, the perform-it job relies on needs.parse-tags.outputs.its, which isn't set by the parse-tags step. Also, the reusable workflow declares its as a string, so avoid passing a Boolean expression directly to is-pg-build. Align them properly, for example:
...
outputs:
tags: ${{ steps.parseTags.outputs.tags }}
spec: ${{ steps.parseTags.outputs.spec }}
+ its: ${{ steps.parseTags.outputs.its }} # Ensure we set this in `test-tag-parser.js`
perform-it:
needs: [ parse-tags ]
if: success() && needs.parse-tags.outputs.its == 'true'
uses: ./.github/workflows/server-build.yml
secrets: inherit
with:
its: 'true'
- is-pg-build: ${{ github.event.pull_request.base.ref == 'pg' }}
+ is-pg-build: ${{ github.event.pull_request.base.ref == 'pg' ? 'true' : 'false' }}Committable suggestion skipped: line range outside the PR's diff.
🧰 Tools
🪛 actionlint (1.7.4)
139-139: property "its" is not defined in object type {matrix: string; spec: string; tags: string}
(expression)
143-143: input "its" is typed as string by reusable workflow "./.github/workflows/server-build.yml". bool value cannot be assigned
(expression)
144-144: input "is-pg-build" is typed as string by reusable workflow "./.github/workflows/server-build.yml". bool value cannot be assigned
(expression)
Description
Tip
Add a TL;DR when the description is longer than 500 words or extremely technical (helps the content, marketing, and DevRel team).
Please also include relevant motivation and context. List any dependencies that are required for this change. Add links to Notion, Figma or any other documents that might be relevant to the PR.
Fixes #
Issue Numberor
Fixes
Issue URLWarning
If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.
Automation
/ok-to-test tags="@tag.Git" it=true
🔍 Cypress test results
Tip
🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/12486019294
Commit: 3807767
Cypress dashboard.
Tags:
@tag.GitSpec:
Tue, 24 Dec 2024 21:26:04 UTC
Communication
Should the DevRel and Marketing teams inform users about this change?