feat: Validate individual tasks in GA CI #1447

tisutisu · 2024-09-19T06:58:15Z

This PR adds a Github Actions CI to validate the task tests (if present inside the task directory) and few example tests for git-clone task for reference.

Story: https://issues.redhat.com/browse/STONEBLD-2910

lcarva · 2024-11-13T08:23:27Z

task/git-clone/0.1/tests/run.yaml

@@ -0,0 +1,30 @@
+---


Have you considering using a Pipeline to encapsulate the Task execution and the required verification? The release folks took this approach, example: https://github.com/konflux-ci/release-service-catalog/blob/development/tasks/check-data-keys/tests/test-check-data-keys-fail-missing-cdn-key.yaml

@lcarva could you please help with more context around where these tests are running what approach they are following... any doc or CI pipeline script?

mmorhun

Can we check task results / logs / created images, etc. ?

mmorhun · 2024-11-12T11:43:53Z

.github/workflows/run-task-tests.yaml

+            task/**
+            test/**
+
+      - name: Free Disk Space


Why do we need this?

took it from here, don't aware of the exact reason, but I think might be helping to create more disk space in the github action env. Don't find any issue to keep it our CI as well.

mmorhun · 2024-11-13T10:49:30Z

README.md

+
+For example refer: Git Clone task [tests](./task/git-clone/0.1/tests/run.yaml)
+
+CI also validates the task yamls using [tektor](https://github.com/lcarva/tektor)


@lcarva is there any plans to move it to some public org from personal account?

Filed https://issues.redhat.com/browse/EC-1019

mmorhun · 2024-11-13T11:03:15Z

hack/run-test.sh

+
+tns=${NAME}-${VERSION//./-}
+
+# Delete the tns if already exists


I'd expand tns here

updated now.

mmorhun · 2024-11-13T11:03:45Z

hack/run-test.sh

+tns=${NAME}-${VERSION//./-}
+
+# Delete the tns if already exists
+${KUBECTL_CMD} delete ns ${tns} >/dev/null 2>/dev/null || :


namespace is used in other places, please use it here too

updated now.

mmorhun · 2024-11-13T11:03:57Z

hack/run-test.sh

+   $KUBECTL_CMD create sa appstudio-pipeline -n ${tns}
+fi
+
+test_resource_creation ${RESOURCE}/${NAME}/${VERSION}/tests


nitpick: missing new line

some left overs, forgot to remove this file, it got moved to test/run-test.sh

mmorhun · 2024-11-13T14:26:44Z

test/common.sh

+    local reason=''
+    local maxloop=60 # 10 minutes max
+
+    set +x


Why do we need set +x here?

to avoid unnecessary logs being printed in the console, while we looping through to check task run status, reason and results, if something fails show_failure() will dump the test resources in the console.

mmorhun · 2024-11-13T14:44:38Z

test/common.sh

+    done
+}
+
+function test_resource_creation() {


The function name and what id does differs

updated it to test_resource_creation_and_validation, hope the name fits with its functionality now

mmorhun · 2024-11-13T14:47:12Z

test/common.sh

+    echo ${CHANGED_FILES} |grep -o 'task/[^\/]*/[^\/]*/*[^/]*.yaml'|xargs -I {} dirname {}|awk '{print $1"/tests"}'
+}
+
+function get_new_changed_tasks() {


function name is not clear

changed it to print_changed_tasks_to_console, hope it better fit its functionality.

mmorhun · 2024-11-13T14:48:07Z

test/common.sh

+    echo ${CHANGED_FILES} |grep "test/" || true
+}
+
+function detect_new_changed_resources() {


function name is unclear

changed it to print_changed_resources_to_console now

mmorhun · 2024-11-13T14:49:44Z

test/run-test.sh

@@ -0,0 +1,47 @@
+#!/bin/bash


Isn't it duplicates hack/run-test.sh?

forgot to remove the old one hack/run-test.sh, this test/run-test.sh is the correct one.

chmeliik · 2024-11-13T13:07:27Z

.github/workflows/run-task-tests.yaml

+      - name: Checkout konflux-ci/konflux-ci Repository
+        if: steps.changed-files.outputs.any_changed == 'true'
+        uses: actions/checkout@v3
+        with:
+          repository: 'konflux-ci/konflux-ci'
+          path: konflux-ci


Should we pin konflux-ci/konflux-ci to a commit (and configure renovate to update it)? To avoid CI breaking overnight due to changes in the repo

chmeliik · 2024-11-13T13:07:48Z

.github/workflows/run-task-tests.yaml

+      - name: Install tektor
+        if: steps.changed-files.outputs.any_changed == 'true'
+        run: |
+          go install github.com/lcarva/tektor@latest


Similar question here, perhaps pin to a commit?

Pinned to a sha now

chmeliik · 2024-11-13T15:00:41Z

README.md

+A helper script called `run-test.sh` is present inside `test` directory to help the developer run the test. Just specify resource type as first argument, resource name as second argument and the version as third argument i.e:
+
+```
+./test/run-test.sh task git-clone 0.1


type as first argument, resource name as second argument and the version as third argument

This seems unnecessary, passing the actual directory as the argument would work much better with tab completion :)

we could also test stepactions in similar ways, that functionality is not added fully in the current PR, which we can extend to in the future, so kept the type (task or stepactions) as first argument in the run-test.sh script.

chmeliik · 2024-11-13T15:04:59Z

test/common.sh

+function validate_task_yaml_using_tektor() {
+    all_tasks="$*"
+    for task in ${all_tasks}; do
+        # Skip if it is kustomize related yaml files
+        if [[ ${task} == *"kustomization.yaml" || ${task} == *"patch.yaml" || ${task} == *"recipe.yaml" ]]; then
+                continue
+        fi
+        tektor validate ${task}
+    done
+}


Let's not mix linting and tests, this should be a separate GH workflow IMO

agree, will separate them to two different workflows.

tisutisu force-pushed the test-tasks branch 2 times, most recently from 17aa0a1 to 0cd56c4 Compare October 21, 2024 05:44

tisutisu force-pushed the test-tasks branch 2 times, most recently from 923122a to 59f79e3 Compare November 4, 2024 12:12

tisutisu force-pushed the test-tasks branch 2 times, most recently from ccafb0b to f93dad2 Compare November 12, 2024 04:37

tisutisu changed the title ~~WIP: Validate individual tasks/stepactions~~ feat: Validate individual tasks in GA CI Nov 12, 2024

tisutisu marked this pull request as ready for review November 12, 2024 06:24

openshift-ci bot requested review from brunoapimentel and tnevrlka November 12, 2024 06:24

tisutisu requested review from mmorhun, chmeliik and lcarva November 12, 2024 07:16

lcarva reviewed Nov 13, 2024

View reviewed changes

mmorhun reviewed Nov 13, 2024

View reviewed changes

chmeliik reviewed Nov 13, 2024

View reviewed changes

tisutisu force-pushed the test-tasks branch from 1237e1d to 6c9cafd Compare November 15, 2024 07:24

tisutisu added 2 commits November 15, 2024 15:24

tests for inidividual tasks

4a28349

Address review comments

9dd1446

tisutisu force-pushed the test-tasks branch from 88cc5ab to 9dd1446 Compare November 15, 2024 09:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Validate individual tasks in GA CI #1447

feat: Validate individual tasks in GA CI #1447

tisutisu commented Sep 19, 2024 •

edited

Loading

lcarva Nov 13, 2024

tisutisu Nov 15, 2024 •

edited

Loading

mmorhun left a comment

mmorhun Nov 12, 2024

tisutisu Nov 15, 2024

mmorhun Nov 13, 2024

lcarva Nov 13, 2024

mmorhun Nov 13, 2024

tisutisu Nov 15, 2024

mmorhun Nov 13, 2024

tisutisu Nov 15, 2024

mmorhun Nov 13, 2024

tisutisu Nov 15, 2024

mmorhun Nov 13, 2024

tisutisu Nov 15, 2024

mmorhun Nov 13, 2024

tisutisu Nov 15, 2024

mmorhun Nov 13, 2024

tisutisu Nov 15, 2024

mmorhun Nov 13, 2024

tisutisu Nov 15, 2024

mmorhun Nov 13, 2024

tisutisu Nov 15, 2024 •

edited

Loading

chmeliik Nov 13, 2024

chmeliik Nov 13, 2024

tisutisu Nov 15, 2024

chmeliik Nov 13, 2024

tisutisu Nov 15, 2024

chmeliik Nov 13, 2024

tisutisu Nov 15, 2024 •

edited

Loading


		For example refer: Git Clone task [tests](./task/git-clone/0.1/tests/run.yaml)

		CI also validates the task yamls using [tektor](https://github.com/lcarva/tektor)


		tns=${NAME}-${VERSION//./-}

		# Delete the tns if already exists

feat: Validate individual tasks in GA CI #1447

Are you sure you want to change the base?

feat: Validate individual tasks in GA CI #1447

Conversation

tisutisu commented Sep 19, 2024 • edited Loading

Choose a reason for hiding this comment

tisutisu Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

mmorhun left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tisutisu Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tisutisu Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

tisutisu commented Sep 19, 2024 •

edited

Loading

tisutisu Nov 15, 2024 •

edited

Loading

tisutisu Nov 15, 2024 •

edited

Loading

tisutisu Nov 15, 2024 •

edited

Loading