Adjust CO e2e profile testing invocation #69497
Conversation
openshift-ci
bot
added
the
approved
|
/pj-rehearse |
|
@rhmdnd: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
rhmdnd
force-pushed
the
update-cac-content-profile-testing-command
branch
from
3e10065 to
b845c65
Compare
|
/pj-rehearse |
|
@rhmdnd: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
Rebasing to pull in #69499 |
rhmdnd
force-pushed
the
update-cac-content-profile-testing-command
branch
from
b845c65 to
46971ed
Compare
|
/pj-rehearse |
|
@rhmdnd: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
ci-operator/config/ComplianceAsCode/content/ComplianceAsCode-content-master.yaml
Outdated
Show resolved
Hide resolved
rhmdnd
force-pushed
the
update-cac-content-profile-testing-command
branch
from
46971ed to
53a2dee
Compare
|
ComplianceAsCode/ocp4e2e#61 will need to land before this will pass rehearsals since it has the make file target to install jq - which some of the manual remediations need. |
rhmdnd
force-pushed
the
update-cac-content-profile-testing-command
branch
from
53a2dee to
7fe0b16
Compare
openshift-merge-robot
added
the
needs-rebase
rhmdnd
force-pushed
the
update-cac-content-profile-testing-command
branch
from
7fe0b16 to
66a2b1b
Compare
openshift-merge-robot
removed
the
needs-rebase
|
/pj-rehearse |
|
@rhmdnd: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@rhmdnd, Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
/pj-rehearse |
|
@rhmdnd: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@rhmdnd, Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
| pushd ocp4e2e | ||
| go test . -run=^TestPlatformCompliance$ -test-type="platform" -content-image="$CONTENT_IMAGE" -content-directory=$PWD | ||
| pushd ocp4e2e; make install-jq | ||
| PATH=$PATH:/tmp/bin go test . -run=^TestPlatformCompliance$ -test-type="platform" -content-image="$CONTENT_IMAGE" -content-directory=$PWD |
rhmdnd
force-pushed
the
update-cac-content-profile-testing-command
branch
from
66a2b1b to
864d7fd
Compare
| export ROOT_DIR=$PWD | ||
| git clone https://github.com/ComplianceAsCode/ocp4e2e.git ocp4e2e | ||
| pushd ocp4e2e | ||
| go test -v -timeout 120m github.com/ComplianceAsCode/ocp4e2e -profile="$PROFILE" -product="$PRODUCT" -content-image="$CONTENT_IMAGE" |
There was a problem hiding this comment.
I might not need to rip out the content image here... that might be fine to leave.
|
/pj-rehearse |
|
@rhmdnd: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
Need to revert the content ds variable change - that obviously didn't work the way I thought it did: |
We recently refactored the ocp4e2e testing to support more test cases, in particular for running all node and platform rules. This means just running all go tests can have adverse side-effects if we only want to test a specific profile. This commit removes unused variables or variables that are redundant with the defaults, and adjusts the test filtering so that profile tests only run TestProfileRemediation, maintaining the older behavior of just running a single profile on a cluster to assess it's compliance posture. It also ensure jq is installed and available for the test suite to use, since some of the tests require manual remediations, and use jq to parse information from the openshift client. Remove 4.13 and 4.15 CI since those versions are no longer supported.
rhmdnd
force-pushed
the
update-cac-content-profile-testing-command
branch
from
864d7fd to
a662455
Compare
|
/pj-rehearse |
|
@rhmdnd: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
[REHEARSALNOTIFIER]
A total of 155 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs. A full list of affected jobs can be found here Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@rhmdnd: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
yuumasato
left a comment
yuumasato
left a comment
There was a problem hiding this comment.
/lgtm
Nice, scans are running now.
I just noticed this in the logs:
=== RUN TestProfileRemediations
[controller-runtime] log.SetLogger(...) was never called; logs will not be displayed.
Detected at:
> goroutine 31 [running]:
> runtime/debug.Stack()
> /usr/lib/golang/src/runtime/debug/stack.go:26 +0x5e
> sigs.k8s.io/controller-runtime/pkg/log.eventuallyFulfillRoot()
> /go/src/github.com/ComplianceAsCode/content/ocp4e2e/vendor/sigs.k8s.io/controller-runtime/pkg/log/log.go:60 +0xcd
> sigs.k8s.io/controller-runtime/pkg/log.(*delegatingLogSink).WithName(0xc00006b9c0, {0x18ac3b2, 0x14})
> /go/src/github.com/ComplianceAsCode/content/ocp4e2e/vendor/sigs.k8s.io/controller-runtime/pkg/log/deleg.go:147 +0x3e
> github.com/go-logr/logr.Logger.WithName({{0x1b3dcd0, 0xc00006b9c0}, 0x0}, {0x18ac3b2?, 0xc000695ca8?})
> /go/src/github.com/ComplianceAsCode/content/ocp4e2e/vendor/github.com/go-logr/logr/logr.go:345 +0x36
> sigs.k8s.io/controller-runtime/pkg/client.newClient(0xc000137ea0?, {0x0, 0xc000137ea0, {0x0, 0x0}, 0x0, 0x0})
> /go/src/github.com/ComplianceAsCode/content/ocp4e2e/vendor/sigs.k8s.io/controller-runtime/pkg/client/client.go:118 +0xdb
> sigs.k8s.io/controller-runtime/pkg/client.New(0xc000137ea0?, {0x0, 0xc000137ea0, {0x0, 0x0}, 0x0, 0x0})
> /go/src/github.com/ComplianceAsCode/content/ocp4e2e/vendor/sigs.k8s.io/controller-runtime/pkg/client/client.go:98 +0x55
> github.com/ComplianceAsCode/ocp4e2e/helpers.GenerateKubeConfig()
> /go/src/github.com/ComplianceAsCode/content/ocp4e2e/helpers/utilities.go:149 +0x23a
> github.com/ComplianceAsCode/ocp4e2e.TestProfileRemediations(0xc000246d00)
> /go/src/github.com/ComplianceAsCode/content/ocp4e2e/e2e_test.go:377 +0xb8
> testing.tRunner(0xc000246d00, 0x19b9180)
> /usr/lib/golang/src/testing/testing.go:1690 +0xf4
> created by testing.(*T).Run in goroutine 1
> /usr/lib/golang/src/testing/testing.go:1743 +0x390
2025/09/30 02:45:32 Found profile ocp4-cis-node
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rhmdnd, yuumasato The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Fixing in ComplianceAsCode/ocp4e2e#65 |
|
/pj-rehearse ack |
|
@yuumasato: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-ci-robot
added
the
rehearsals-ack
openshift-merge-bot
bot
merged commit f8a026a
into
openshift:master
We recently refactored the ocp4e2e testing to support more test cases, in particular for running all node and platform rules. This means just running all go tests can have adverse side-effects if we only want to test a specific profile. This commit removes unused variables or variables that are redundant with the defaults, and adjusts the test filtering so that profile tests only run TestProfileRemediation, maintaining the older behavior of just running a single profile on a cluster to assess it's compliance posture. It also ensure jq is installed and available for the test suite to use, since some of the tests require manual remediations, and use jq to parse information from the openshift client. Remove 4.13 and 4.15 CI since those versions are no longer supported.
We recently refactored the ocp4e2e testing to support more test cases, in particular for running all node and platform rules. This means just running all go tests can have adverse side-effects if we only want to test a specific profile. This commit removes unused variables or variables that are redundant with the defaults, and adjusts the test filtering so that profile tests only run TestProfileRemediation, maintaining the older behavior of just running a single profile on a cluster to assess it's compliance posture. It also ensure jq is installed and available for the test suite to use, since some of the tests require manual remediations, and use jq to parse information from the openshift client. Remove 4.13 and 4.15 CI since those versions are no longer supported.
We recently refactored the ocp4e2e testing to support more test cases, in particular for running all node and platform rules. This means just running all go tests can have adverse side-effects if we only want to test a specific profile. This commit removes unused variables or variables that are redundant with the defaults, and adjusts the test filtering so that profile tests only run TestProfileRemediation, maintaining the older behavior of just running a single profile on a cluster to assess it's compliance posture. It also ensure jq is installed and available for the test suite to use, since some of the tests require manual remediations, and use jq to parse information from the openshift client. Remove 4.13 and 4.15 CI since those versions are no longer supported.
4 participants
We recently refactored the ocp4e2e testing to support more test cases,
in particular for running all node and platform rules. This means just
running all go tests can have adverse side-effects if we only want to
test a specific profile.
This commit removes unused variables or variables that are redundant
with the defaults, and adjusts the test filtering so that profile tests
only run TestProfileRemediation, maintaining the older behavior of just
running a single profile on a cluster to assess it's compliance posture.