Skip to content

CNTRLPLANE-1309: set up openshift-tests-extension for cluster-kube-scheduler-operator and add a sanity test #585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

CNTRLPLANE-1309: set up openshift-tests-extension for cluster-kube-scheduler-operator and add a sanity test #585

wants to merge 2 commits into from

Conversation

@sandeepknd
Copy link

@sandeepknd sandeepknd commented Oct 16, 2025

As part of the https://issues.redhat.com/browse/CNTRLPLANE-1221 and sub task https://issues.redhat.com/browse/CNTRLPLANE-1309, raised this PR which introduces a binary and supporting structure to enable the execution of cluster-kube-scheduler-operator tests using the Open Test Environment (OTE).

Changes:

Add cmd/cluster-kube-scheduler-operator-tests-ext/ with main.go
Add test/extended/ directory with comprehensive README.md and sanity test
Update Makefile with flexible OTE build targets and helper commands
Update go.mod with required dependencies for ginkgo and openshift-tests-extension
Add vendor dependencies for new OTE framework

Test suites:
openshift/cluster-kube-scheduler-operator/conformance/parallel
openshift/cluster-kube-scheduler-operator/conformance/serial
openshift/cluster-kube-scheduler-operator/optional/slow
openshift/cluster-kube-scheduler-operator/all

Please find the OTE test results.

$ ./cluster-kube-scheduler-operator-tests-ext --help
Cluster Kube Scheduler Operator Tests Extension

Usage:
   [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  images      List test images
  info        Display extension metadata
  list        List items
  run-suite   Run a group of tests by suite. This is more limited than origin, and intended for light local development use. Orchestration parameters, scheduling, isolation, etc are not obeyed, and Ginkgo tests are executed serially.
  run-test    Runs tests by name
  update      Update test metadata

Flags:
  -h, --help   help for this command

Use " [command] --help" for more information about a command.


$ ./cluster-kube-scheduler-operator-tests-ext info
{
    "apiVersion": "v1.1",
    "source": {
        "commit": "",
        "build_date": "",
        "git_tree_state": ""
    },
    "component": {
        "product": "openshift",
        "type": "payload",
        "name": "cluster-kube-scheduler-operator"
    },
    "suites": [
        {
            "name": "openshift/cluster-kube-scheduler-operator/conformance/parallel",
            "description": "",
            "parents": [
                "openshift/conformance/parallel"
            ],
            "qualifiers": [
                "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (!(name.contains(\"[Serial]\") || name.contains(\"[Slow]\")))"
            ]
        },
        {
            "name": "openshift/cluster-kube-scheduler-operator/conformance/serial",
            "description": "",
            "parents": [
                "openshift/conformance/serial"
            ],
            "qualifiers": [
                "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (name.contains(\"[Serial]\"))"
            ]
        },
        {
            "name": "openshift/cluster-kube-scheduler-operator/optional/slow",
            "description": "",
            "parents": [
                "openshift/optional/slow"
            ],
            "qualifiers": [
                "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (name.contains(\"[Slow]\"))"
            ]
        },
        {
            "name": "openshift/cluster-kube-scheduler-operator/all",
            "description": "",
            "qualifiers": [
                "source == \"openshift:payload:cluster-kube-scheduler-operator\""
            ]
        }
    ],
    "images": null
}


$ ./cluster-kube-scheduler-operator-tests-ext list
[
  {
    "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
    "labels": {},
    "resources": {
      "isolation": {}
    },
    "source": "openshift:payload:cluster-kube-scheduler-operator",
    "codeLocations": [
      "/home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:8",
      "/home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9"
    ],
    "lifecycle": "blocking",
    "environmentSelector": {}
  }
]


$ ./cluster-kube-scheduler-operator-tests-ext list tests
[
  {
    "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
    "labels": {},
    "resources": {
      "isolation": {}
    },
    "source": "openshift:payload:cluster-kube-scheduler-operator",
    "codeLocations": [
      "/home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:8",
      "/home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9"
    ],
    "lifecycle": "blocking",
    "environmentSelector": {}
  }
]


$ ./cluster-kube-scheduler-operator-tests-ext list suites
[
  {
    "name": "openshift/cluster-kube-scheduler-operator/conformance/parallel",
    "description": "",
    "parents": [
      "openshift/conformance/parallel"
    ],
    "qualifiers": [
      "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (!(name.contains(\"[Serial]\") || name.contains(\"[Slow]\")))"
    ]
  },
  {
    "name": "openshift/cluster-kube-scheduler-operator/conformance/serial",
    "description": "",
    "parents": [
      "openshift/conformance/serial"
    ],
    "qualifiers": [
      "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (name.contains(\"[Serial]\"))"
    ]
  },
  {
    "name": "openshift/cluster-kube-scheduler-operator/optional/slow",
    "description": "",
    "parents": [
      "openshift/optional/slow"
    ],
    "qualifiers": [
      "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (name.contains(\"[Slow]\"))"
    ]
  },
  {
    "name": "openshift/cluster-kube-scheduler-operator/all",
    "description": "",
    "qualifiers": [
      "source == \"openshift:payload:cluster-kube-scheduler-operator\""
    ]
  }
]


$ ./cluster-kube-scheduler-operator-tests-ext run-suite "openshift/cluster-kube-scheduler-operator/conformance/parallel"
  Running Suite:  - /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator
  ===================================================================================
  Random Seed: 1760627429 - will randomize all specs

  Will run 1 of 1 specs
  ------------------------------
  [Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]
  /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9
  • [0.000 seconds]
  ------------------------------

  Ran 1 of 1 Specs in 0.000 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
  {
    "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 0,
    "startTime": "2025-10-16 15:10:29.326215 UTC",
    "endTime": "2025-10-16 15:10:29.326924 UTC",
    "result": "passed",
    "output": ""
  }



$ ./cluster-kube-scheduler-operator-tests-ext run-suite "openshift/cluster-kube-scheduler-operator/all"
  Running Suite:  - /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator
  ===================================================================================
  Random Seed: 1760627500 - will randomize all specs

  Will run 1 of 1 specs
  ------------------------------
  [Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]
  /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9
  • [0.000 seconds]
  ------------------------------

  Ran 1 of 1 Specs in 0.000 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
  {
    "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 0,
    "startTime": "2025-10-16 15:11:40.539414 UTC",
    "endTime": "2025-10-16 15:11:40.540043 UTC",
    "result": "passed",
    "output": ""
  }


$ ./cluster-kube-scheduler-operator-tests-ext run-test "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]"
  Running Suite:  - /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator
  ===================================================================================
  Random Seed: 1760627671 - will randomize all specs

  Will run 1 of 1 specs
  ------------------------------
  [Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]
  /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9
  • [0.000 seconds]
  ------------------------------

  Ran 1 of 1 Specs in 0.000 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
  {
    "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 0,
    "startTime": "2025-10-16 15:14:31.221668 UTC",
    "endTime": "2025-10-16 15:14:31.221958 UTC",
    "result": "passed",
    "output": ""
  }

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 16, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 16, 2025 •
edited by openshift-ci bot
Loading

@sandeepknd: This pull request references CNTRLPLANE-1309 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.21.0" version, but no target version was set.

In response to this:

As part of the https://issues.redhat.com/browse/CNTRLPLANE-1221 and sub task https://issues.redhat.com/browse/CNTRLPLANE-1309, raised this PR which introduces a binary and supporting structure to enable the execution of cluster-kube-scheduler-operator tests using the Open Test Environment (OTE).

Changes:

Add cmd/cluster-kube-scheduler-operator-tests-ext/ with main.go
Add test/extended/ directory with comprehensive README.md and sanity test
Update Makefile with flexible OTE build targets and helper commands
Update go.mod with required dependencies for ginkgo and openshift-tests-extension
Add vendor dependencies for new OTE framework

Test suites:
openshift/cluster-kube-scheduler-operator/conformance/parallel
openshift/cluster-kube-scheduler-operator/conformance/serial
openshift/cluster-kube-scheduler-operator/optional/slow
openshift/cluster-kube-scheduler-operator/all

Please find the OTE test results.

$ ./cluster-kube-scheduler-operator-tests-ext --help
Cluster Kube Scheduler Operator Tests Extension

Usage:
  [command]

Available Commands:
 completion  Generate the autocompletion script for the specified shell
 help        Help about any command
 images      List test images
 info        Display extension metadata
 list        List items
 run-suite   Run a group of tests by suite. This is more limited than origin, and intended for light local development use. Orchestration parameters, scheduling, isolation, etc are not obeyed, and Ginkgo tests are executed serially.
 run-test    Runs tests by name
 update      Update test metadata

Flags:
 -h, --help   help for this command

Use " [command] --help" for more information about a command.


$ ./cluster-kube-scheduler-operator-tests-ext info
{
   "apiVersion": "v1.1",
   "source": {
       "commit": "",
       "build_date": "",
       "git_tree_state": ""
   },
   "component": {
       "product": "openshift",
       "type": "payload",
       "name": "cluster-kube-scheduler-operator"
   },
   "suites": [
       {
           "name": "openshift/cluster-kube-scheduler-operator/conformance/parallel",
           "description": "",
           "parents": [
               "openshift/conformance/parallel"
           ],
           "qualifiers": [
               "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (!(name.contains(\"[Serial]\") || name.contains(\"[Slow]\")))"
           ]
       },
       {
           "name": "openshift/cluster-kube-scheduler-operator/conformance/serial",
           "description": "",
           "parents": [
               "openshift/conformance/serial"
           ],
           "qualifiers": [
               "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (name.contains(\"[Serial]\"))"
           ]
       },
       {
           "name": "openshift/cluster-kube-scheduler-operator/optional/slow",
           "description": "",
           "parents": [
               "openshift/optional/slow"
           ],
           "qualifiers": [
               "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (name.contains(\"[Slow]\"))"
           ]
       },
       {
           "name": "openshift/cluster-kube-scheduler-operator/all",
           "description": "",
           "qualifiers": [
               "source == \"openshift:payload:cluster-kube-scheduler-operator\""
           ]
       }
   ],
   "images": null
}


$ ./cluster-kube-scheduler-operator-tests-ext list
[
 {
   "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
   "labels": {},
   "resources": {
     "isolation": {}
   },
   "source": "openshift:payload:cluster-kube-scheduler-operator",
   "codeLocations": [
     "/home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:8",
     "/home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9"
   ],
   "lifecycle": "blocking",
   "environmentSelector": {}
 }
]


$ ./cluster-kube-scheduler-operator-tests-ext list tests
[
 {
   "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
   "labels": {},
   "resources": {
     "isolation": {}
   },
   "source": "openshift:payload:cluster-kube-scheduler-operator",
   "codeLocations": [
     "/home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:8",
     "/home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9"
   ],
   "lifecycle": "blocking",
   "environmentSelector": {}
 }
]


$ ./cluster-kube-scheduler-operator-tests-ext list suites
[
 {
   "name": "openshift/cluster-kube-scheduler-operator/conformance/parallel",
   "description": "",
   "parents": [
     "openshift/conformance/parallel"
   ],
   "qualifiers": [
     "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (!(name.contains(\"[Serial]\") || name.contains(\"[Slow]\")))"
   ]
 },
 {
   "name": "openshift/cluster-kube-scheduler-operator/conformance/serial",
   "description": "",
   "parents": [
     "openshift/conformance/serial"
   ],
   "qualifiers": [
     "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (name.contains(\"[Serial]\"))"
   ]
 },
 {
   "name": "openshift/cluster-kube-scheduler-operator/optional/slow",
   "description": "",
   "parents": [
     "openshift/optional/slow"
   ],
   "qualifiers": [
     "(source == \"openshift:payload:cluster-kube-scheduler-operator\") \u0026\u0026 (name.contains(\"[Slow]\"))"
   ]
 },
 {
   "name": "openshift/cluster-kube-scheduler-operator/all",
   "description": "",
   "qualifiers": [
     "source == \"openshift:payload:cluster-kube-scheduler-operator\""
   ]
 }
]


$ ./cluster-kube-scheduler-operator-tests-ext run-suite "openshift/cluster-kube-scheduler-operator/conformance/parallel"
 Running Suite:  - /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator
 ===================================================================================
 Random Seed: 1760627429 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]
 /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-10-16 15:10:29.326215 UTC",
   "endTime": "2025-10-16 15:10:29.326924 UTC",
   "result": "passed",
   "output": ""
 }



$ ./cluster-kube-scheduler-operator-tests-ext run-suite "openshift/cluster-kube-scheduler-operator/all"
 Running Suite:  - /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator
 ===================================================================================
 Random Seed: 1760627500 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]
 /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-10-16 15:11:40.539414 UTC",
   "endTime": "2025-10-16 15:11:40.540043 UTC",
   "result": "passed",
   "output": ""
 }


$ ./cluster-kube-scheduler-operator-tests-ext run-test "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]"
 Running Suite:  - /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator
 ===================================================================================
 Random Seed: 1760627671 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]
 /home/skundu/automation/Scheduler/cluster-kube-scheduler-operator/test/extended/main.go:9
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[Jira:kube-scheduler][sig-kube-scheduler] sanity test should always pass [Suite:openshift/cluster-kube-scheduler-operator/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-10-16 15:14:31.221668 UTC",
   "endTime": "2025-10-16 15:14:31.221958 UTC",
   "result": "passed",
   "output": ""
 }

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@sandeepknd
Copy link
Author

sandeepknd commented Oct 16, 2025

/assign @gangwgr
/assign @wangke19

@openshift-ci openshift-ci bot requested a review from ingvagabund October 16, 2025 15:16
@sandeepknd
Copy link
Author

sandeepknd commented Oct 17, 2025

/assign @ingvagabund

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 18, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sandeepknd
Once this PR has been reviewed and has the lgtm label, please ask for approval from ingvagabund. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sandeepknd sandeepknd force-pushed the ote-sch-op branch 2 times, most recently from be103cc to 801c0fe Compare October 18, 2025 13:16
@sandeepknd
Copy link
Author

sandeepknd commented Oct 19, 2025

/test okd-scos-e2e-aws-ovn

@sandeepknd
Copy link
Author

sandeepknd commented Oct 19, 2025

/test e2e-aws-ovn

@sandeepknd
Copy link
Author

sandeepknd commented Oct 19, 2025

/test okd-scos-e2e-aws-ovn
/test e2e-aws-ovn

@sandeepknd
Copy link
Author

sandeepknd commented Oct 19, 2025

/test okd-scos-e2e-aws-ovn

1 similar comment
@sandeepknd
Copy link
Author

sandeepknd commented Oct 19, 2025

/test okd-scos-e2e-aws-ovn

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 19, 2025

@sandeepknd: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-e2e-aws-ovn 09ffe45 link false /test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@wangke19
Copy link

wangke19 commented Oct 20, 2025 •
edited
Loading

Please hold on. Currently, some teams encountered issue using comat_otp to migrate the test cases from the private repo to their own repo, see https://redhat-internal.slack.com/archives/C07RDCVEYJG/p1760624199090789, I was trying to build test-extension and component seperately, avoid conflicts of rebase for component, because product code needs to use latest version of openshift other dependencies. Most of this work has been completed, you can refer to that later. See openshift/service-ca-operator#283

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ingvagabund ingvagabund Awaiting requested review from ingvagabund

Assignees

@ingvagabund ingvagabund

@wangke19 wangke19

@gangwgr gangwgr

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sandeepknd @openshift-ci-robot @wangke19 @ingvagabund @gangwgr