Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "[tls] Add CA bundle from OpenStackCtlplane to controller" #737

Closed
wants to merge 1 commit into from

Conversation

cjeanner
Copy link
Collaborator

@cjeanner cjeanner commented Oct 27, 2023

This is a partial revert of 00e8d24.

We've seen constant failures in a CI job, linked to certificate
validation:
https://review.rdoproject.org/zuul/builds?job_name=podified-multinode-edpm-e2e-nobuild-tagged-crc&project=openstack-k8s-operators/ci-framework

An example:
FAILED - RETRYING: [localhost]: Wait for keystone endpoint to exist in DNS (1 retries left).
fatal: [localhost]: FAILED! => {"attempts": 20, "changed": false, "elapsed": 0, "msg": "Status code was -1 and not [200, 300, 301, 302, 401, 402, 403]: Request failed: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1129)>", "redirected": false, "status": -1, "url": "https://keystone-public-openstack.apps-crc.testing/"}

Reverting the CA verification toggle patch seems the safest way to get
back to a green CI.
It's still supposed to fetch and install the CA at this point.

We're seeing tempest failures when we do a complete reverse.

Note: the new failure may be related to a recent patch:
openstack-k8s-operators/openstack-operator#502

As a pull request owner and reviewers, I checked that:

  • Appropriate testing is done and actually running

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 27, 2023

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from cjeanner. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rlandy
Copy link
Collaborator

rlandy commented Oct 27, 2023

/lgtm

@softwarefactory-project-zuul
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/5c66387583d843f39f22773c52977344

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 20m 00s
podified-multinode-edpm-deployment-crc FAILURE in 56m 01s
✔️ noop SUCCESS in 0s
✔️ cifmw-doc SUCCESS in 1m 55s
podified-multinode-edpm-e2e-nobuild-tagged-crc FAILURE in 59m 58s
✔️ cifmw-molecule-edpm_prepare SUCCESS in 5m 14s
✔️ cifmw-molecule-os_net_setup SUCCESS in 3m 52s
✔️ cifmw-molecule-tempest SUCCESS in 5m 27s

@openshift-ci openshift-ci bot removed the lgtm label Oct 27, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 27, 2023

New changes are detected. LGTM label has been removed.

@cjeanner cjeanner mentioned this pull request Oct 27, 2023
1 task
@softwarefactory-project-zuul
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/ffe8a2e752eb46c4885576b77d4ff110

✔️ openstack-k8s-operators-content-provider SUCCESS in 57m 34s
podified-multinode-edpm-deployment-crc FAILURE in 42m 12s
✔️ noop SUCCESS in 0s
✔️ cifmw-doc SUCCESS in 2m 01s
podified-multinode-edpm-e2e-nobuild-tagged-crc FAILURE in 40m 40s
✔️ cifmw-molecule-edpm_prepare SUCCESS in 4m 51s
✔️ cifmw-molecule-os_net_setup SUCCESS in 3m 48s
✔️ cifmw-molecule-tempest SUCCESS in 5m 01s

This is a *partial* revert of 00e8d24.

We've seen constant failures in a CI job, linked to certificate
validation:
https://review.rdoproject.org/zuul/builds?job_name=podified-multinode-edpm-e2e-nobuild-tagged-crc&project=openstack-k8s-operators/ci-framework

An example:
FAILED - RETRYING: [localhost]: Wait for keystone endpoint to exist in DNS (1 retries left).
fatal: [localhost]: FAILED! => {"attempts": 20, "changed": false, "elapsed": 0, "msg": "Status code was -1 and not [200, 300, 301, 302, 401, 402, 403]: Request failed: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1129)>", "redirected": false, "status": -1, "url": "https://keystone-public-openstack.apps-crc.testing"}

Reverting the CA verification toggle patch seems the safest way to get
back to a green CI.
It's still supposed to fetch and install the CA at this point.

We're seeing tempest failures when we do a complete reverse.

Note: the new failure may be related to a recent patch:
openstack-k8s-operators/openstack-operator#502
@softwarefactory-project-zuul
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/5f0a20838e9c4edca78f412c85cffa39

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 19m 21s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 04m 03s
✔️ noop SUCCESS in 0s
✔️ cifmw-doc SUCCESS in 1m 57s
podified-multinode-edpm-e2e-nobuild-tagged-crc FAILURE in 59m 12s
✔️ cifmw-molecule-edpm_prepare SUCCESS in 4m 58s
✔️ cifmw-molecule-os_net_setup SUCCESS in 3m 57s
✔️ cifmw-molecule-tempest SUCCESS in 4m 58s

@cjeanner
Copy link
Collaborator Author

Issue isn't really on our side in fact.

The failing job is consuming published content without any rebuild - meaning we may need a bump of the openstack-operator or related change.

We'll move the tagged-nobuild job to periodic since the edpm-deployment is already running on each PR.

@cjeanner cjeanner closed this Oct 27, 2023
@cjeanner cjeanner deleted the bundle-ca/revert branch November 7, 2023 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants