New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validate Cinder operations #649

Open

rajathere wants to merge 1 commit into openstack-k8s-operators:main from rajathere:cinder_validation

+57 −0

Contributor

rajathere commented Oct 10, 2024 •

edited

Loading

After the adoption, we check for the correct status and size of the
Cinder resources but don't validate that operations still work on them.
This patch adds the following operations:

Detach 'disk' volume from 'test' server
Revert to snapshot 'snapshot' to 'disk' volume
Attach 'disk' volume to 'test' server

This will help verify that the resources are in consistent state before
and after the adoption.

NOTE: We are skipping restore backup since it's failing even with 20
retries with 10 second delay. Will add that in a follow up PR after
resolving the issues.

softwarefactory-project-zuul bot commented Oct 10, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/4f2b18d208c34646b095f1658057761c

❌ adoption-standalone-to-crc-ceph FAILURE in 2h 43m 01s
❌ adoption-standalone-to-crc-no-ceph FAILURE in 2h 45m 26s

rajathere force-pushed the cinder_validation branch 3 times, most recently from 769d469 to 6763758 Compare

October 23, 2024 09:20

softwarefactory-project-zuul bot commented Oct 23, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/a9fd12c3a3e14a98a9e2d59fb171ad92

❌ adoption-standalone-to-crc-ceph FAILURE in 51m 07s
❌ adoption-standalone-to-crc-no-ceph FAILURE in 3h 10m 46s

rajathere force-pushed the cinder_validation branch from 6763758 to 6792d7c Compare

October 25, 2024 10:57

softwarefactory-project-zuul bot commented Oct 25, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/c22679b04ba042c7be2ac85f268a0878

❌ adoption-standalone-to-crc-ceph FAILURE in 3h 14m 19s
❌ adoption-standalone-to-crc-no-ceph FAILURE in 3h 13m 04s

rajathere force-pushed the cinder_validation branch from 6792d7c to fa3872d Compare

October 28, 2024 10:37

softwarefactory-project-zuul bot commented Oct 28, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/037f52b8c18643f5b2c0448cf954adf7

❌ adoption-standalone-to-crc-ceph FAILURE in 2h 45m 40s
❌ adoption-standalone-to-crc-no-ceph FAILURE in 2h 45m 44s

Contributor Author

rajathere commented Oct 29, 2024

recheck heat isssue shoud be fixed by https://review.rdoproject.org/r/c/rdo-jobs/+/55212

softwarefactory-project-zuul bot commented Oct 29, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/f75adcff76c247d8adbf76d9d3e3797c

❌ adoption-standalone-to-crc-ceph FAILURE in 3h 12m 06s
❌ adoption-standalone-to-crc-no-ceph FAILURE in 1h 11m 43s

rajathere force-pushed the cinder_validation branch from fa3872d to 0d01bc5 Compare

October 29, 2024 13:27

softwarefactory-project-zuul bot commented Oct 29, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/30661a6ae848471f9550b060689efaa2

❌ adoption-standalone-to-crc-ceph FAILURE in 3h 20m 42s
✔️ adoption-standalone-to-crc-no-ceph SUCCESS in 3h 08m 47s

rajathere force-pushed the cinder_validation branch from 0d01bc5 to cda6e92 Compare

November 5, 2024 09:57

jistr approved these changes

View reviewed changes

softwarefactory-project-zuul bot commented Nov 5, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/7d9b0f98880749a9ae7695e1a89b2c1e

❌ adoption-standalone-to-crc-ceph FAILURE in 3h 14m 20s
✔️ adoption-standalone-to-crc-no-ceph SUCCESS in 3h 13m 07s


          Validate Cinder operations

7ba0dba

After the adoption, we check for the correct status and size of the
Cinder resources but don't validate that operations still work on them.
This patch adds the following operations:
1. Detach 'disk' volume from 'test' server
2. Revert to snapshot 'snapshot' to 'disk' volume
3. Attach 'disk' volume to 'test' server

This will help verify that the resources are in consistent state before
and after the adoption.

NOTE: We are skipping restore backup since it's	failing	even with 20
retries	with 10 second delay. Will add that in a follow up PR after
resolving the issues.

rajathere force-pushed the cinder_validation branch from cda6e92 to 7ba0dba Compare

November 5, 2024 18:21

Contributor Author

rajathere commented Nov 7, 2024

All checks are passing and tasks being executed

TASK [dataplane_adoption : Detach the disk data volume] ************************
changed: [localhost] => {"changed": true, "cmd": "alias openstack="oc exec -t openstackclient -- openstack"\n\n${BASH_ALIASES[openstack]} server remove volume test disk\n", "delta": "0:00:03.570518", "end": "2024-11-05 16:10:50.637879", "msg": "", "rc": 0, "start": "2024-11-05 16:10:47.067361", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

TASK [dataplane_adoption : Wait for disk data volume to be available after detach] ***
changed: [localhost] => {"attempts": 1, "changed": true, "cmd": "alias openstack="oc exec -t openstackclient -- openstack"\n\n${BASH_ALIASES[openstack]} volume list -c Name -c Status -f value | grep "disk available"\n", "delta": "0:00:02.901613", "end": "2024-11-05 16:10:53.852860", "msg": "", "rc": 0, "start": "2024-11-05 16:10:50.951247", "stderr": "", "stderr_lines": [], "stdout": "disk available", "stdout_lines": ["disk available"]}

TASK [dataplane_adoption : Restore snapshot to disk volume] ********************
changed: [localhost] => {"changed": true, "cmd": "alias openstack="oc exec -t openstackclient -- openstack"\n\n${BASH_ALIASES[openstack]} --os-volume-api-version 3.40 volume revert snapshot\n", "delta": "0:00:02.297317", "end": "2024-11-05 16:10:56.485878", "msg": "", "rc": 0, "start": "2024-11-05 16:10:54.188561", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

TASK [dataplane_adoption : Wait for disk data volume to be available after revert] ***
changed: [localhost] => {"attempts": 1, "changed": true, "cmd": "alias openstack="oc exec -t openstackclient -- openstack"\n\n${BASH_ALIASES[openstack]} volume list -c Name -c Status -f value | grep "disk available"\n", "delta": "0:00:02.740869", "end": "2024-11-05 16:10:59.510535", "msg": "", "rc": 0, "start": "2024-11-05 16:10:56.769666", "stderr": "", "stderr_lines": [], "stdout": "disk available", "stdout_lines": ["disk available"]}

TASK [dataplane_adoption : Attach the disk data volume] ************************
changed: [localhost] => {"changed": true, "cmd": "alias openstack="oc exec -t openstackclient -- openstack"\n\n${BASH_ALIASES[openstack]} server add volume test disk\n", "delta": "0:00:03.519618", "end": "2024-11-05 16:11:03.343445", "msg": "", "rc": 0, "start": "2024-11-05 16:10:59.823827", "stderr": "", "stderr_lines": [], "stdout": "+-----------------------+--------------------------------------+\n| Field | Value |\n+-----------------------+--------------------------------------+\n| ID | 092e2eb2-68d1-41c5-9dbe-3cd103a4a33b |\n| Server ID | ac0ff6c9-7f8b-4bb8-85f4-693e7cc92fef |\n| Volume ID | 092e2eb2-68d1-41c5-9dbe-3cd103a4a33b |\n| Device | /dev/vdc |\n| Tag | None |\n| Delete On Termination | False |\n+-----------------------+--------------------------------------+", "stdout_lines": ["+-----------------------+--------------------------------------+", "| Field | Value |", "+-----------------------+--------------------------------------+", "| ID | 092e2eb2-68d1-41c5-9dbe-3cd103a4a33b |", "| Server ID | ac0ff6c9-7f8b-4bb8-85f4-693e7cc92fef |", "| Volume ID | 092e2eb2-68d1-41c5-9dbe-3cd103a4a33b |", "| Device | /dev/vdc |", "| Tag | None |", "| Delete On Termination | False |", "+-----------------------+--------------------------------------+"]}

TASK [dataplane_adoption : Wait for disk data volume to be in-use after attach] ***
changed: [localhost] => {"attempts": 1, "changed": true, "cmd": "alias openstack="oc exec -t openstackclient -- openstack"\n\n${BASH_ALIASES[openstack]} volume list -c Name -c Status -f value | grep "disk in-use"\n", "delta": "0:00:02.683598", "end": "2024-11-05 16:11:06.417319", "msg": "", "rc": 0, "start": "2024-11-05 16:11:03.733721", "stderr": "", "stderr_lines": [], "stdout": "disk in-use", "stdout_lines": ["disk in-use"]}

fmount approved these changes

View reviewed changes

Contributor

fmount left a comment

/lgtm

bogdando approved these changes

View reviewed changes

Contributor

bogdando left a comment

/lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet