From 3def92ce9592c68ab4d9ad562591ac62b0a466fb Mon Sep 17 00:00:00 2001 From: Bohdan Dobrelia Date: Fri, 29 Sep 2023 14:39:00 +0200 Subject: [PATCH] WIP Document Nova services adoption (no extra cell) Note about remapping cell names from OSP cells naming scheme to the NG scheme with the superconductor layout. Signed-off-by: Bohdan Dobrelia --- docs/contributing/development_environment.md | 35 +++- docs/openstack/backend_services_deployment.md | 94 +++++++-- docs/openstack/nova_adoption.md | 187 ++++++++++++++++++ 3 files changed, 301 insertions(+), 15 deletions(-) create mode 100644 docs/openstack/nova_adoption.md diff --git a/docs/contributing/development_environment.md b/docs/contributing/development_environment.md index c2652e424..725d3c70c 100644 --- a/docs/contributing/development_environment.md +++ b/docs/contributing/development_environment.md @@ -53,9 +53,40 @@ to create a virtual machine connected to the isolated networks. Create the edpm-compute-0 virtual machine. ``` cd install_yamls/devsetup -make standalone +make standalone EDPM_COMPUTE_CEPH_ENABLED=false ``` +**TODO (Q1/24)** + + # Deployment of a multi-stack topology for Nova cells v2 + + Create the edpm-compute-1 and edpm-compute-2 virtual machinee for Nova compute cells. + ``` + make edpm_compute EDPM_COMPUTE_SUFFIX=1 + make edpm_compute_repos EDPM_COMPUTE_SUFFIX=1 + ``` + + Omit the `edpm_deploy` make target to not making it managed from the control + plane running on OCP. + + Instead, deploy it as a 2nd TripleO standalone Heat stack, + with an extra OSP compute: + ``` + make standalone EDPM_COMPUTE_SUFFIX=1 EDPM_COMPUTE_CEPH_ENABLED=false + ``` + + Ssh into deployed standalone host: + ``` + ssh -i ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa root@192.168.122.100 + ``` + Discover the deployed compute host keys (a passwordless access is required by + the Nova Live-migration feature): + ``` + ssh-copy-id -i ~/.ssh/id_rsa root@192.168.122.101 + ``` + + TODO Discover the remote compute node from the central controller node + ### Convenience steps To make our life easier we can copy the deployment passwords we'll be using @@ -116,6 +147,8 @@ export OS_CLOUD=standalone source ~/install_yamls/devsetup/scripts/edpm-deploy-instance.sh ``` +TODO(bogdando): document creating a 2nd instance on another compute cell + Confirm the image UUID can be seen in Ceph's images pool. ``` ssh -i ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa root@192.168.122.100 sudo cephadm shell -- rbd -p images ls -l diff --git a/docs/openstack/backend_services_deployment.md b/docs/openstack/backend_services_deployment.md index 056b48107..48a22b39c 100644 --- a/docs/openstack/backend_services_deployment.md +++ b/docs/openstack/backend_services_deployment.md @@ -62,6 +62,59 @@ podified OpenStack control plane services. ## Pre-checks +> **NOTE** The multi-cells v2 topology under adoption testinig is a central stack without multiple cells. Multi-stack topology will result +> in different cells DBs mappings and will be using a different naming scheme. + +Get the database_connection and transport_url cells mappings on the source OSP environment, before those get renamed during +the [databases adoption](mariadb_copy.md). Note those below to compare it later with the renamed post-adoption values. Post-adoption, the `nova_cell0` mapping should remain intact, while the `nova` one should get renamed to `nova_cell1`. The `nova_api` DB (not shown here) should retain its name as well. + +Firstly, check the templated view stored in the galera database of the source environment: +```bash +> podman exec -it galera-bundle-podman-0 mysql -uroot nova_api -e"select uuid,name,transport_url,database_connection,disabled from cell_mappings" | grep '|' +| uuid | name | transport_url | database_connection | disabled | +| 00000000-0000-0000-0000-000000000000 | cell0 | none:/// | mysql+pymysql://{username}:{password}@192.168.122.99/nova_cell0?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo | 0 | +| 58036269-e862-4b3d-943d-2dbd0fc2d915 | default | rabbit://{username}:{password}@standalone.ctlplane.localdomain:5672/?ssl=0 | mysql+pymysql://{username}:{password}@192.168.122.99/nova?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo | 0 | +``` + +Then note its Nova representation (showing names and passwords): +```bash +> podman exec -it nova_api nova-manage cell_v2 list_cells --verbose | grep '|' +| Name | UUID | Transport URL | Database Connection | Disabled | +| cell0 | 00000000-0000-0000-0000-000000000000 | none:/// | mysql+pymysql://nova:4w6xaYHjGSGJDv66z7xhHQioz@192.168.122.99/nova_cell0?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo | False | +| default | 58036269-e862-4b3d-943d-2dbd0fc2d915 | rabbit://guest:FxEbJ5yKxaGiYthgI0PJFk3y1@standalone.ctlplane.localdomain:5672/?ssl=0 | mysql+pymysql://nova:4w6xaYHjGSGJDv66z7xhHQioz@192.168.122.99/nova?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo | False | +``` + +Also note the databases list on the source environment: +```bash +> podman exec -it galera-bundle-podman-0 mysql -uroot nova_api -e"show databases;" ++--------------------+ +| Database | ++--------------------+ +| cinder | +| glance | +| information_schema | +| keystone | +| mysql | +| nova | +| nova_api | +| nova_cell0 | +| ovs_neutron | +| performance_schema | +| placement | ++--------------------+ +``` +Note how the `nova`, `nova_api`, `nova_cell0` databases reside in the same DB instance of the central standalone stack. + +Finally, note the cell ID for the started earlier workload: +``` +> podman exec -it galera-bundle-podman-0 mysql -uroot nova_api -e"select cell_id from nova_api.instance_mappings;" ++---------+ +| cell_id | ++---------+ +| 2 | ++---------+ +``` + ## Procedure - backend services deployment * Make sure you are using the OpenShift namespace where you want the @@ -127,10 +180,15 @@ podified OpenStack control plane services. dns: enabled: true template: - externalEndpoints: - - ipAddressPool: ctlplane - loadBalancerIPs: - - 192.168.122.80 + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 192.168.122.80 + spec: + type: LoadBalancer options: - key: server values: @@ -200,18 +258,26 @@ podified OpenStack control plane services. rabbitmq: templates: rabbitmq: - externalEndpoint: - loadBalancerIPs: - - 172.17.0.85 - ipAddressPool: internalapi - sharedIP: false + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.85 + spec: + type: LoadBalancer replicas: 1 rabbitmq-cell1: - externalEndpoint: - loadBalancerIPs: - - 172.17.0.86 - ipAddressPool: internalapi - sharedIP: false + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.86 + spec: + type: LoadBalancer replicas: 1 telemetry: diff --git a/docs/openstack/nova_adoption.md b/docs/openstack/nova_adoption.md new file mode 100644 index 000000000..6fc2f96e8 --- /dev/null +++ b/docs/openstack/nova_adoption.md @@ -0,0 +1,187 @@ +# Nova adoption + +## Prerequisites + +* Previous Adoption steps completed. Notably, the service databases + must already be imported into the podified MariaDB. See [related + documentation]() + +## Variables + +(There are no shell variables necessary currently.) + +## Procedure - Nova adoption + +* Patch OpenStackControlPlane to deploy Nova: + + ``` + oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' + spec: + nova: + enabled: true + secret: osp-secret + databaseInstance: openstack + apiOverride: # FIXME: could this be skipped? + route: {} + template: + cellTemplates: + cell0: + cellDatabaseUser: nova_cell0 # renamed from OSP17 'nova' + noVNCProxyServiceTemplate: + replicas: 0 # must remain 0? + metadataServiceTemplate: + replicas: 0 # must remain 0? + conductorServiceTemplate: + replicas: 0 # must remain 0 + externalEndpoints: + - endpoint: internal + ipAddressPool: internalapi + loadBalancerIPs: + - 172.17.0.80 + hasAPIAccess: true + cell1: + cellDatabaseUser: nova_cell1 # renamed from OSP17 'nova_cell0' + cellDatabaseInstance: openstack-cell1 + cellMessageBusInstance: rabbitmq-cell1 + noVNCProxyServiceTemplate: + replicas: 1 + metadataServiceTemplate: + replicas: 1 + conductorServiceTemplate: + replicas: 1 + hasAPIAccess: true + externalEndpoints: + - endpoint: internal + ipAddressPool: internalapi + loadBalancerIPs: + - 172.17.0.80 + apiServiceTemplate: + replicas: 1 + override: + service: + internal: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + metadataServiceTemplate: + replicas: 1 + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + schedulerServiceTemplate: + replicas: 1 + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + noVNCProxyServiceTemplate: + replicas: 1 #FIXME: or only need to set if just for cell1? + # FIXME: not needed for VNCservice? + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + ' + ``` + +## Post-checks + +> **NOTE** Post-adoption, there should be no compute hosts in the central (standalone) stack for +> the only supported multi-cell topology there. + +The former pre-adoption central cell's `nova` DB (in terms of OSP) is going to +be no longer used. + +FIXME: How to/should we adopt Nova compute hosts from the +central stack of the source envirnment, if any? + +Verify the other cells databases naming (should become unchanged): +``` +nova_api -> nova_api +nova_cell0 -> nova_cell0 +nova_cell1 -> nova_cell1 +nova -> no longer in use FXIME: its records should migrate to? +``` + +* See that Nova endpoints are defined and pointing to the + podified FQDNs and that Nova API responds. + + ```bash + alias openstack="oc exec -t openstackclient -- openstack" + + openstack endpoint list | grep nova + + # Without OpenStack client for Nova installed: + NOVA_PUBLIC_URL=$(openstack endpoint list -c 'Service Name' -c 'Service Type' -c URL | grep nova | grep public | awk '{ print $6; }') + curl "$NOVA_PUBLIC_URL" + + # With OpenStack client for Nova installed: + openstack server list + ``` + +* Check the cells mappings, compare it to the values [noted earlier](backend_services_deployment.md) + + ```bash + > oc rsh openstack-galera-0 mysql -u root --password=12345678 -e 'select * from nova_api.cell_mappings;' + Defaulted container "galera" out of: galera, mysql-bootstrap (init) + +---------------------+---------------------+----+--------------------------------------+-------+--------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+----------+ + | created_at | updated_at | id | uuid | name | transport_url | database_connection | disabled | + +---------------------+---------------------+----+--------------------------------------+-------+--------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+----------+ + | 2023-10-10 11:31:22 | 2023-10-10 11:31:23 | 1 | 00000000-0000-0000-0000-000000000000 | cell0 | rabbit:// | mysql+pymysql://nova_cell0:12345678@openstack/nova_cell0 | 0 | + | 2023-10-10 11:31:43 | NULL | 2 | 10e461ba-b9de-4545-b180-11e03915ca3f | cell1 | rabbit://default_user_WKaxHjAoXETXcRzQQX2:RVwgQpE3BvK6jnmOQ_uTVHFE2mnMJ5Sd@rabbitmq-cell1.openstack.svc:5672 | mysql+pymysql://nova_cell1:12345678@openstack-cell1/nova_cell1 | 0 | + +---------------------+---------------------+----+--------------------------------------+-------+--------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+----------+ + ``` + Note how cell1's `nova` DB became `nova_cell1`. + TODO bring the nova-manage list cells equivalent + + ``` + > oc rsh openstack-galera-0 mysql -u root --password=12345678 -e 'show databases;' + Defaulted container "galera" out of: galera, mysql-bootstrap (init) + +--------------------+ + | Database | + +--------------------+ + | cinder | + | glance | + | information_schema | + | keystone | + | mysql | + | neutron | + | nova_api | + | nova_cell0 | + | performance_schema | + | placement | + +--------------------+ + > oc rsh openstack-cell1-galera-0 mysql -u root --password=12345678 -e 'show databases;' + Defaulted container "galera" out of: galera, mysql-bootstrap (init) + +--------------------+ + | Database | + +--------------------+ + | information_schema | + | mysql | + | nova_cell1 | + | performance_schema | + +--------------------+ + ``` + Note how cells' databases has been assigned to corresponding DB instances. + + TODO: check for a VM previosely running in 'nova' cell, should be now in `nova_cell1`