From 5fe4b95931f6954795bdf6f96f52eb22ac3b8d7e Mon Sep 17 00:00:00 2001 From: Leif Madsen Date: Fri, 8 Sep 2023 09:56:58 -0400 Subject: [PATCH 1/2] Don't enable event collection by default on OSP Closes STF-1498 --- ...or-director-operator-for-the-overcloud.adoc | 8 -------- ...g-the-stf-connection-for-the-overcloud.adoc | 12 +----------- ...k-environment-file-for-multiple-clouds.adoc | 18 ++++-------------- ...guration-for-director-operator-for-stf.adoc | 4 ++-- ...reating-the-base-configuration-for-stf.adoc | 12 ++++++------ .../modules/proc_deploying-the-overcloud.adoc | 2 +- .../proc_planning-amqp-address-prefixes.adoc | 4 ++-- 7 files changed, 16 insertions(+), 44 deletions(-) diff --git a/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-director-operator-for-the-overcloud.adoc b/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-director-operator-for-the-overcloud.adoc index 6bd7550d..892d1317 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-director-operator-for-the-overcloud.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-director-operator-for-the-overcloud.adoc @@ -44,19 +44,11 @@ data: MetricsQdrSSLProfiles: - name: sslProfile - CeilometerQdrEventsConfig: - driver: amqp - topic: cloud1-event - CeilometerQdrMetricsConfig: driver: amqp topic: cloud1-metering CollectdAmqpInstances: - cloud1-notify: - notify: true - format: JSON - presettle: false cloud1-telemetry: format: JSON presettle: false diff --git a/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc b/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc index 484e7c45..4c1bbf12 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc @@ -2,7 +2,7 @@ = Configuring the {ProjectShort} connection for the overcloud [role="_abstract"] -To configure the {Project} ({ProjectShort}) connection, you must create a file that contains the connection configuration of the {MessageBus} for the overcloud to the {ProjectShort} deployment. Enable the collection of events and storage of the events in {ProjectShort} and deploy the overcloud. The default configuration is for a single cloud instance with the default message bus topics. For configuration of multiple cloud deployments, see xref:configuring-multiple-clouds_assembly-completing-the-stf-configuration[]. +To configure the {Project} ({ProjectShort}) connection, you must create a file that contains the connection configuration of the {MessageBus} for the overcloud to the {ProjectShort} deployment. Enable the collection of metrics and storage of the metrics in {ProjectShort} and deploy the overcloud. The default configuration is for a single cloud instance with the default message bus topics. For configuration of multiple cloud deployments, see xref:configuring-multiple-clouds_assembly-completing-the-stf-configuration[]. .Prerequisites @@ -48,19 +48,11 @@ ifdef::include_when_13,include_when_17[] -----END CERTIFICATE----- endif::include_when_13,include_when_17[] - CeilometerQdrEventsConfig: - driver: amqp - topic: cloud1-event - CeilometerQdrMetricsConfig: driver: amqp topic: cloud1-metering CollectdAmqpInstances: - cloud1-notify: - notify: true - format: JSON - presettle: false cloud1-telemetry: format: JSON presettle: false @@ -76,9 +68,7 @@ ifdef::include_when_13,include_when_17[] * Replace the `caCertFileContent` parameter with the contents retrieved in xref:getting-ca-certificate-from-stf-for-overcloud-configuration_assembly-completing-the-stf-configuration[]. endif::include_when_13,include_when_17[] * Replace the `host` sub-parameter of `MetricsQdrConnectors` with the value that you retrieved in xref:retrieving-the-qdr-route-address_assembly-completing-the-stf-configuration[]. -* Set `topic` value of `CeilometerQdrEventsConfig` to define the topic for Ceilometer events. The value is a unique topic idenifier for the cloud such as `cloud1-event`. * Set `topic` value of `CeilometerQdrMetricsConfig.topic` to define the topic for Ceilometer metrics. The value is a unique topic identifier for the cloud such as `cloud1-metering`. -* Set `CollectdAmqpInstances` sub-paramter to define the topic for collectd events. The section name is a unique topic identifier for the cloud such as `cloud1-notify`. * Set `CollectdAmqpInstances` sub-parameter to define the topic for collectd metrics. The section name is a unique topic identifier for the cloud such as `cloud1-telemetry`. ifndef::include_when_13[] * Set `CollectdSensubilityResultsChannel` to define the topic for collectd-sensubility events. The value is a unique topic identifier for the cloud such as `sensubility/cloud1-telemetry`. diff --git a/doc-Service-Telemetry-Framework/modules/proc_creating-openstack-environment-file-for-multiple-clouds.adoc b/doc-Service-Telemetry-Framework/modules/proc_creating-openstack-environment-file-for-multiple-clouds.adoc index 2371ae5d..61e9bcb4 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_creating-openstack-environment-file-for-multiple-clouds.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_creating-openstack-environment-file-for-multiple-clouds.adoc @@ -24,7 +24,7 @@ = Creating the {OpenStack} environment file for multiple clouds [role="_abstract"] -To label traffic according to the cloud of origin, you must create a configuration with cloud-specific instance names. Create an `stf-connectors.yaml` file and adjust the values of `CeilometerQdrEventsConfig`, `CeilometerQdrMetricsConfig` and `CollectdAmqpInstances` to match the AMQP address prefix scheme. +To label traffic according to the cloud of origin, you must create a configuration with cloud-specific instance names. Create an `stf-connectors.yaml` file and adjust the values of `CeilometerQdrMetricsConfig` and `CollectdAmqpInstances` to match the AMQP address prefix scheme. ifdef::include_when_16[] [NOTE] @@ -46,10 +46,10 @@ ifdef::include_when_13,include_when_17[* You have retrieved the CA certificate f . Create a configuration file called `stf-connectors.yaml` in the `/home/stack` directory. ifdef::include_when_13[] -. In the `stf-connectors.yaml` file, configure the `MetricsQdrConnectors` address to connect to the {MessageBus} on the overcloud deployment. Configure the `CeilometerQdrEventsConfig`, `CeilometerQdrMetricsConfig`, and `CollectdAmqpInstances` topic values to match the AMQP address that you want for this cloud deployment. +. In the `stf-connectors.yaml` file, configure the `MetricsQdrConnectors` address to connect to the {MessageBus} on the overcloud deployment. Configure the `CeilometerQdrMetricsConfig`, and `CollectdAmqpInstances` topic values to match the AMQP address that you want for this cloud deployment. endif::include_when_13[] ifndef::include_when_13[] -. In the `stf-connectors.yaml` file, configure the `MetricsQdrConnectors` address to connect to the {MessageBus} on the overcloud deployment. Configure the `CeilometerQdrEventsConfig`, `CeilometerQdrMetricsConfig`, `CollectdAmqpInstances`, and `CollectdSensubilityResultsChannel` topic values to match the AMQP address that you want for this cloud deployment. +. In the `stf-connectors.yaml` file, configure the `MetricsQdrConnectors` address to connect to the {MessageBus} on the overcloud deployment. Configure the `CeilometerQdrMetricsConfig`, `CollectdAmqpInstances`, and `CollectdSensubilityResultsChannel` topic values to match the AMQP address that you want for this cloud deployment. endif::include_when_13[] + // The following configuration should match the contents in modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc. If you have changes to make, please make the same changes to both files. @@ -77,19 +77,11 @@ ifdef::include_when_13,include_when_17[] -----END CERTIFICATE----- endif::include_when_13,include_when_17[] - CeilometerQdrEventsConfig: - driver: amqp - topic: cloud1-event - CeilometerQdrMetricsConfig: driver: amqp topic: cloud1-metering CollectdAmqpInstances: - cloud1-notify: - notify: true - format: JSON - presettle: false cloud1-telemetry: format: JSON presettle: false @@ -105,9 +97,7 @@ ifdef::include_when_13,include_when_17[] * Replace the `caCertFileContent` parameter with the contents retrieved in xref:getting-ca-certificate-from-stf-for-overcloud-configuration_assembly-completing-the-stf-configuration[]. endif::include_when_13,include_when_17[] * Replace the `host` sub-parameter of `MetricsQdrConnectors` with the value that you retrieved in xref:retrieving-the-qdr-route-address_assembly-completing-the-stf-configuration[]. -* Set `topic` value of `CeilometerQdrEventsConfig` to define the topic for Ceilometer events. The value is a unique topic idenifier for the cloud such as `cloud1-event`. * Set `topic` value of `CeilometerQdrMetricsConfig.topic` to define the topic for Ceilometer metrics. The value is a unique topic identifier for the cloud such as `cloud1-metering`. -* Set `CollectdAmqpInstances` sub-paramter to define the topic for collectd events. The section name is a unique topic identifier for the cloud such as `cloud1-notify`. * Set `CollectdAmqpInstances` sub-parameter to define the topic for collectd metrics. The section name is a unique topic identifier for the cloud such as `cloud1-telemetry`. ifndef::include_when_13[] * Set `CollectdSensubilityResultsChannel` to define the topic for collectd-sensubility events. The value is a unique topic identifier for the cloud such as `sensubility/cloud1-telemetry`. @@ -123,7 +113,7 @@ ifndef::include_when_13[The value for sensubility is the full topic path and has For an example of a cloud configuration in the `ServiceTelemetry` object referring to the full topic address, see xref:clouds_assembly-installing-the-core-components-of-stf[]. ==== -. Ensure that the naming convention in the `stf-connectors.yaml` file aligns with the `spec.bridge.amqpUrl` field in the Smart Gateway configuration. For example, configure the `CeilometerQdrEventsConfig.topic` field to a value of `cloud1-event`. +. Ensure that the naming convention in the `stf-connectors.yaml` file aligns with the `spec.bridge.amqpUrl` field in the Smart Gateway configuration. For example, configure the `CeilometerQdrMetricsConfig.topic` field to a value of `cloud1-metering`. . Log in to the undercloud host as the `stack` user. diff --git a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc index f99f2fca..a4d942b1 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc @@ -41,7 +41,7 @@ data: # enable Ceilometer metrics and events CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: true + CeilometerQdrPublishEvents: false # enable collection of API status CollectdEnableSensubility: true @@ -58,7 +58,7 @@ data: CollectdExtraPlugins: - vmem - # set standard prefixes for where metrics and events are published to QDR + # set standard prefixes for where metrics are published to QDR MetricsQdrAddresses: - prefix: 'collectd' distribution: multicast diff --git a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc index 47090273..25b41fcd 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc @@ -28,9 +28,9 @@ parameter_defaults: ManagePolling: true ManagePipeline: true - # enable Ceilometer metrics and events + # enable Ceilometer metrics CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: true + CeilometerQdrPublishEvents: false # set collectd overrides for higher telemetry resolution and extra plugins to load CollectdConnectionType: amqp1 @@ -39,7 +39,7 @@ parameter_defaults: CollectdExtraPlugins: - vmem - # set standard prefixes for where metrics and events are published to QDR + # set standard prefixes for where metrics are published to QDR MetricsQdrAddresses: - prefix: 'collectd' distribution: multicast @@ -104,9 +104,9 @@ parameter_defaults: ManagePolling: true ManagePipeline: true - # enable Ceilometer metrics and events + # enable Ceilometer metrics CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: true + CeilometerQdrPublishEvents: false # enable collection of API status CollectdEnableSensubility: true @@ -123,7 +123,7 @@ parameter_defaults: CollectdExtraPlugins: - vmem - # set standard prefixes for where metrics and events are published to QDR + # set standard prefixes for where metrics are published to QDR MetricsQdrAddresses: - prefix: 'collectd' distribution: multicast diff --git a/doc-Service-Telemetry-Framework/modules/proc_deploying-the-overcloud.adoc b/doc-Service-Telemetry-Framework/modules/proc_deploying-the-overcloud.adoc index a8a3ba7b..bbea3297 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_deploying-the-overcloud.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_deploying-the-overcloud.adoc @@ -27,7 +27,7 @@ $ source ~/stackrc -e /home/stack/stf-connectors.yaml ---- -* Include the `ceilometer-write-qdr.yaml` file to ensure that Ceilometer telemetry and events are sent to {ProjectShort}. +* Include the `ceilometer-write-qdr.yaml` file to ensure that Ceilometer telemetry is sent to {ProjectShort}. * Include the `qdr-edge-only.yaml` file to ensure that the message bus is enabled and connected to {ProjectShort} message bus routers. * Include the `enable-stf.yaml` environment file to ensure that the defaults are configured correctly. * Include the `stf-connectors.yaml` environment file to define the connection to {ProjectShort}. diff --git a/doc-Service-Telemetry-Framework/modules/proc_planning-amqp-address-prefixes.adoc b/doc-Service-Telemetry-Framework/modules/proc_planning-amqp-address-prefixes.adoc index 3783254a..82d9c0ce 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_planning-amqp-address-prefixes.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_planning-amqp-address-prefixes.adoc @@ -4,7 +4,7 @@ [role="_abstract"] ifdef::include_when_16[] -By default, {OpenStack} ({OpenStackShort}) nodes receive data through two data collectors; collectd and Ceilometer. The collectd-sensubility plugin requires a unique address. These components send telemetry data or notifications to the respective AMQP addresses, for example, `collectd/telemetry`. {ProjectShort} Smart Gateways listen on those AMQP addresses for data. To support multiple clouds and to identify which cloud generated the monitoring data, configure each cloud to send data to a unique address. Add a cloud identifier prefix to the second part of the address. The following list shows some example addresses and identifiers: +By default, {OpenStack} ({OpenStackShort}) nodes retrieve data through two data collectors; collectd and Ceilometer. The collectd-sensubility plugin requires a unique address. These components send telemetry data or notifications to the respective AMQP addresses, for example, `collectd/telemetry`. {ProjectShort} Smart Gateways listen on those AMQP addresses for data. To support multiple clouds and to identify which cloud generated the monitoring data, configure each cloud to send data to a unique address. Add a cloud identifier prefix to the second part of the address. The following list shows some example addresses and identifiers: * `collectd/cloud1-telemetry` * `collectd/cloud1-notify` @@ -21,7 +21,7 @@ By default, {OpenStack} ({OpenStackShort}) nodes receive data through two data c endif::include_when_16[] ifdef::include_when_13[] -By default, {OpenStack} nodes receive data through two data collectors; collectd and Ceilometer. These components send telemetry data or notifications to the respective AMQP addresses, for example, collectd/telemetry. {ProjectShort} Smart Gateways listen on the AMQP addresses for monitoring data. To support multiple clouds and to identify which cloud generated the monitoring data, configure each cloud to send data to a unique address. Add a cloud identifier prefix to the second part of the address. The following list shows some example addresses and identifiers: +By default, {OpenStack} nodes retrieve data through two data collectors; collectd and Ceilometer. These components send telemetry data or notifications to the respective AMQP addresses, for example, collectd/telemetry. {ProjectShort} Smart Gateways listen on the AMQP addresses for monitoring data. To support multiple clouds and to identify which cloud generated the monitoring data, configure each cloud to send data to a unique address. Add a cloud identifier prefix to the second part of the address. The following list shows some example addresses and identifiers: * `collectd/cloud1-telemetry` * `collectd/cloud1-notify` From 379551fa6acdf02c1c5925ef925ff8ee326bfbb4 Mon Sep 17 00:00:00 2001 From: Leif Madsen Date: Mon, 11 Sep 2023 14:42:30 -0400 Subject: [PATCH 2/2] Remove events configurations and use defaults The defaults for events pipeline and Ceilometer QDR publish events is disabled and does not need to be called out specifically. --- ...he-base-configuration-for-director-operator-for-stf.adoc | 2 -- .../proc_creating-the-base-configuration-for-stf.adoc | 6 +----- 2 files changed, 1 insertion(+), 7 deletions(-) diff --git a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc index a4d942b1..ee04f485 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc @@ -32,7 +32,6 @@ data: enable-stf.yaml: | parameter_defaults: # only send to STF, not other publishers - EventPipelinePublishers: [] PipelinePublishers: [] # manage the polling and pipeline configuration files for Ceilometer agents @@ -41,7 +40,6 @@ data: # enable Ceilometer metrics and events CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: false # enable collection of API status CollectdEnableSensubility: true diff --git a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc index 25b41fcd..537bee27 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc @@ -12,7 +12,7 @@ To configure the base parameters to provide a compatible data collection and tra + [IMPORTANT] ==== -Setting `EventPipelinePublishers` and `PipelinePublishers` to empty lists results in no event or metric data passing to {OpenStackShort} telemetry components, such as Gnocchi or Panko. If you need to send data to additional pipelines, the Ceilometer polling interval of `30` seconds, that you specify in `ExtraConfig`, might overwhelm the {OpenStackShort} telemetry components. You must increase the interval to a larger value, such as `300`, which results in less telemetry resolution in {ProjectShort}. +Setting `PipelinePublishers` to an empty list results in no metric data passing to {OpenStackShort} telemetry components, such as Gnocchi or Panko. If you need to send data to additional pipelines, the Ceilometer polling interval of `30` seconds, that you specify in `ExtraConfig`, might overwhelm the {OpenStackShort} telemetry components. You must increase the interval to a larger value, such as `300`, which results in less telemetry resolution in {ProjectShort}. ==== + .enable-stf.yaml @@ -21,7 +21,6 @@ ifdef::include_when_13[] ---- parameter_defaults: # only send to STF, not other publishers - EventPipelinePublishers: [] PipelinePublishers: [] # manage the polling and pipeline configuration files for Ceilometer agents @@ -30,7 +29,6 @@ parameter_defaults: # enable Ceilometer metrics CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: false # set collectd overrides for higher telemetry resolution and extra plugins to load CollectdConnectionType: amqp1 @@ -97,7 +95,6 @@ ifdef::include_when_16[] ---- parameter_defaults: # only send to STF, not other publishers - EventPipelinePublishers: [] PipelinePublishers: [] # manage the polling and pipeline configuration files for Ceilometer agents @@ -106,7 +103,6 @@ parameter_defaults: # enable Ceilometer metrics CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: false # enable collection of API status CollectdEnableSensubility: true