From d4b241f451674e8f611bb589477680341006ad2b Mon Sep 17 00:00:00 2001 From: jack-berg <34418638+jack-berg@users.noreply.github.com> Date: Wed, 16 Aug 2023 14:25:39 -0500 Subject: [PATCH 1/2] Add synchronous gauge instrument, clarify temporality selection influence on metric point persistence (#3540) Resolves #2318. Extends the metrics API to include a new synchronous gauge instrument, whose value can be set as measurements occur. This is useful when the measurements are made accessible via subscription, such as JFR events. There are interesting questions around the behavior of `start_time_unix_nano` and whether or not sync gauges continue to export series without new measurements. This PR has gone through some iteration (available to see via commit history), but as it currently stands, I've made the following changes: - Add new language stating that metric reader's configurable aggregation temporality as a function of instrument type influences the persistence of metric points across successive collections, as described in this https://github.com/open-telemetry/opentelemetry-specification/pull/3540#issuecomment-1641011802. This influence occurs regardless of whether the resulting metric point has an aggregation temporality field. In other words, sync gauge persistence behavior is influenced by aggregation temporality selection. - Add new language stating that metric reader's configurable aggregation temporality as a function of instrument type influences the starting timestamp of metric points. When an instrument is cumulative, its starting timestamp will always be the same. When an instrument is delta, its starting timestamp advances with successive collections. This influence occurs regardless of whether the resulting metric point has an aggregation temporality field. In other words, sync gauge starting timestamp is influenced by aggregation temporality selection. --------- Co-authored-by: Reiley Yang --- CHANGELOG.md | 3 ++ specification/metrics/api.md | 95 ++++++++++++++++++++++++++++++++++++ specification/metrics/sdk.md | 46 ++++++++++++++--- 3 files changed, 137 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f6eb8152b1..024fae7e470 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,6 +66,9 @@ release. ([#3613](https://github.com/open-telemetry/opentelemetry-specification/pull/3613)) - Refine `MetricProvider.ForceFlush` and define `ForceFlush` for periodic exporting MetricReader. ([#3563](https://github.com/open-telemetry/opentelemetry-specification/pull/3563)) +- Add synchronous gauge instrument, clarify temporality selection influence on + metric point persistence. + ([#3540](https://github.com/open-telemetry/opentelemetry-specification/pull/3540)) ### Logs diff --git a/specification/metrics/api.md b/specification/metrics/api.md index f6824e83b47..0a33868236f 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -37,6 +37,10 @@ linkTitle: API + [Histogram creation](#histogram-creation) + [Histogram operations](#histogram-operations) - [Record](#record) + * [Gauge](#gauge) + + [Gauge creation](#gauge-creation) + + [Gauge operations](#gauge-operations) + - [Record](#record-1) * [Asynchronous Gauge](#asynchronous-gauge) + [Asynchronous Gauge creation](#asynchronous-gauge-creation) + [Asynchronous Gauge operations](#asynchronous-gauge-operations) @@ -774,6 +778,97 @@ httpServerDuration.Record(50, ("http.request.method", "POST"), ("url.scheme", "h httpServerDuration.Record(100, new HttpRequestAttributes { method = "GET", scheme = "http" }); ``` +### Gauge + +**Status**: [Experimental](../document-status.md) + +`Gauge` is a [synchronous Instrument](#synchronous-instrument-api) which can be +used to record non-additive value(s) (e.g. the background noise level - it makes +no sense to record the background noise level value from multiple rooms and sum +them up) when changes occur. + +Note: If the values are additive (e.g. the process heap size - it makes sense to +report the heap size from multiple processes and sum them up, so we get the +total heap usage), use [UpDownCounter](#asynchronous-updowncounter). + +Note: Synchronous Gauge is normally used when the measurements are exposed via a +subscription to change events ( +i.e. `backgroundNoiseLevel.onChange(value -> gauge.record(value))`). If the +measurement is exposed via an accessor, +use [Asynchronous Gauge](#asynchronous-gauge) to invoke the accessor in a +callback function ( +i.e. `createObservableGauge(observable -> observable.record(backgroundNoiseLevel.getCurrentValue()))`. + +Example uses for Gauge: + +* subscribe to change events for the background noise level +* subscribe to change events for the CPU fan speed + +#### Gauge creation + +There MUST NOT be any API for creating a `Gauge` other than with a +[`Meter`](#meter). This MAY be called `CreateGauge`. If strong type is +desired, [OpenTelemetry API](../overview.md#api) authors MAY decide the language +idiomatic name(s), for example `CreateUInt64Gauge`, `CreateDoubleGauge`, +`CreateGauge`, `CreateGauge`. + +See the [general requirements for synchronous instruments](#synchronous-instrument-api). + +Here are some examples that [OpenTelemetry API](../overview.md#api) authors +might consider: + +```java +// Java + +DoubleGauge backgroundNoiseLevel = meter.gaugeBuilder("facility.noise.level") + .setDescription("Background noise level of rooms") + .setUnit("B") + .build(); +``` + +#### Gauge operations + +##### Record + +Record the Gauge current value. + +This API SHOULD NOT return a value (it MAY return a dummy value if required by +certain programming languages or systems, for example `null`, `undefined`). + +This API MUST accept the following parameter: + +* A numeric value. The current absolute value. + + The value needs to be provided by a user. If possible, this API + SHOULD be structured so a user is obligated to provide this parameter. If it + is not possible to structurally enforce this obligation, this API MUST be + documented in a way to communicate to users that this parameter is needed. +* [Attributes](../common/README.md#attribute) to associate with the value. + + Users can provide attributes to associate with the value, but it is + up to their discretion. Therefore, this API MUST be structured to accept a + variable number of attributes, including none. + +The [OpenTelemetry API](../overview.md#api) authors MAY decide to allow flexible +[attributes](../common/README.md#attribute) to be passed in as arguments. If +the attribute names and types are provided during the [gauge +creation](#gauge-creation), the [OpenTelemetry API](../overview.md#api) +authors MAY allow attribute values to be passed in using a more efficient way +(e.g. strong typed struct allocated on the callstack, tuple). The API MUST allow +callers to provide flexible attributes at invocation time rather than having to +register all the possible attribute names during the instrument creation. Here +are some examples that [OpenTelemetry API](../overview.md#api) authors might +consider: + +```java +// Java +Attributes roomA = Attributes.builder().put("room.id", "Rack A"); +Attributes roomB = Attributes.builder().put("room.id", "Rack B"); + +backgroundNoiseLevel.record(4.3, roomA); +backgroundNoiseLevel.record(2.5, roomB); +``` + ### Asynchronous Gauge Asynchronous Gauge is an [asynchronous Instrument](#asynchronous-instrument-api) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index dc220c21570..c270295d8a1 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -527,6 +527,7 @@ an aggregation and `advice` to influence aggregation configuration parameters | [Asynchronous Counter](./api.md#asynchronous-counter) | [Sum Aggregation](./sdk.md#sum-aggregation) | | [UpDownCounter](./api.md#updowncounter) | [Sum Aggregation](./sdk.md#sum-aggregation) | | [Asynchronous UpDownCounter](./api.md#asynchronous-updowncounter) | [Sum Aggregation](./sdk.md#sum-aggregation) | +| [Gauge](./api.md#gauge) | [Last Value Aggregation](./sdk.md#last-value-aggregation) | | [Asynchronous Gauge](./api.md#asynchronous-gauge) | [Last Value Aggregation](./sdk.md#last-value-aggregation) | | [Histogram](./api.md#histogram) | [Explicit Bucket Histogram Aggregation](./sdk.md#explicit-bucket-histogram-aggregation), with `ExplicitBucketBoundaries` from [advice](./api.md#instrument-advice) if provided | @@ -703,7 +704,8 @@ given instrument before starting a subsequent round of collection. The implementation SHOULD NOT produce aggregated metric data for a previously-observed attribute set which is not observed during a successful -callback. +callback. See [MetricReader](#metricreader) for more details on the persistence +of metrics across successive collections. ### Cardinality limits @@ -1081,18 +1083,48 @@ typically with push-based metrics collection. The `MetricReader` MUST ensure that data points from OpenTelemetry [instruments](./api.md#instrument) are output in the configured aggregation -temporality for each instrument kind. For synchronous instruments being output -with Cumulative temporality, this means converting [Delta to Cumulative](supplementary-guidelines.md#synchronous-example-cumulative-aggregation-temporality) -aggregation temporality. For asynchronous instruments being output -with Delta temporality, this means converting [Cumulative to -Delta](supplementary-guidelines.md#asynchronous-example-delta-temporality) aggregation -temporality. +temporality for each instrument kind. For synchronous instruments with +Cumulative aggregation temporality, this means +converting [Delta to Cumulative](supplementary-guidelines.md#synchronous-example-cumulative-aggregation-temporality) +aggregation temporality. For asynchronous instruments with Delta temporality, +this means +converting [Cumulative to Delta](supplementary-guidelines.md#asynchronous-example-delta-temporality) +aggregation temporality. The `MetricReader` is not required to ensure data points from a non-SDK [MetricProducer](#metricproducer) are output in the configured aggregation temporality, as these data points are not collected using OpenTelemetry instruments. +The `MetricReader` selection of `temporality` as a function of instrument kind +influences the persistence of metric data points across collections. For +synchronous instruments with Cumulative aggregation +temporality, [MetricReader.Collect](#collect) MUST receive data points exposed +in previous collections regardless of whether new measurements have been +recorded. For synchronous instruments with Delta aggregation +temporality, [MetricReader.Collect](#collect) MUST only receive data points with +measurements recorded since the previous collection. For asynchronous +instruments with Delta or Cumulative aggregation +temporality, [MetricReader.Collect](#collect) MUST only receive data points with +measurements recorded since the previous collection. These rules apply to all +metrics, not just those whose [point kinds](./data-model.md#point-kinds) +includes an aggregation temporality field. + +The `MetricReader` selection of `temporality` as a function of instrument kind +influences the starting timestamp (i.e. `StartTimeUnixNano`) of metrics data +points received by [MetricReader.Collect](#collect). For instruments with +Cumulative aggregation temporality, successive data points received by +successive calls to [MetricReader.Collect](#collect) MUST repeat the same +starting timestamps (e.g. `(T0, T1], (T0, T2], (T0, T3]`). For instruments with +Delta aggregation temporality, successive data points received by successive +calls to [MetricReader.Collect](#collect) MUST advance the starting timestamp ( +e.g. `(T0, T1], (T1, T2], (T2, T3]`). The ending timestamp (i.e. `TimeUnixNano`) +MUST always be equal to time the metric data point took effect, which is equal +to when [MetricReader.Collect](#collect) was invoked. These rules apply to all +metrics, not just those whose [point kinds](./data-model.md#point-kinds) includes +an aggregation temporality field. +See [data model temporality](./data-model.md#temporality) for more details. + The SDK MUST support multiple `MetricReader` instances to be registered on the same `MeterProvider`, and the [MetricReader.Collect](#collect) invocation on one `MetricReader` instance SHOULD NOT introduce side-effects to other `MetricReader` From bc76c97525eef0f81550d939a3f87f44b182cef9 Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Wed, 16 Aug 2023 16:16:43 -0400 Subject: [PATCH 2/2] Clarify that metric advice is non-identifying (#3661) Fixes https://github.com/open-telemetry/opentelemetry-specification/issues/3622 The language for resolving conflicts matches the language used for naming conflicts. cc @MrAlias @jack-berg @jmacd --- CHANGELOG.md | 2 ++ specification/metrics/api.md | 2 +- specification/metrics/sdk.md | 5 +++++ 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 024fae7e470..7b48d55987c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,8 @@ release. ([#3242](https://github.com/open-telemetry/opentelemetry-specification/pull/3242)) - Promote MetricProducer specification to feature-freeze. ([#3600](https://github.com/open-telemetry/opentelemetry-specification/pull/3600)) +- Clarify that advice is non-identifying. + ([#3661](https://github.com/open-telemetry/opentelemetry-specification/pull/3661)) ### Logs diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 0a33868236f..33f2121fe4c 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -190,7 +190,7 @@ will have the following fields: * Optional `advice` (**experimental**) Instruments are associated with the Meter during creation. Instruments -are identified by all of these fields. +are identified by the `name`, `kind`, `unit`, and `description`. Language-level features such as the distinction between integer and floating point numbers SHOULD be considered as identifying. diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index c270295d8a1..371987b7b38 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -856,6 +856,11 @@ When a Meter creates an instrument, it SHOULD validate the instrument advice parameters. If an advice parameter is not valid, the Meter SHOULD emit an error notifying the user and proceed as if the parameter was not provided. +If multiple [identical Instruments](api.md#instrument) are created with +different advice parameters, the Meter MUST return an instrument using the +first-seen advice parameters and log an appropriate error as described in +[duplicate instrument registrations](#duplicate-instrument-registration). + ## Attribute limits **Status**: [Stable](../document-status.md)