From 6e6f91014c64c5c612e95197f0633c596e26d264 Mon Sep 17 00:00:00 2001 From: jack-berg <34418638+jack-berg@users.noreply.github.com> Date: Mon, 29 Jan 2024 13:55:42 -0600 Subject: [PATCH] Define sdk extension component support in file configuration (#3802) Part of incorporating [OTEP #225](https://github.com/open-telemetry/oteps/pull/225) into the specification. Followup to #3744. This defines how file configuration works with custom SDK extension components (Samplers, Exporters, etc). It defines the concept of a Component Provider: - Component providers are registered with the type of extension component they provide and a name. Component providers are registered automatically or manually based on what is idiomatic in the language. - Component providers have a Create Plugin method, which passes configuration properties as a parameter and returns the configured component - When Create is called to interpret a file configuration model, and it comes across a reference to a extension component which is not built-in, it invokes Create Plugin on the corresponding component provider. If no corresponding component provider exists, or if Create Plugin returns an Error, Create returns an error. Prototype implementation in java here: https://github.com/open-telemetry/opentelemetry-java-examples/pull/227 cc @open-telemetry/configuration-maintainers --- CHANGELOG.md | 2 + spec-compliance-matrix.md | 27 ++-- .../configuration/file-configuration.md | 152 ++++++++++++++++-- 3 files changed, 155 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 73cdb9ca2bf..c78bff6ac42 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,8 @@ release. - Add file configuration section to spec compliance matrix. ([#3804](https://github.com/open-telemetry/opentelemetry-specification/pull/3804)) +- Define mechanism for SDK extension components. + ([#3802](https://github.com/open-telemetry/opentelemetry-specification/pull/3802)) ### Common diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md index c9950b9e5e3..f19a36477f9 100644 --- a/spec-compliance-matrix.md +++ b/spec-compliance-matrix.md @@ -313,18 +313,21 @@ Note: Support for environment variables is optional. See [File Configuration](./specification/configuration/file-configuration.md) for details. -| Feature | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | -|------------------------------------------------------------------|----|------|----|--------|------|--------|-----|------|-----|------|-------| -| `Parse` a configuration file | | | | | | | | | | | | -| The `Parse` operation accepts the configuration YAML file format | | | | | | | | | | | | -| The `Parse` operation performs environment variable substitution | | | | | | | | | | | | -| The `Parse` operation returns configuration model | | | | | | | | | | | | -| `Create` SDK components | | | | | | | | | | | | -| The `Create` operation accepts configuration model | | | | | | | | | | | | -| The `Create` operation returns `TracerProvider` | | | | | | | | | | | | -| The `Create` operation returns `MeterProvider` | | | | | | | | | | | | -| The `Create` operation returns `LoggerProvider` | | | | | | | | | | | | -| The `Create` operation returns `Propagators` | | | | | | | | | | | | +| Feature | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | +|-------------------------------------------------------------------------------------------------------------------------|----|------|----|--------|------|--------|-----|------|-----|------|-------| +| `Parse` a configuration file | | | | | | | | | | | | +| The `Parse` operation accepts the configuration YAML file format | | | | | | | | | | | | +| The `Parse` operation performs environment variable substitution | | | | | | | | | | | | +| The `Parse` operation returns configuration model | | | | | | | | | | | | +| The `Parse` operation resolves extension component configuration to `properties` | | | | | | | | | | | | +| `Create` SDK components | | | | | | | | | | | | +| The `Create` operation accepts configuration model | | | | | | | | | | | | +| The `Create` operation returns `TracerProvider` | | | | | | | | | | | | +| The `Create` operation returns `MeterProvider` | | | | | | | | | | | | +| The `Create` operation returns `LoggerProvider` | | | | | | | | | | | | +| The `Create` operation returns `Propagators` | | | | | | | | | | | | +| The `Create` operation calls `CreatePlugin` of corresponding `ComponentProvider` when encountering extension components | | | | | | | | | | | | +| Register a `ComponentProvider` | | | | | | | | | | | | ## Exporters diff --git a/specification/configuration/file-configuration.md b/specification/configuration/file-configuration.md index 8b2bcec2564..b530ee73ab1 100644 --- a/specification/configuration/file-configuration.md +++ b/specification/configuration/file-configuration.md @@ -16,9 +16,13 @@ linkTitle: File * [Environment variable substitution](#environment-variable-substitution) - [SDK Configuration](#sdk-configuration) * [In-Memory Configuration Model](#in-memory-configuration-model) + * [SDK Extension Components](#sdk-extension-components) + + [Component Provider](#component-provider) + + [Create Plugin](#create-plugin) * [Operations](#operations) + [Parse](#parse) + [Create](#create) + + [Register Component Provider](#register-component-provider) - [References](#references) @@ -142,6 +146,93 @@ to provide this in-memory representation in a manner that is idiomatic for their language. If an SDK needs to expose a class or interface, the name `Configuration` is RECOMMENDED. +### SDK Extension Components + +The SDK supports a variety of +extension [plugin interfaces](../glossary.md#sdk-plugins), allowing users and +libraries to customize behaviors including the sampling, processing, and +exporting of data. In general, the [configuration model](#configuration-model) +defines specific types for built-in implementations of these plugin interfaces. +For example, +the [BatchSpanProcessor](https://github.com/open-telemetry/opentelemetry-configuration/blob/f38ac7c3a499ae5f81924ef9c455c27a56130562/schema/tracer_provider.json#L22) +type refers to the +built-in [Batching span processor](../trace/sdk.md#batching-processor). The +schema SHOULD also support the ability to specify custom implementations of +plugin interfaces defined by libraries or users. + +For example, a custom [span exporter](../trace/sdk.md#span-exporter) might be configured as follows: + +```yaml +tracer_provider: + processors: + - batch: + exporter: + my-exporter: + config-parameter: value +``` + +Here we specify that the tracer provider has a batch span processor +paired with a custom span exporter named `my-exporter`, which is configured +with `config-parameter: value`. For this configuration to succeed, +a [component provider](#component-provider) must +be [registered](#register-component-provider) with `type: SpanExporter`, +and `name: my-exporter`. When [parse](#parse) is called, the implementation will +encounter `my-exporter` and translate the corresponding configuration to an +equivalent generic `properties` representation ( +i.e. `properties: {config-parameter: value}`). When [create](#create) is called, +the implementation will encounter `my-exporter` and +invoke [create plugin](#create-plugin) on the registered component provider +with the configuration `properties` determined during `parse`. + +Given the inherent differences across languages, the details of extension +component mechanisms are likely to vary to a greater degree than is the case +with other APIs defined by OpenTelemetry. This is to be expected and is +acceptable so long as the implementation results in the defined behaviors. + +#### Component Provider + +A component provider is responsible for interpreting configuration and returning +an implementation of a particular type of SDK extension plugin interface. + +Component providers are registered with an SDK implementation of configuration +via [register](#register-component-provider). This MAY be done automatically or +require manual intervention by the user based on what is possible and idiomatic +in the language ecosystem. For example in Java, component providers might be +registered automatically using +the [service provider interface (SPI)](https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html) +mechanism. + +See [create](#create), which details component provider usage in file +configuration interpretation. + +#### Create Plugin + +Interpret configuration to create a instance of a SDK extension plugin +interface. + +**Parameters:** + +* `properties` - The configuration properties. Properties MUST fully represent + the configuration as specified in + the [configuration file](#configuration-file), including the ability to access + scalars, mappings, and sequences (of scalars and other structures). It MUST be + possible to determine if a particular property is present. It SHOULD be + possible to access properties in a type safe manner, based on what is idiomatic + in the language. + +**Returns:** A configured SDK extension plugin interface implementation. + +The plugin interface MAY have properties which are optional or required, and +have specific requirements around type or format. The set of properties a +component provider accepts, along with their requirement level and expected +type, comprise a configuration schema. A component provider SHOULD document its +configuration schema. + +When Create Plugin is invoked, the component provider interprets `properties` +and attempts to extract data according to its configuration schema. If this +fails (e.g. a required property is not present, a type is mismatches, etc.), +Create Plugin SHOULD return an error. + ### Operations SDK implementations of configuration MUST provide the following operations. @@ -157,10 +248,6 @@ with OpAmp Parse and validate a [configuration file](#configuration-file). -Parse MUST perform [environment variable substitution](#environment-variable-substitution). - -Parse MUST interpret null as equivalent to unset. - **Parameters:** * `file`: The [configuration file](#configuration-file) to parse. This MAY be a @@ -173,7 +260,17 @@ Parse MUST interpret null as equivalent to unset. **Returns:** [configuration model](#in-memory-configuration-model) -This SHOULD return an error if: +Parse MUST perform [environment variable substitution](#environment-variable-substitution). + +Parse MUST interpret null as equivalent to unset. + +When encountering a reference to +a [SDK extension component](#sdk-extension-components) which is not built in to +the SDK, Parse MUST resolve corresponding configuration to a +generic `properties` representation as described +in [Create Plugin](#create-plugin). + +Parse SHOULD return an error if: * The `file` doesn't exist or is invalid * The parsed `file` content does not conform to @@ -183,15 +280,6 @@ This SHOULD return an error if: Interpret [configuration model](#in-memory-configuration-model) and return SDK components. -If a field is null or unset and a default value is defined, Create MUST ensure -the SDK component is configured with the default value. If a field is null or -unset and no default value is defined, Create SHOULD return an error. For -example, if configuring -the [span batching processor](../trace/sdk.md#batching-processor) and -the `scheduleDelayMillis` field is null or unset, the component is configured -with the default value of `5000`. However, if the `exporter` field is null or -unset, Create fails fast since there is no default value for `exporter`. - **Parameters:** * `configuration` - The configuration model. @@ -206,12 +294,48 @@ unset, Create fails fast since there is no default value for `exporter`. The multiple responses MAY be returned using a tuple, or some other data structure encapsulating the components. +If a field is null or unset and a default value is defined, Create MUST ensure +the SDK component is configured with the default value. If a field is null or +unset and no default value is defined, Create SHOULD return an error. For +example, if configuring +the [span batching processor](../trace/sdk.md#batching-processor) and +the `scheduleDelayMillis` field is null or unset, the component is configured +with the default value of `5000`. However, if the `exporter` field is null or +unset, Create fails fast since there is no default value for `exporter`. + +When encountering a reference to +a [SDK extension component](#sdk-extension-components) which is not built in to +the SDK, Create MUST resolve the component using [Create Plugin](#create-plugin) +of the [component provider](#component-provider) of the corresponding `type` +and `name` used to [register](#register-component-provider), including the +configuration `properties` as an argument. If no component provider is +registered with the `type` and `name`, Create SHOULD return an error. +If [Create Plugin](#create-plugin) returns an error, Create SHOULD propagate the +error. + This SHOULD return an error if it encounters an error in `configuration` (i.e. fail fast) in accordance with initialization [error handling principles](../error-handling.md#basic-error-handling-principles). TODO: define behavior if some portion of configuration model is not supported +#### Register Component Provider + +The file configuration implementation MUST provide a mechanism to +register [component providers](#component-provider). + +**Parameters:** + +* `component_provider` - The [component provider](#component-provider). +* `type` - The type of plugin interface it provides (e.g. SpanExporter, Sampler, + etc). +* `name` - The name used to identify the type of component. This is used + in [configuration files](#configuration-file) to specify that the + corresponding `component_provider` is to provide the component. + +The `type` and `name` comprise a unique key. Register MUST return an error if it +is called multiple times with the same `type` and `name` combination. + ## References * Configuration