Randomness requirements following W3C Trace Context level 2

CHANGELOG.md

@@ -15,6 +15,8 @@ release.
   ([#4161](https://github.com/open-telemetry/opentelemetry-specification/pull/4161))
 - Clarify that `Export` MUST NOT be called by simple and batching processors concurrently.
   ([#4205](https://github.com/open-telemetry/opentelemetry-specification/pull/4205))
+- Define randomness value requirements for W3C Trace Context Level 2.
+  ([#4162](https://github.com/open-telemetry/opentelemetry-specification/pull/4162))
 
 ### Metrics
 

spec-compliance-matrix.md

@@ -87,6 +87,8 @@ formats is required. Implementing more than one format is optional.
 | [Built-in `SpanProcessor`s implement `ForceFlush` spec](specification/trace/sdk.md#forceflush-1) |          |     | +    |     | +      | +    | +      | +   | +    | +   | +    |       |
 | [Attribute Limits](specification/common/README.md#attribute-limits)                              | X        |     | +    |     | +      | +    | +      | +   |      |     |      |       |
 | Fetch InstrumentationScope from ReadableSpan                                                     |          |     | +    |     | +      |      |        | +   |      |     |      |       |
+| TraceID generator implements W3C Trace Context Level 2 randomness                                | X        |     |      |     |        |      |        |     |      |     |      |       |
+| OpenTelemetry explicit randomness inserted for non-random IdGenerators                           | X        |     |      |     |        |      |        |     |      |     |      |       |
 
 ## Baggage
 

specification/context/api-propagators.md

@@ -31,6 +31,7 @@
   * [Get Global Propagator](#get-global-propagator)
   * [Set Global Propagator](#set-global-propagator)
 - [Propagators Distribution](#propagators-distribution)
+  * [W3C Trace Context Requirements](#w3c-trace-context-requirements)
   * [B3 Requirements](#b3-requirements)
     + [B3 Extract](#b3-extract)
     + [B3 Inject](#b3-inject)
@@ -355,6 +356,17 @@ Additional `Propagator`s implementing vendor-specific protocols such as AWS
 X-Ray trace header protocol MUST NOT be maintained or distributed as part of
 the Core OpenTelemetry repositories.
 
+### W3C Trace Context Requirements
+
+A W3C Trace Context propagator MUST parse and validate the `traceparent` and `tracestate` HTTP headers as specified in [W3C Trace Context Level 2](https://www.w3.org/TR/trace-context-2/).  A W3C Trace Context propagator MUST propagate a valid `traceparent` value using the same header.  A W3C Trace Context propagator MUST propagate a valid `tracestate` unless the value is empty, in which case the `tracestate` header may be omitted.
+
+When injecting and extracting trace context to or from a carrier, the following fields from the `SpanContext` are propagated.
+
+- TraceID (16 bytes)
+- SpanID (8 bytes)
+- TraceFlags (8 bits)
+- TraceState (string, unless empty)
+
 ### B3 Requirements
 
 B3 has both single and multi-header encodings. It also has semantics that do not

specification/trace/api.md

@@ -233,16 +233,17 @@ non-zero byte.
 `SpanId` A valid span identifier is an 8-byte array with at least one non-zero
 byte.
 
-`TraceFlags` contain details about the trace. Unlike TraceState values,
-TraceFlags are present in all traces. The current version of the specification
-only supports a single flag called [sampled](https://www.w3.org/TR/trace-context/#sampled-flag).
-
-`TraceState` carries vendor-specific trace identification data, represented as a list
-of key-value pairs. TraceState allows multiple tracing
-systems to participate in the same trace. It is fully described in the [W3C Trace Context
-specification](https://www.w3.org/TR/trace-context/#tracestate-header). For
-specific OTel values in `TraceState`, see the [TraceState Handling](tracestate-handling.md)
-document.
+`TraceFlags` contain details about the trace.
+Unlike TraceState values, TraceFlags are present in all traces.
+The current version of the specification supports two flags:
+
+- [Sampled](https://www.w3.org/TR/trace-context-2/#sampled-flag)
+- [Random](https://www.w3.org/TR/trace-context-2/#random-trace-id-flag)
+
+`TraceState` carries tracing-system-specific trace identification data, represented as a list of key-value pairs.
+TraceState allows multiple tracing systems to participate in the same trace.
+It is fully described in the [W3C Trace Context specification](https://www.w3.org/TR/trace-context-2/#tracestate-header).
+For specific OpenTelemetry values in `TraceState`, see the [TraceState Handling](tracestate-handling.md) document.
 
 `IsRemote`, a boolean indicating whether the SpanContext was received from somewhere
 else or locally generated, see [IsRemote](#isremote).

specification/trace/sdk.md

@@ -23,6 +23,7 @@ linkTitle: SDK
 - [Sampling](#sampling)
   * [Recording Sampled reaction table](#recording-sampled-reaction-table)
   * [SDK Span creation](#sdk-span-creation)
+    + [Span flags](#span-flags)
   * [Sampler](#sampler)
     + [ShouldSample](#shouldsample)
     + [GetDescription](#getdescription)
@@ -33,6 +34,12 @@ linkTitle: SDK
       - [Requirements for `TraceIdRatioBased` sampler algorithm](#requirements-for-traceidratiobased-sampler-algorithm)
     + [ParentBased](#parentbased)
     + [JaegerRemoteSampler](#jaegerremotesampler)
+  * [Sampling Requirements](#sampling-requirements)
+    + [TraceID randomness](#traceid-randomness)
+    + [Random trace flag](#random-trace-flag)
+    + [Explicit trace randomness](#explicit-trace-randomness)
+    + [Presumption of TraceID randomness](#presumption-of-traceid-randomness)
+    + [IdGenerator randomness](#idgenerator-randomness)
 - [Span Limits](#span-limits)
 - [Id Generators](#id-generators)
 - [Span processor](#span-processor)
@@ -265,7 +272,7 @@ The OpenTelemetry API has two properties responsible for the data collection:
   receive them unless the `Sampled` flag was also set.
 * `Sampled` flag in `TraceFlags` on `SpanContext`. This flag is propagated via
   the `SpanContext` to child Spans. For more details see the [W3C Trace Context
-  specification](https://www.w3.org/TR/trace-context/#sampled-flag). This flag indicates that the `Span` has been
+  specification][W3CCONTEXTSAMPLEDFLAG]. This flag indicates that the `Span` has been
   `sampled` and will be exported. [Span Exporters](#span-exporter) MUST
   receive those spans which have `Sampled` flag set to true and they SHOULD NOT receive the ones
   that do not.
@@ -300,10 +307,9 @@ When asked to create a Span, the SDK MUST act as if doing the following in order
 1. If there is a valid parent trace ID, use it. Otherwise generate a new trace ID
    (note: this must be done before calling `ShouldSample`, because it expects
    a valid trace ID as input).
-2. Query the `Sampler`'s [`ShouldSample`](#shouldsample) method
-   (Note that the [built-in `ParentBasedSampler`](#parentbased) can be used to
-   use the sampling decision of the parent,
-   translating a set SampledFlag to RECORD and an unset one to DROP).
+2. Query the `Sampler`'s [`ShouldSample`](#shouldsample) method.
+   The [built-in `ParentBasedSampler`](#parentbased) can be used to
+   take the sampling decision of the parent context using the `Sampled` flag.
 3. Generate a new span ID for the `Span`, independently of the sampling decision.
    This is done so other components (such as logs or exception handling) can rely on
    a unique span ID, even if the `Span` is a non-recording instance.
@@ -316,6 +322,14 @@ When asked to create a Span, the SDK MUST act as if doing the following in order
    `Span` is created without an SDK installed or as described in
    [wrapping a SpanContext in a Span](api.md#wrapping-a-spancontext-in-a-span).
 
+#### Span flags
+
+The OTLP representation for Span and Span Link includes a 32-bit field declared as Span Flags.
+
+Bits 0-7 of the Span Flags field are reserved for the 8 bits of Trace Context flags,
+specified in the [W3C Trace Context Level 2][W3CCONTEXTMAIN] Candidate Recommendation.
+[See the list of recognized flags](./api.md#spancontext).
+
 ### Sampler
 
 `Sampler` interface allows users to create custom samplers which will return a
@@ -466,6 +480,54 @@ The following configuration properties should be available when creating the sam
 [jaeger-remote-sampling-api]: https://www.jaegertracing.io/docs/1.41/apis/#remote-sampling-configuration-stable
 [jaeger-adaptive-sampling]: https://www.jaegertracing.io/docs/1.41/sampling/#adaptive-sampling
 
+### Sampling Requirements
+
+The [W3C Trace Context Level 2][W3CCONTEXTMAIN] Candidate Recommendation includes [a Random trace flag][W3CCONTEXTRANDOMFLAG] for indicating that the TraceID contains 56 random bits, specified for statistical purposes.
+This flag indicates that [the least-significant ("rightmost") 7 bytes or 56 bits of the TraceID are random][W3CCONTEXTTRACEID].
+
+Note the Random flag does not propagate through [Trace Context Level 1][W3CCONTEXTLEVEL1] implementations, which do not recognize the flag.
+When this flag is 1, it is considered meaningful.  When this flag is 0, it may be due to a non-random TraceID or because a Trace Context Level 1 propagator was used.
+To enable sampling in this and other situations where TraceIDs lack sufficient randomness,
+OpenTelemetry defines an optional [explicit randomness value][OTELRVALUE] encoded in the [W3C TraceState field][W3CCONTEXTTRACESTATE].
+
+This specification recommends the use of either TraceID randomness or explicit trace randomness,
+which ensures that samplers always have sufficient randomness when using W3C Trace Context propagation.
+
+[W3CCONTEXTMAIN]: https://www.w3.org/TR/trace-context-2
+[W3CCONTEXTLEVEL1]: https://www.w3.org/TR/trace-context
+[W3CCONTEXTTRACEID]: https://www.w3.org/TR/trace-context-2/#randomness-of-trace-id
+[W3CCONTEXTTRACESTATE]: https://www.w3.org/TR/trace-context-2/#tracestate-header
+[W3CCONTEXTSAMPLEDFLAG]: https://www.w3.org/TR/trace-context-2/#sampled-flag
+[W3CCONTEXTRANDOMFLAG]: https://www.w3.org/TR/trace-context-2/#random-trace-id-flag
+[OTELRVALUE]: ./tracestate-handling.md#explicit-randomness-value-rv
+
+#### TraceID randomness
+
+For root span contexts, the SDK SHOULD implement the TraceID randomness requirements of the [W3C Trace Context Level 2][W3CCONTEXTTRACEID] Candidate Recommendation when generating TraceID values.
+
+#### Random trace flag
+
+For root span contexts, the SDK SHOULD set the `Random` flag in the trace flags when it generates TraceIDs that meet the [W3C Trace Context Level 2 randomness requirements][W3CCONTEXTTRACEID].
+
+#### Explicit trace randomness
+
+For root span contexts, the when the SDK generates a TraceID that does not meet the [W3C Trace Context Level 2 randomness requirements][W3CCONTEXTTRACEID], and when the initial `TraceState` does not already define the [`rv` sub-key of the OpenTelemetry TraceState][OTELRVALUE], the SDK SHOULD insert an explicit trace randomness value into the OpenTelemetry TraceState value containing 56 random bits.
+
+For example, here's a W3C Trace Context with non-random identifiers and an explicit randomness value:
+
+```
+traceparent: 00-ffffffffffffffffffffffffffffffff-ffffffffffffffff-00
+tracestate: ot=rv:7479cfb506891d
+```
+
+#### Presumption of TraceID randomness
+
+For all span contexts, OpenTelemetry samplers SHOULD presume that TraceIDs meet the W3C Trace Context Level 2 randomness requirements, unless an explicit randomness value is present in the [`rv` sub-key of the OpenTelemetry TraceState][OTELRVALUE].
+
+#### IdGenerator randomness
+
+If the SDK uses an `IdGenerator` extension point, the SDK SHOULD allow the extension to determine whether the Random flag is set when new IDs are generated.  When an `IdGenerator` instance does not meet the randomness requirements, the SDK SHOULD insert explicit randomness instead, otherwise samplers will incorrectly presume TraceID randomness.
+
 ## Span Limits
 
 Span attributes MUST adhere to the [common rules of attribute limits](../common/README.md#attribute-limits).
@@ -530,6 +592,13 @@ public interface IdGenerator {
 }
 ```
 
+Custom implementations of the `IdGenerator` SHOULD support setting the
+W3C Trace Context `random` flag when all generated TraceID values meet
+the [W3C Trace Context Level 2 randomness
+requirements][W3CCONTEXTTRACEID].  This is presumed to be a static
+property of the `IdGenerator` implementation which can be inferred
+using language features, for example by extending a marker interface.
+
 Additional `IdGenerator` implementing vendor-specific protocols such as AWS
 X-Ray trace id generator MUST NOT be maintained or distributed as part of the
 Core OpenTelemetry repositories.

specification/trace/tracestate-handling.md

@@ -83,3 +83,27 @@ if ok {
   // traceState was not updated.
 }
 ```
+
+## Pre-defined OpenTelemetry sub-keys
+
+The following values have been defined by OpenTelemetry.
+
+### Explicit randomness value `rv`
+
+The OpenTelemetry TraceState `rv` sub-key defines an alternative source of randomness to the use of TraceID randomness when used by samplers decisions.
+Values of `rv` MUST be exactly 14 lower-case hexadecimal digits:
+
+```
+hexdigit = DIGIT ; a-f
+```
+
+The explicit randomness value is meant to be used instead of extracting randomness from TraceIDs, therefore it contains the same number of bits as a W3C Trace Context Level 2 recommends for TraceIDs.
+
+Explicit randomness values are meant to propagate through [span contexts](../context/README.md) unmodified.
+Explicit randomness values SHOULD NOT be erased from the OpenTelemetry TraceState or modified once associated with a new TraceID, so that sampling decisions made using the explicit randomness value are consistent across signals.
+
+For example, here is a W3C TraceState value including an OpenTelemetry explicit randomness value:
+
+```
+tracestate: ot=rv:6e6d1a75832a2f
+```