Sensitive Data Redaction

[svrnm]

This OTEP proposes changes that will enable sensitive data redaction. This is a follow up to open-telemetry/semantic-conventions#971 and open-telemetry/semantic-conventions#961 and the SemConv spec meeting (2024-04-29)

[lmolkova]

It's probably more of a spec concern and we'll have more time to design and polish API, but I'd like to entertain the idea of

• Some SensitivityConfig instrumentation can request from Tracer
• SensitivityConfig can be configured by the SDK/distro/etc and can have different implementations for different regulations
• SensitivityConfig has convenience methods to redact common things like URLs. E.g. String SensitivityConfig.redactQueryParams(Uri uri)
• It may recognize common attributes which are known to be problematic (e.g. client.address on the server spans is potentially PII) and could be applied implicitly without instrumentation code

E.g.

Span clientSpan = tracer.startSpan("GET", CLIENT);
String redactedUri = tracer.getSensitivityConfig().redactQueryParams(uri);
clientSpan.setAttribute("url.full", redactedUri);

or

Span serverSpan = tracer.startSpan("GET /foo", SERVER);

// the `SensitivityConfig` associated with the tracer knows that `client.address` 
// may be sensitive and would apply whatever it's configured to do 
// allow, drop, anonymize, etc)
serverSpan.setAttribute("client.address", request.getClientIp()); 

[svrnm]

Is this a suggestion as an alternative to what I suggested or an addition? It looks like an addition (and I think it's great!), but I wanted to verify first before discussing.

[lmolkova]

I think it's both :)

I'd like to be able to add redaction/sanitization without requiring instrumentation to change their code or worry about it.

I.e. the alternative proposal is to build a reasonable story within existing API surface first.
But the proposal you made in the OTEL will still be necessary - we'll need to redact/sanitize custom attributes or to optimize and fine-tune instrumentation code for those who want it.

[lmolkova]

Throwing in one more suggestion - we can auto-generate semantic convention code along with the redaction.

Today we generate constants like:

class UrlAttributes {
   public static final String URL_FULL = "url.full";
}

we could instead generate a method

class UrlAttributes {
   public static final String URL_FULL = "url.full";
   public static void recordUrlFull(Span span, Uri uri) {
       String redacted = getSensitivityConfig().redactQueryParams(uri);
       span.setAttribute(URL_FULL, redacted);
   }
}

We still need to have a central config and a way to pass it around and make it accessible on the API level, but we can hide a lot of boilerplate and be able to partially automate things across languages.

/cc @lquerel

[svrnm]

I'd like to be able to add redaction/sanitization without requiring instrumentation to change their code or worry about it.

Yes, that's an important feature! With me suggestions for how to write that down in the semantic conventions this should be feasible. Let me add some wording to make this clear.

Looking at the solutions I would prefer the first one over the updated code generation for the semantic conventions, i.e. setAttribute should do the redaction internally, because having a method per attribute seems excessive to me (and people will forget to use it, vs not using the constants comes with less penalties) and people can configure their own redaction through configuration (let's say the use an application they do not have code access and this application emits "enduser.id" without redaction, they can update their sensitivity config and make sure that it is redacted)

[yurishkuro]

Library authors do not have enough context to make reduction decisions. The best they can do is tag attributes with some semantic annotation saying "this is sensitive" (e.g. in my company we have a taxonomy of "purpose policy" annotations for data elements). If I deploy this library as a user, I should be able to decide if I actually want "sensitive" to be redacted or not - I may be debugging locally and need to see all data, or maybe in production I have different pathways where the data flows, some require reduction and others require raw data.

And to add these annotations it's better to go with the schemas approach, without changing the API.

[reyang]

And to add these annotations it's better to go with the schemas approach, without changing the API.

+1

[svrnm]

I think we are talking about the same thing, @yurishkuro: Indeed a library author can not make reduction decision, this is the responsibility of the application operator, but the library author needs capabilities to express that some data may be sensitive and provide redaction options similar to what we want to do in the semantic conventions. Here is an example: A library author writes a method that calls a payment provider HTTP endpoint, which has a unique query parameter to carry the token (let's say acmeToken). That library author now wants to say "when I call this endpoint, and a HTTP span is generated, make sure that url.query is redacted in a way that acmeToken is treated like other potentially sensitive attributes (see #971). Ideally they can do something similar to what is proposed for the semantic conventions, e.g. a way to write down what to do when DEFAULT, STRICT or STRICTER (or any other profile is selected). Then, the end user of that library and of opentelemetry can make the decision of redaction.

[yurishkuro]

@svrnm yes, this is a pretty complex use case, but I would still err on the side of a centralized solution over federating these responsibilities to the call sites. It's conceivable that the semantic convention / schema for url.query could be extended with conditional clauses like "if I am in this instrumentation scope or the URL contains this domain name, these are the additional rules / annotations for the query elements". And I think this approach is going to be necessary in many cases anyway, because the specific HTTP instrumentation may not have any clue about this business use case, e.g. it could be a generic HTTP client interceptor that generates client spans and records URL, and only user code would have the theoretical capability of understanding the sensitivity of the query params.

[svrnm]

It's conceivable that the semantic convention / schema for url.query could be extended with conditional clauses like "if I am in this instrumentation scope or the URL contains this domain name, these are the additional rules / annotations for the query elements".

That's what is part of this proposal, I need to optimize the wording for that, but eventually the configuration for redaction needs to be that fine-grained.

but I would still err on the side of a centralized solution over federating these responsibilities to the call sites.

Yes, centralized is the core of it, but library/application authors still need a way to provide a hint that a redaction needs to be applied. Rethinking my example, we actually have the following situation:

A library author is likely using a HTTP client library as dependency themselves to interact with a 3rd partyAPI, so in some pseudo code they might have something like the following:

function authenticateWithMyService($token) {
  // they want to let OpenTelemetry know that $token is sensitive, and when
  // it is injected into the `url.query` attribute certain redaction may be required.
  use3rdPartyHttpClient("https://myservice//?myAcmeToken=$token")
}

How would they implement that? They need some ways to annotate that. Also those annotations need to be composable somehow.

[reyang]

I feel metrics should take an exception here due to the nature of cardinality and performance. Do we know if there were credential/privacy leaks in metrics systems?

[svrnm]

I would need to research on that, but I think performance will be an issue for all signals, I added that to the trade-off section already. So I wonder if we need to do some experiments eventually.

[reyang]

Metrics API is designed to be much faster than other APIs (e.g. it's not uncommon to have metrics APIs that are 20x faster than tracing APIs), so the perf impact would be bigger if measured by the % drop of calls/second.

Cardinality could also be an issue, for example, if the intention is to get the "unique count of users via email address", the result will be 1 if email address got redacted to "[email protected]".

[svrnm]

Thanks for clarification, that's a signification performance difference indeed.

On cardinality I think there are 2 options to address that:

• if the redaction happens late (before exporting data) it is still possible to create that metric properly
• there are options to redact emails and still keep their uniqueness, e.g. by hashing the emails (or parts of it), note that hashing is a suboptimal solution since the hashed words are easy to guess & break (use a dictionary with common first and lastnames, etc.)

[svrnm]

I still need to add this into the document.

[reyang]

Would this apply to the SDK globally or can be applied on specific pipelines/processors/exporters? e.g. if the goal is to send audit logs (which have EUII such as email address and IP address) to destination A without performing any redaction, and send normal logs to destination B with redaction.

[svrnm]

The suggestion here was globally, but I see the point you are making for having it split out by exporter.

[svrnm]

I still need to add this into the document.

[reyang]

I think yes.

[dmathieu]

It feels like there should still be some stability definition.
I wouldn't want to see something that was redacted suddenly not be.

[svrnm]

That's a good point, maybe "adding redactions" may be outside, but "removing redactions" may require additional guard rails. It is probably unlikely that redactions will be removed, the only case I can think of is that something is deprecated for a very long time

[svrnm]

I will leave the question open, I can add some words on adding/removing redaction

[yurishkuro]

I would suggest to start with the most generic solution that can address all leaks, even if it's less efficient. Anything that relies on instrumentation callsite changes is fundamentally not capable of full coverage - someone will always forget some sensitivity parameter or a call to sanitizer. Whereas having a config-driven sanitizer will be able to see all data and apply rules reliably.

[dmathieu]

Is the not on purpose?
It seems like at least ALWAYS could be useful, for example as a temporary mitigation when a leakage is discovered, and before it can be properly fixed.

Suggested change

	This call will make sure that redactions applied follow the `STRICT` profile. If not set the `DEFAULT` will be used. Additionally there are 2 levels that can not be used in sensitivity details:
	This call will make sure that redactions applied follow the `STRICT` profile. If not set the `DEFAULT` will be used. Additionally there are 2 levels that can be used in sensitivity details:

[svrnm]

I think what I wanted to express here is that ALWAYS and NEVER can not be reconfigured through <SENSITIVITY_DETAILS> as outlined above in the document. Because if allowed it would break expectations

[joaopgrassi]

Yeah that statement was a bit confusing to me as well, maybe it can be rephrased to express what you said above?

[svrnm]

I still need to add this into the document.

[danielgblanco]

Are we considering Baggage as a signal here? And would we consider in-band data (context propagation) as well as out-of-band data?

I think open-telemetry/opentelemetry-specification#1633 may benefit from any spec changes related to being able to specify sensitivity of Baggage attributes to be propagated across system boundaries. I don't think this OTEP would solve the context propagation boundary issue (Propagators would still need to define what they consider a system boundary) but I wonder, providing that Baggage is considered here, if the SensitivityConfig should contain something related to defining if a Baggage attribute is safe to propagate or not. I'm not convinced either way myself just now, but thought I'd raise it as we're saying "Every API".

[svrnm]

That's a very interesting thought. Since baggage may carry sensitive data it's worth looking into it (we might later to move into "future considerations"). @lmolkova example with client.address in #255 (comment) already called out that it might be relevant to add contextual information to the redaction, the baggage propagation is another example of that.

[svrnm]

I still need to add this into the document.

[reyang]

💯 Nice summary @svrnm!

I suggest that we turn these into a list of principles and call it out clearly in this OTEP.

Here are some examples I could think of:

1. OpenTelemetry MUST allow the end-user to meet with their security/privacy/compliance requirements regarding the data being collected.
2. OpenTelemetry MUST not provide redaction offerings that lead to bigger security issues such as https://en.wikipedia.org/wiki/ReDoS (e.g. the redaction logic is poorly implemented, so a hacker could forge certain input to DDoS the redaction engine itself).
3. OpenTelemetry SHOULD allow the telemetry data to apply different redaction logic per telemetry pipeline/exporter in a single process.

[jpkrohling]

About 2: while we cannot guarantee all SDKs will provide a bug-free implementation of the redaction logic, we could offer a logic to be followed by all SDKs.

[svrnm]

Thanks @reyang, that's a good proposal, will incorporate that (+ the suggestion from @jpkrohling which I agree to, I'll try to find some wording to express that)

[svrnm]

I added those at the top of the document. I used "MUST avoid" vs "MUST not provide" since as @jpkrohling said this is impossible to guarantee, but we can put mechanisms in place to avoid them

[jpkrohling]

I believe this goes in the right direction. You did list a "NEVER" option, effectively disabling redaction, but I think it would be more prominently mentioned that and end-user should be able to easily enable/disable this globally.

[jpkrohling]

About 2: while we cannot guarantee all SDKs will provide a bug-free implementation of the redaction logic, we could offer a logic to be followed by all SDKs.

[jpkrohling]

I think this is already implicit, but to me, it's important to let users disable the redaction (ie, having a sensitivity details config that is permissive). This would allow them to not pay a performance penalty at runtime, and apply the redaction logic at the collector if this model satisfies their requirements.

[jpkrohling]

s/IIN/Issuer Identification Number (IIN)/

[svrnm]

applied

[jpkrohling]

I think we'll eventually need to have something similar to this as well: semconv attributes would need a marker, about the common sensitivity of the attribute's value.

[svrnm]

I need to review those 2 PRs in detail and incorporate whatever we want to include here as well. I added a little bit of wording on classification (see my credit card example)

[pyohannes]

I like the direction this proposal is going toward, however, I'd like to challenge the notion that redaction is treated as an instrumentation issue, as opposed to a configuration issue. I wonder if it would be possible to treat redaction as a configuration issue, like we do sampling.

Reading through this document and arriving at this part, I was thinking about a solution that relies solely on SDK components (without any API changes needed), by providing SDK redactors and configuring them via "redaction profiles" (which consists of mappings from attribute names to redaction rules, similar to the example below). SDKs could provide some default redaction profiles (for example for redacting strict, stricter, or never), which are based on definitions in semantic conventions.

The main challenges of this approach would be considering multiple versions of semantic conventions, furthermore it wouldn't be that straightforward for authors of instrumentation libraries to add redaction rules.

However, it would give the benefit to have redaction rules defined at a central place. Given that the examples above (different regular expressions for different redaction levels), it might be error prone to implement those consistently across all usages of an attribute in different instrumentation libraries in different languages.

[svrnm]

I wonder if it would be possible to treat redaction as a configuration issue, like we do sampling.

For 90% of the cases, I would say the answer to this is yes, but especially library authors need ways to express additional redaction requirements very locally, e.g. if they call a 3rd party service via a HTTP GET and they know that in this one specific case they need to apply additional redaction. Without a functionality for that they can either over-configure (e.g. apply it globally but potentially have other calls redacted without the need for that) or do their own logic on top of what we provide them. Both things are OK and we might decide against having the API for that (in a first version, or forever), but then we need to indiciate that somehow in the specification.

[svrnm]

See the updated version, I moved towards leading with the configuration centric approach and mentioned API capabilities towards the end

[lmolkova]

a reasonable default (or strict mode?) for PII data may be anonymization. If we defined an algorithm, we'd be able to do it consistently across implementations and preserve some level of observability.

[svrnm]

What algorithm would you have in mind for anonymization? Often hashing is used for that, but depending on the underlying data it's pseudo-anonymization (there is a paper on that from the EU, I need to dig up), e.g. email addresses can be guessed easily by tuning your dictionaries.

[lmolkova]

good question, I don't have an algorithm in mind. My intention was to have some form of AMONYMIZE mode listed, but details can be out of scope of this otep.

If you believe there is no common and reliable algorithm, then I'm taking my comment back.

[svrnm]

https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2014/wp216_en.pdf is the document I was referring to, they give a really good overview of different approaches to Anonymization (e.g. stripping octets from IPs would fall into "Generalization" or "Aggregation") and also discuss Pseudonymization (e.g. hashing).

I don't know if this is doable, my point is that if we provide ANONYMIZE as an algorithm it should do exactly that (choosing from the approaches listed in that document), and if we want to use hashing etc an appropriate name should be used (PSEUDONYMIZE?)

It's worth to explore this, but before doing that we need to have the tooling for that in place.

[lquerel]

The anonymization method often depends on the underlying data type. The byte-stripping technique you mentioned is applicable to IPs but not necessarily to all data types. This raises the question of what we can support/propose based on the metadata available or not for an attribute (e.g., semconv or schema), for example.

[svrnm]

I am thinking about that as well right now, I try to come up with a suggestion in the updated proposal.

[svrnm]

I reworked the document to address (most) of the feedback, please take another look, thanks!

[lmolkova]

Suggested change

	span.setAttribute("url.query", OpenTelemetry.redact(url.toString(), <SENSITIVITY_DETAILS>);
	span.setAttribute("url.query", OpenTelemetry.redact(url.toString(), <SENSITIVITY_DETAILS>));

[lmolkova]

In this example the caller is the one doing redaction, but from the OTel API side it doesn't know it already happened and the SDK would do the redaction again.

In the best case it's redundant, but it's also possible that the different redaction layers are inconsistent.

I believe we should allow instrumentations to do the redaction because they may have more context on the data, but then we should avoid doing it again.

---

The suggestion here is to

1. keep span.setAttribute(string name, any value) for the cases when SDK does redaction.
2. allow to express that data is already safe and sanitized with a different API. E.g. span.setAttribute(string name, any value, <SAFE_DO_NOT_CHANGE>)

[svrnm]

I would prefer to add parameters to setAttribute, I just wanted to explore how to do it if we want to avoid that.

In my previous proposal I had something like

span.setAttribute(name, value)
span.redactAttribute(name, <SENSITIVITY_DETAILS>)

I removed and replaced that because I was not sure anymore why I did it, your comment reminded me of that! My idea was that the end user can give redaction hints, but I also like your suggestion of signaling that the application/library took care of redaction themselves (which is related to #255 (comment) and #255 (comment))

[jmacd]

I wonder, if we were asked to design an instrumentation API from the start with support for redaction functionality from the start, whether we woiuld try to create a first-class object out of the attribute key.

Instead of writing "url.query" in the code, shouldn't we have attribute Key declarations with support for optional details? Then in the code you would refer to conventions::url::QueryKey or similar, instead of being expected to supply and process the sensitivity details at each call site.

As you said above

OpenTelemetry MUST avoid redaction offerings that lead to bigger security issues

I worry that the API details here are critically important. If the SDK has to do complex processing of optional argument structures at the call site, where performance is critical, complex code will be the result. I would prefer to see us embrace code-generation using the semantic conventions database and https://github.com/open-telemetry/weaver, so that attribute sensitivity and redaction requirements are compiled in, using dedicated APIs with clear, simple processing rules.

[svrnm]

Based on the feedback so far major consensus is that the redaction mainly should happen in the background, so even if you provide url.query as a string the SDK knows what to do with that (e.g. by leveraging code generated by the weaver).

The big discussion point is what to do if someone wants to add additional redaction requirements. We can not expect that what we provide via the semconv + code-generation is covering all the requirements that users have.

[lquerel]

The generator or the generated code could be aware of an external configuration/schema containing the definition of sensitive data. For the instrumented application that will be totally transparent.

[lmolkova]

The stability guarantees cover the part of telemetry that goes over the wire (i.e. the telemetry consumer side), which, AFAIK is not the case for the sensitivity details.

The part sensitivity details affect is the attribute value, but we don't (at least yet) provide any guarantees there.

So I think we can remove this open question.

[svrnm]

Agreed, only guarantee we should consider to give is that redactions are never "downgraded" (see #255 (comment)), such that endusers can trust the redaction to work as expected when they update opentelemetry.

[jmacd]

I wonder, if we were asked to design an instrumentation API from the start with support for redaction functionality from the start, whether we woiuld try to create a first-class object out of the attribute key.

Instead of writing "url.query" in the code, shouldn't we have attribute Key declarations with support for optional details? Then in the code you would refer to conventions::url::QueryKey or similar, instead of being expected to supply and process the sensitivity details at each call site.

As you said above

OpenTelemetry MUST avoid redaction offerings that lead to bigger security issues

I worry that the API details here are critically important. If the SDK has to do complex processing of optional argument structures at the call site, where performance is critical, complex code will be the result. I would prefer to see us embrace code-generation using the semantic conventions database and https://github.com/open-telemetry/weaver, so that attribute sensitivity and redaction requirements are compiled in, using dedicated APIs with clear, simple processing rules.

[jmacd]

For the trace SDK-- yes, this is functionality that is similar to samplers. Sampling SIG has been working on requirements for a V2 sampler API, and I would include redaction functionality there. See open-telemetry/opentelemetry-specification#4012.

For the metrics SDK -- there will be questions about whether redaction happens before or after aggregation. The present SDK specification doesn't support a processor that happens before aggregation; the filtering that is possible through metrics Views doesn't exactly support redaction, since modifications to the attribute set need to be considered before aggregation (or else re-aggregation has to be performed following redaction). There has been a requested feature, which goes by the name MeasurementProcessor in past discussions, that would be the proper place to (a) redact / drop attributes, (b) extend attributes from context.

In the logs SDK, this is form of processing is the subject of an open discussion, see open-telemetry/opentelemetry-specification#4010.

[svrnm]

Thanks, I think I wrote that "similar to a sampler" without giving it a lot of thought beyond "this needs to be a component that can be evolved independently (like samplers) and is also independent of the signal.

I'll review your feedback and try to incorporate it into the document

[jsuereth]

Overall want to give a HUGE THANK YOU for making a dent in this problem and throwing a proposal out there.

I have a lot of comments. I wish I had more time to give viable solutions to my comments, but I do not. (I also lack the time to be brief, so this is long winded).

Overall -

This has the bones of what I think we want to build.

• A configurable set of redaction rules
• A mechanism to annotate attributes with meta-infromation about sensitivity/security
• A simple user knob of "ON/OFF/my-custom-thing" for important use cases.

The details need a bit more fleshing out in particular:

• The YAML model (and OTEL API) for annotating attributes with baseline sensitivity/redaction controls. You may need MULTIPLE redaction methods for each level of redaction you support, e.g.
• Stricter lines / definitions (less possible bikeshed) on Sensitivity levels.
• More details on the interaction with the SDK, in particular is it the same interface for all three signals or different?
• More granular model (Span name redaction, e.g.)

Overall a great start. Let me know if there's anything I can do to help push areas of this forward or flesh out any of your ideas.

[jsuereth]

Do you plan to have these be only global on attributes, or fine grained -> e.g. this specific Span an this specific Attribute?

[svrnm]

This is just an example, I think we need more than attributes, see below I even suggested to use OTTL (might be an overkill thinking about the performance impact)

[jsuereth]

I'm not sure I see a lot of difference between DEFAULT/STRICT/STRICTER. At a minimum, I see too many "vague" notions where we could wind up bikeshedding what belong in one vs. another.

Can we be more principled in terms of defining:

1. What each profile means.
2. A set of annotations / meta-semantics for attributes?

E.g.

NONE - no redaction
NO_USER - redact any information that is input by a user of the system or could be used to identify a user of the system (email, name, IP, etc.)
NO_LOCATION - redact any location information (geo lat/lon, address, etc.)

Basically - I don't think the current set is going to be something we can specify. This needs more rigor and strict lines.

[svrnm]

Wow, I like that, this is a great idea!

[jsuereth]

Do we need to only specify "redaction" that can be implemented in all languages? Will this be a new specification of allowable redactions?

[svrnm]

maybe? I think at some point we need to do some prototyping to figure out what redaction is allowed to do and what not. Performance is the main reason but "this needs to be implemented in "any" language" is another complex requirement.

[jsuereth]

This looks like a regex rule. The reality here is there's a lot of nuanced-differences between regex across languages and libraries. You'll want to pick a specific implementation to reference or avoid exposing it here.

[svrnm]

Yes, regular expressions might not be the right tool to express the redaction requirements. Having differences in languages is the one thing, but also the ReDOS example comes to mind.

[jsuereth]

OTTL needs better formal specificity to be language-neutral before this is viable.

I'd actually like to see that formal specification happen, but I also don't think its current state is a good candidate for this, as it relegates all actual runtime behavior to functions (Today it even relgates the data model via implementation-specific meaning for .).

TL;DR; This is akin to suggesting we have a set of hardcoded redaction functions.

[jsuereth]

Can you call out how this works across signals?

Right now we have no direct ties between Span/Metric/Trace via the SDK. They actually only communicate with each other via Context API.

[svrnm]

The honest answer is "I don't know", @jmacd made some suggestions on this, I need to take a look into this

[jsuereth]

Seeing this no, would love to go back to your definitions above and have sensitivity/redaction be:

ALL_KNOWN, KNOWN_SECURITY, KNOWN_PRIVACY, NONE

I pick these words because we can offer NO GUARANTEES on redaction given our open API.

[jsuereth]

I'm not sure we need to be defining the level of sensitivity for users. For example, a system able to be Privacy compliant could likely handle wipeout rules and protection for all of these. OpenTelemetry should not enforce level of sensitivity, but allow our users to interact with their own definitions using our system.

I'd cut this section or call it out as something for our users to define, not us.

[svrnm]

This is already a kind of "appendix" section, I can make it clearer from the document structure.

[jsuereth]

This should be the first thing stated in this section. We provide tooling to allow users to be compliant, but they are ultimately responsible.

[jsuereth]

This has security implications. E.g. in Java we you wouldn't want sensitive passwords interned in Strings because they can be exfiltrated.

I like what you're suggesting, but I'd go the other way. You could memoize the redaction function when generating the attribute value in some fashion, or use codegen/weaver to optimise these use cases (if we get the yaml model sorted).

[svrnm]

Overall want to give a HUGE THANK YOU for making a dent in this problem and throwing a proposal out there.

Happy to get this conversation started, a big thank you from me to everyone who provided their feedback so far.

I have a lot of comments. I wish I had more time to give viable solutions to my comments, but I do not. (I also lack the time to be brief, so this is long winded).

I appreciate your feedback, a lot of things I can take and try to provide a solution for.

This has the bones of what I think we want to build.

* A configurable set of redaction rules

* A mechanism to annotate attributes with meta-infromation about sensitivity/security

* A simple user knob of "ON/OFF/my-custom-thing" for important use cases.

💯

The details need a bit more fleshing out in particular:

* The YAML model (and OTEL API) for annotating attributes with baseline sensitivity/redaction controls.  You may need MULTIPLE redaction methods for each level of redaction you support, e.g.

* Stricter lines / definitions (less possible bikeshed) on Sensitivity levels.

* More details on the interaction with the SDK, in particular is it the same interface for all three signals or different?

* More granular model (Span name redaction, e.g.)

I'll take another look into all feedback and try to evolve the document with it.

Overall a great start. Let me know if there's anything I can do to help push areas of this forward or flesh out any of your ideas.

Thanks!

[lquerel]

Even though it may seem obvious to some, I find it interesting to clarify the relationship and the functioning mode between these profiles and the configuration files presented in the following section.

[svrnm]

I am reworking that

[lquerel]

Personally, I would expand this section dedicated to semconvs by including the concept of a telemetry schema, which could eventually override what is defined in a semconv but specifically for a service or application and for a given deployment. In a way, the previously presented configuration file could be reorganized or seen as a subset of this telemetry schema for a specific context.

[lquerel]

The anonymization method often depends on the underlying data type. The byte-stripping technique you mentioned is applicable to IPs but not necessarily to all data types. This raises the question of what we can support/propose based on the metadata available or not for an attribute (e.g., semconv or schema), for example.

[lquerel]

Maybe I am misinterpreting the proposal, but at the semcons level, I don't think we should have something defining the action to take on an attribute or a body. Instead, we should have a property characterizing the potential sensitivity of the data carried by this attribute. In my view, the decision on what to do with this type of sensitive data (e.g., partial or complete combination of name, data type, sensitivity level) should be determined by the profile or the local configuration/schema.

[svrnm]

You are right, I plan to change this. I consider to suggest that we keep some kind of "suggested default redaction" if that is feasible, but beyond that it's something that needs to be decided locally.

[lquerel]

The decision about what is sensitive and what to do with sensitive data in terms of privacy is likely to be made by people different from those who implement the application or its instrumentation. Moreover, this decision can evolve over time. Therefore, I think it is more interesting to maintain this separation and have such decisions at the schema, profile, or configuration file level, so there is no need to rebuild the application itself. This approach also ensures that there are no omissions or misuses.

[svrnm]

Developers need to be able to provide an annotation(!) that data is "potentially sensitive". I try to express this better, that it's not their job to make decisions on that but that they need to be part of highlighting that.

[lquerel]

The generator or the generated code could be aware of an external configuration/schema containing the definition of sensitive data. For the instrumented application that will be totally transparent.

[lquerel]

Suggested change

	one that does not exist), it will default to apply `DROP` to avoid leakage of
	one that does not exist), it will default to apply `DROP` to avoid leakage of data.

[lquerel]

The identification of sensitive data could also be applied at a semantic level. For example, as a company, I might want to apply a redaction rule to ALL URLs, another rule to ALL IP addresses, and so on. One possible approach would be to introduce a new type of group in the semconv that would allow the definition of semantic types, which could be attached to existing attributes in addition to the "physical" data type that we already have. The redaction rules would then apply to the semantic types.

[svrnm]

Yes 🙌 , this was one of the things I wanted to include in my updated proposal, based on your and @jsuereth's suggestion

[lquerel]

Having support for lineage associated with this type of metadata could help identify where a redaction operation originates from. We have started to support the concept of lineage in Weaver, and I think it could be used in this specific case.

[svrnm]

Yes 🙌 , I mentioned lineage somewhere in the doc already because it seems to be highly related here.

[lquerel]

Another complementary option could be to support this type of sensitive data management at the code generation level with a tool like Weaver.

[svrnm]

my understanding from the discussions so far is that if we integrate the sensitive data details in the semconv a tool like weaver could be part of the solution, not an alternative?

[svrnm]

Thank you all for the great feedback. I was not sure what a good practice is, but I reset the PR to draft because I am in the process of incorporating all the feedback and coming back with a significant change.

[svrnm]

open-telemetry/semantic-conventions#971 (comment)

I tried to rework it but there are so many moving pieces that I couldn't get a good handle of it, while also running into some bandwidth issues on my end, since other things required my attention. I still want to follow up on this, but it will take much more time than anticipated.

[pellared]

Related to open-telemetry/semantic-conventions#128