Use opentelemetry.io* namespace for extracing resource and attribute labels from Kubernetes manifests

[austinlparker]

Currently, the Operator infers certain OpenTelemetry resource values (like service.name) by looking at various labels on the manifest (https://github.com/open-telemetry/opentelemetry-operator/blob/main/pkg/instrumentation/sdk.go#L352-L371) such as Deployment Name, Stateful Set Name, etc.

I would like for OpenTelemetry to reserve the *.opentelemetry.io/* namespace (as in resource.opentelemetry.io/service.version or attribute.opentelemetry.io/my.cool.attributte), and have the operator (and various resource detectors) parse this namespace in order to set the associated values.

[jaronoff97]

Yeah i think we could use these and give them priority over what's there currently. Would also make it easier to inject resource attributes generally. Would we allow these to use environment variables? I'm not sure how that would work

[Horiodino]

hi @austinlparker @jaronoff97 i would love to work on this issue , i have used the API and manifests so many times.
so it wont be trouble for me .
can i work on it ?

[jaronoff97]

@Horiodino that would be great, thank you!

[Horiodino]

@Horiodino that would be great, thank you!

Thank you @jaronoff97 I will start working on it right now🙂.

[Horiodino]

@jaronoff97 so i need to reserve a namespace so all the resources with mentioned labels falls under it .
can you provide the file to make changes .

[pavolloffay]

Currently, the Operator infers certain OpenTelemetry resource values (like service.name) by looking at various labels on the manifest

The operator is not looking at k8s object labels. It queries API server and gets deployment name, RS name through owner references. On top of that users can define additional attributes in the CR https://github.com/open-telemetry/opentelemetry-operator/blob/main/apis/v1alpha1/instrumentation_types.go#L87

I would like for OpenTelemetry to reserve the .opentelemetry.io/ namespace (as in resource.opentelemetry.io/service.version or attribute.opentelemetry.io/my.cool.attributte),

@austinlparker @jaronoff97 is there a spec documenting these values? I am not sure I fully understand the proposal in this ticket. Is the intention to change there resouce attribute names used by the operator, or make the operator to support these new opentelemetry.io attbiutes by propagating them from labels/annotations?

[austinlparker]

@austinlparker @jaronoff97 is there a spec documenting these values? I am not sure I fully understand the proposal in this ticket. Is the intention to change there resouce attribute names used by the operator, or make the operator to support these new opentelemetry.io attbiutes by propagating them from labels/annotations?

Well, ideally we can figure out the spec here... my thought would be that the default behavior wouldn't change, but that if someone used them it would set the appropriate attributes.

for example, if I was to set 'resource.opentelemetry.io/service.name' on a pod, then use instrumentation CRDs to add instrumentation, the value of that annotation would be propagated to the instrumentation.

[Horiodino]

@austinlparker @jaronoff97 is there a spec documenting these values? I am not sure I fully understand the proposal in this ticket. Is the intention to change there resouce attribute names used by the operator, or make the operator to support these new opentelemetry.io attbiutes by propagating them from labels/annotations?

Well, ideally we can figure out the spec here... my thought would be that the default behavior wouldn't change, but that if someone used them it would set the appropriate attributes.

for example, if I was to set 'resource.opentelemetry.io/service.name' on a pod, then use instrumentation CRDs to add instrumentation, the value of that annotation would be propagated to the instrumentation.

OK so you are saying use crd to extract the info and save related info in the namespace such as opentelemetry.io
can you provide some basic info regarding it. it will be more easier

[jaronoff97]

Agreed with austin. For where the change is made... it will mainly be in this file. What you should aim to do is add something that detects the patterns that austin mentions above and translate them in to the respective resource attributes or normal attributes. These can take precedence over the existing ones. Let me know if you need any more info.

[pavolloffay]

I am not sure how we can tell the SDK to attach regular attributes. There is no env var that can feed SDK with this info https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/configuration/sdk-environment-variables.md. Also note that the service name has a separate ENV variable, although it can be set as a resource attbitue https://github.com/open-telemetry/semantic-conventions/blob/main/docs/resource/README.md#semantic-attributes-with-dedicated-environment-variable

[Horiodino]

not sure about it but may be i got what you guys thinking . we need this namespace opentelemetry.io/ and if someone sets values in this namespace (like 'resource.opentelemetry.io/service.name') on a pod, we want the opentelemetry operator to recognize and use these values and do some stuff with it .

[pavolloffay]

Correct if an annotation on pod/deployment is set to resource.opentelemetry.io/my.attribute: some-value then this attribute should be set as a resource attribute.

[Horiodino]

OK i will start asap.

[Horiodino]

where i can identify (find ) where the operator infers values ,like service.name .
if this value set resource.opentelemetry.io/service.name' on a pod, then code (that i am going to code )should be able to understand and use this value.

[jaronoff97]

You will probably be making the changes here https://github.com/open-telemetry/opentelemetry-operator/blob/main/pkg/instrumentation/sdk.go#L406-L443

[Horiodino]

@jaronoff97 i have looked into the sdkinjector but its not retrieving the info that is required , mostly functions returns errors not the list of resources , but the function defination says List retrieves list of objects for a given namespace and list options. On a successful call, Items field in the list will be populated with the result returned from the server which list it is populating .

[jaronoff97]

How are you running it? This implies either a bad kubernetes configuration or no pods in your cluster.

[Horiodino]

im not running yet, im adding the functionality , for example to get the pod labels and annotations i need to get the pods running in cluster , so i need to use the Kubernetes api , and i see that opentalimetry have already used api defined with some modifications . when i use the sdkinjector it doesnot have the info required such as if i want to get the pods list using List function or Get the pods using Get function it doesnot return any value .

FUNCTION DEFINATION

List retrieves list of objects for a given namespace and list options. On a successful call, Items field in the list will be populated with the result returned from the server.```

[jaronoff97]

Like is done here you should be able to look at the pods' labels and annotations from the metadata field.

[Horiodino]

sorry i was getting thing wrong previously , got it and almost completed , after i find about the pod have
opentemetry.io/ as a prefix what should i do !

[Horiodino]

sorry for taking so long, i almost forgot about the issue, but now I just made simpler and more removed unnecessary things here is the output and also added for deployment too:

[Debug] Found label with opentelemetry.io/ prefix: opentelemetry.io/test-label
[Recognized] Pod: nginx-pod, Label Key: opentelemetry.io/test-label, Label Value: test-value
[Pod: nginx-pod, is opentelemetry.io/ label recognized: true ]

[Horiodino]

@jaronoff97

[jaronoff97]

Closing this in favor of the recent progress made by @zeitlinger in #3112 and #3204