Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Exporter] Prevent infinite recursion with grpc #3039

Open
marcalff opened this issue Aug 28, 2024 · 2 comments
Open

[Exporter] Prevent infinite recursion with grpc #3039

marcalff opened this issue Aug 28, 2024 · 2 comments
Labels
bug Something isn't working issue:blocked Fix blocked, waiting for other fixes as prerequisites issue:blocking This issue is preventing other fixes

Comments

@marcalff
Copy link
Member

marcalff commented Aug 28, 2024

The grpc C++ library is instrumented using opentelemetry-cpp:

The opentelemetry-cpp OTLP GRPC exporter uses grpc

In the OTLP GRPC exporter implementation, opentelemetry-cpp needs to:

  • disable instrumentation before making calls to grpc
  • reenable instrumentation after grpc calls

so avoid infinite recursion.

This is blocking:

  • linking opentelemetry-cpp with an instrumented grpc library is likely to cause issues.

This is blocked by the spec:

@marcalff marcalff added the bug Something isn't working label Aug 28, 2024
@github-actions github-actions bot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Aug 28, 2024
@marcalff marcalff added issue:blocking This issue is preventing other fixes issue:blocked Fix blocked, waiting for other fixes as prerequisites and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Aug 28, 2024
@lalitb
Copy link
Member

lalitb commented Aug 30, 2024

Thanks for raising the issue. Just to add the approach used by otel-python - https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-python%20_SUPPRESS_INSTRUMENTATION_KEY%20&type=code

  • Add the SUPPRESSION_KEY as true in the current active context - this can be done at the start on the Exporter::export() method, and set back to false at the end.
  • Check for this key status from the context within the processor, and don't send the span to the exporter if set as true.

However, this approach has some constraint. Any instrumentation that needs to be suppressed in the libraries used by the OTLP exporter should be invoked from the exporter thread since the active context is thread-local in otel-cpp.

Similar approach used in otel-dotnet - https://github.com/open-telemetry/opentelemetry-dotnet/blob/1a836432840d657926923ccce4d09c4a4e08728c/src/OpenTelemetry/SuppressInstrumentationScope.cs#L12

@punya
Copy link
Member

punya commented Sep 23, 2024

Here is a similar issue with a fix (OTel-C++ but a different gRPC-based exporter): googleapis/google-cloud-cpp#14611

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working issue:blocked Fix blocked, waiting for other fixes as prerequisites issue:blocking This issue is preventing other fixes
Projects
None yet
Development

No branches or pull requests

3 participants