Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(AWS SDK for Java) Automatically disable segments in SDK background processes #330

Open
millems opened this issue Apr 5, 2022 · 4 comments

Comments

@millems
Copy link

millems commented Apr 5, 2022

On the AWS SDK for Java 2.x, we sometimes make calls to AWS services as async background tasks, for example to cloudwatch to upload SDK metrics or to STS to refresh assumed-role credentials.

Customers that are using the automatic SDK instrumentation for x-ray are getting failures or warnings about a segment not being started in these background tasks because the SDK isn't starting a segment, and the customer doesn't control these service calls.

The AWS SDK for Java doesn't compile against the x-ray SDK, so we can't start a segment on behalf of the customer, but could we provide a "background task" flag to the x-ray execution interceptor to let it know that a segment isn't required because it isn't related to an ongoing trace?

I could even provide a pull request for this, as it should be relatively simple to implement on both sides myself.

Additional context from an affected user: aws/aws-sdk-java-v2#2919

@pingleig
Copy link
Member

pingleig commented Apr 5, 2022

@Zhenye-Na
Copy link

Any updates for issue aws/aws-sdk-java-v2#3217 ?

Thanks!

@atshaw43
Copy link
Contributor

atshaw43 commented Mar 27, 2023

Can you try removing the submodule aws-sdk-instrumentor and instrumenting only the clients you need?
https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-awssdkclients.html

You can also ignore missing context strategies to remove this from the logs, but this could hide issues in the future.
https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-configuration.html

@mjgp2
Copy link

mjgp2 commented Jun 21, 2023

We've got multiple issues with this happening where we have having to put workarounds in place where there isn't a context in background jobs/threads, e.g. publishing cloudwatch metrics, using the S3 client, or even using quartz jobs.

It's really poor that we have this experience, and I am hoping that we can get some visibility on this somehow to figure out how we can get a working "out of the box" experience for xray.

Perhaps could we configure xray to create a segment if one is missing - although I am unsure where the segment name would come from, perhaps the thread name?

CC @atshaw43

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants