How can I associate metrics and traces

[Woodrow1026]

I am using opentelemetry to collect metrics and traces, and I realized that metrics include Exemplar, which can be used to correlate traces. However, when I send the indicator data to sigNoz, I found that the labels no longer contain Exemplar, and I cannot directly jump to the tracking in the indicator ( P99 in Services can do this).

Is there any way to achieve an exemplar-like effect here please?

[pranay01]

@Woodrow1026 We don't have exemplars support yet but you can go from metrics to traces for latency charts for services. If you click on any points in the latency chart you should see an option to View Traces which you can click to see the relevant traces.

Does this serve your use case?

[Woodrow1026]

@pranay01 I am wondering, in our scenario, whether it is possible to associate metrics with traces, by carrying spanId in metrics or similar

[pranay01]

I think looking for more context on failed calls looks like a use case for traces. You should go to traces page, filter based on status code ( which will give you which calls have failed), filter for a specific service and then look into the details of traces

A quick video
https://www.loom.com/share/3f74cd323ac644a8b581e026f3a058c7

[Woodrow1026]

@pranay01 Thanks for your answer, but it doesn't seem to quite accomplish my goal

When the request volume is relatively large, traces may not be sampled (probability sampling), and the response of the filtering operation is usually slower than directly displaying metrics.

The ideal situation is that after defining the corresponding metrics, I can jump directly from the metrics to the failed trace and get the context I need. But this means that traces and metrics must be associated??

[ankitnayan]

@Woodrow1026 we can also enable switching directly to a trace by simply choosing any trace matching the applied filters and hence skipping the list of representing traces in between. We can also think about supporting exemplars.
We want to understand more on Exemplars.

1. How do you add traceID to exemplars? Is it manual?
2. Also, how do you limit the number of exemplars (how do you choose which one will be added when there are millions matching the criteria)?

[Woodrow1026]

@ankitnayan I'm sorry for the slow response, but I hope the following information is enough to answer your question.
1.Exemplars is specified in OTel Doc, and its content includes

• the value of the Measurement that was recorded by the API call.
• The time the API call was made to record a Measurement.
• The set of Attributes associated with the Measurement not already included in a metric data point.
• The associated trace id and span id of the active Span within Context of the Measurement at API call time.

2.OpenTelemetry distinguishes Metrics into asynchronous and synchronous in the SDK. In the practice of Java SDK, I found that synchronous Metrics will automatically carry Exemplars. According to the documentation, the SDK provides ExemplarFilter for filtering, but I haven't practiced it yet.