How to properly use OpenTelemetry to instrument fork-join (or in fact, join-fork / gather & scatter) of multiple traces?

[mitoihs]

Question

Describe your environment.

Microservices in C#. Diagram which shows a problematic part:

What are you trying to achieve?

One of my microservices batches messages from other ones and process them in a mixed way - processing operations and messages are in many-to-many relationship. It can process multiple messages in a single operation and a single message can be processed part by part with multiple operations.

I thought that I should create traces like:

but the documentation says:

When using the scatter/gather (also called fork/join) pattern, the root operation starts multiple downstream processing operations and all of them are aggregated back in a single Span. This last Span is linked to many operations it aggregates. All of them are the Spans from the same Trace. And similar to the Parent field of a Span. It is recommended, however, to not set the parent of the Span in this scenario as semantically the parent field represents a single parent scenario, in many cases the parent Span fully encloses the child Span. This is not the case in scatter/gather and batch scenarios.

So what should I do when I have a span which is related to multiple traces?

I'm aware that there are no tools that could display traces from my scenario in a logically correct way, that's not an issue for me. (but if you know any which may support such a scenario, I'd be grateful)

[rholdorf]

I have a very similar scenario. Any progress on how to do it properly?

[mjpowersjr]

This may be useful: https://opentelemetry.io/docs/specs/otel/overview/#links-between-spans