task_local, logging context and tokio::task::spawn_blocking

[mgumbley-resilient]

Hi, I'm using Rocket 0.5 (Tokio 1.6.1) for a microservice. The incoming requests are JSON documents which all contain a correlation id.

In earlier versions that were thread-based, we added this correlation id to thread-local storage, and used log4rs/log_mdc to emit the id on every log statement emitted during processing of the call, so we could easily see which downstream systems/crates were used for this call.

This does not work in Rocket 0.5 since it's async.. the various async tasks in our code are switched between threads so that a task may generate logging with the wrong correlation id. The upgrade to Rocket 0.5 meant we had to bridge between async and sync code using tokio::task::spawn_blocking and tokio::task::block_in_place(move || handle.block_on(... The system works, but is messy.

I see that tokio has task_local! and after reading "How does task_local! work" at #6111 tried to switch to it. The correlation id logging works until the first tokio::task::spawn_blocking, then the thread changes, and it's not subsequently logged.

Is task_local! incompatible with sync/async bridging? Must we fully convert all code to async to get this working?

Thank you in advance,
Matt Gumbley

[Darksonn]

Unfortunately, the task_local! utility does not work once you leave the current task. The contents of spawn_blocking are considered different tasks, so the id will not be reflected within them.

You could make your own spawn_blocking wrapper that captures the value before spawning, and sets it inside the spawn_blocking task.

[mgumbley-resilient]

Thank you Alice, I'll try the wrapper approach. Changing all sync code to async would be best but will have to wait for now.