still-attached tasks aborted during runtime shutdown. expected? how to avoid?

[dan-da]

I made a JobQueue that is supposed to run jobs in serialized fashion, one at a time.

Internally it spawns two sub-tasks, and when a new job is added via JobQueue::add_job() a message is sent over an mpsc channel to a sub-task.

Everything works great except that under certain conditions during runtime shutdown the add_job() method gets called by a long-running application task (external to job-queue) and somehow the channel send() fails with a "channel closed" SendError.

This is a bit surprising to me because JobQueue has not yet been dropped and is still holding JoinHandle for the two sub-tasks in a TaskTracker, so they are not detached.

My questions are:

1. is this expected behavior during tokio runtime shutdown or could it possibly be a tokio bug?
2. if it is expected behavior then what can I do differently within JobQueue to prevent the send() from failing? If it fails and add_job() returns a SenderError then it is exposing internal details and breaks the encapsulation.

I read the tokio docs carefully but they are unclear on if attached tasks are handled differently from detached tasks during shutdown, and what the ordering is, if any. My (naive?) expectation would be that the primary tokio task would be aborted, and its cleanup would eventually drop my JobQueue, which would detach its sub-tasks, which would then be aborted by the runtime.

I have a fully documented playground here with JobQueue and a unit test that demonstrates the issue.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=d888dff82b5940950e1ae2b862592249

[Darksonn]

When the Runtime is shut down, all tasks are killed. Attached vs detached makes no difference.

[dan-da]

thx, that helps solidify my understanding.

btw, I was able to fix the test case in the playground by aborting and then awaiting the long-running task before the runtime is shutdown.

fixed playground