Architecture: Tasks

Use cases

• Fire a single, independent piece of code (single task)
• Fire a single task with a single reply executed asynchronously
• Fire a single task and wait for reply (see Waitable tasks)
• Chain multiple tasks, in specific order (see Chaining multiple tasks)
• Fire a timed task (see Timed tasks)
• Cancel a task (see Cancellable tasks)

Waitable tasks

This feature has been fully implemented.

Waiting on a task is possible via its handle (see base::task_handle).

Cancellable tasks

See

• MSDN
• TBB

Task handle

Task and task handle should have reference to common task state which should store information such as: whether task cancellation was requested or the task has started, etc.

cancel() returning bool

It might be convenient if the cancel() method returned information whether cancellation succeeded.

Main disadvantage is that it may make task internals more complex.

See task status.

task status

Task's shared state should contain information about task's current status, i.e. whether it was started, cancelled, finished and so on. This should be an atomic and it should be updated using compare_and_exchange_strong() API. This should solve the problem with returning whether task was successfully canceled or not from the cancel() API.

cancellation token

All the related tasks share a cancellation token which on cancel should stop all the not yet executed tasks from being executed and synchronise with the already running tasks. The easiest way would be to pass to cancellation token the task object and on cancel acquire handle for that task and try to cancel or wait for it via a handle. But, what if client already acquire the handle? Should it be assumed that client either uses the handle explicitly or uses cancellation token, i.e. he can't use both features at the same time?

Questions

• What about tasks spawned by already running tasks?
  • They should inherit the cancellation token of the parent task, but how to implement this.

Chaining multiple tasks

• Use task continuation, preferably by adding then() API to the task class or task_handle class (which one is better?)
• continuations should not be invoked directly, they should go through message loop, otherwise a long continuation chain might cause starvation of other (unrelated) tasks
  With current design, message_loop is not able to retrieve continuation from the task, so it can't enqueue it. Only task itself could do it but that would cause coupling from task to message_loop.

Possible approaches:

• then() returns task object

  In such case task should be re-written so that it is only a proxy API to the internal task state (not shared state) created on the heap and shared among task instances as a std::shared_ptr. This would allow for task to be copyable (in fact, it would even be required).

• then() returns reference to the continuation task stored inside

  Add component (similar to callable) that would be able to store the task. Additionally, on setting the task it should return reference to that task. But what if someone moves that task out of the continuation object?

• then() returns pointer to the task

  To keep the API consistent tasks should be passed via pointers (exactly what I did not wanted).

• then() is task_handle API

  Somehow less convenient. Might require additional synchronisation. Continuations could be set after the task has completed so (probably) task_handle should be responsible for posting the continuation to the message loop but that gives rise to a set of new problems, e.g. which message loop? task_handle would have to be coupled with message_loop, etc. On the other hand, APIs such as wait_for_any() or wait_for_all() would have to take task_handle (and probably return one) so it might be confusing if they operate on a task_handle but then() is an API of a task But, wait*() and then() implement different concurrency models. The first one represents fork-join model and the latter one continuations, so maybe it is ok that they are separate.

Things to reconsider

• What if user passes task and its continuation manually to different threads?

Timed tasks

Tasks that should be executed no sooner than specified amount of time units.

This feature has been fully implemented.

Things to reconsider

Cancelled tasks sustain shared state

To throw cancelled exception, shared state has to be valid each time user calls get() on a task handle. But this would prolong the life of the shared state which, in normal case, is removed after the first call to get(). What should/can be done with this?

To introduce this swap (in base::task_handle) registration of deleting the shared state with throwing cancelled exception.

Currently, cancelled exception is thrown only at first call to base::task_handle::get(). All consecutive calls result in no state exception.

Task scheduler

Many of the API calls could be done asynchronously on any thread which should improve responsiveness of some components. This could be achieved with a task scheduler that would spread tasks on a pool of worker threads.

Single-threaded tasks

Some tasks are created, executed and cancelled on the same thread. For such tasks, there is no need for synchronisation. Removing it could improve performance.

Boost.Asio

Check if part of the features could be implemented with Boost.Asio (this could simplify part of the code related to task management).

lock-free non-priority queue

Most of the tasks are not timed tasks, i.e. they are not post with a delay value set. So the majority of tasks don't need to be enqueued on priority queues.

Thus the task scheduler could dispatch non-timed tasks to non-priority queues and timed tasks to worker threads that operate on priority queues. This would allow to utilise lock-free queues (see e.g. Boost.Lockfree) as task queues for non-timed tasks (I am not aware of a implementation of lock-free priority queue).

However, whether or not lock-free queues speed anything up that would have to be measured. Most probably that would be application-specific. So maybe the decision should be left to the users who first should benchmark their application with and without lock-free data structures and then decide which one should they use.

benchmark

A simple benchmark framework should be implemented (or "borrowed" from some other project). What should be benchmarked:

• compare usage of a simple type erased callable to std::function (btw. there was some implementation of similar component that claimed to be faster than std::function, check this)