HPX, APEX and nested annotations

[albestro]

I'm working on a library based on HPX that I'd like to trace.

Without any knowledge of APEX, I would go for annotating all relevant functions, and I find myself annotating both very generic and very specific tasks.

I start by annotating a generic function like generic_op and for free I get annotations in all algorithms that uses this function...but it's not completely meaningful, maybe I can add a better name for each specific algorithm that uses this generic function, e.g. algorithmA::first_op or algorithmB::update or whatever else.
So, I end up having an annotated function that has a nested annotated function call in it.

From what I see, this concept of "nested annotation" is not available, indeed if I do general+specific annotation I end up with all the tasks showing up in the trace as the most generic annotation (nested), while containing ones are really marginal. From a quick discussion with an HPX guy, it seems that APEX annotations are closed as soon as another annotation is started, explaining why if I analyze the specific annotations they are not representative of the entire task duration.

My main questions are:

• am I expecting too much from the annotations? if yes, can you suggest a different approach/guideline about tracing?
• would it be possible to introduce the concept of nested annotations? I see that in Google Trace Format there is this concept of "parent task", which looks interesting from the output point of view.

@khuck I'm asking here on HPX repo instead of APEX one because it might be a matter of HPX+APEX details and not just APEX.

[khuck]

@albestro The annotation has a couple of different implementations, with unfortunately confusing names.

annotated_function() is specified when the task is created, and will result in an apex::update_task() call to assign a new identifier to the task before it is started. Here is an example:

hpx/tests/unit/apex/annotation_check.cpp

Lines 150 to 152 in b01c5bc

	hpx::util::annotated_function(
	[](auto&&, auto&&) { dummy_task(std::size_t(1000)); }, dfs),
	f1, f2));

annotate_function() is another HPX method that is used to add annotations, but I don't think it is used anywhere. It has the same effect, that eventually apex::update_task() is called. If I am assuming correctly, generic_op results in a new HPX task that is automatically annotated (as you mentioned).

If I understand correctly, in your case, you have:

specific algorithm {
    ...
    hpx::async( ... annotated_function ( ... generic_operation () ) );
    ...
}

and you'd like to see both annotated, with a parent-child relationship. APEX already tracks parent-child relationships between tasks. I think we want:

specific algorithm {
    std::string timerName{"specific"};
    // the scoped timer will automatically start on construction, and stop when it goes out of scope
    auto timer = hpx::util::external_timer::scoped_timer(hpx::util::external_timer::new_task(name));
    ...
    hpx::async( ... annotated_function ( ... generic_operation () ) );
    ...
}

Does this sound right?

Depending on your needs, you might also like to use the new "task tree" output - just set APEX_TASKTREE_OUTPUT=1 before running, and each locality will write out a task tree.0.dot file that can be rendered with graphviz/dot with the command dot -Tpng -O tasktree.0.dot. Here's an OpenMP example to see what I mean: http://www.nic.uoregon.edu/~khuck/tasktree.0.dot.pdf