Proposal: Task Abstraction Framework for Multi-Task Management in MJPC

[huihuaNvidia2023]

The MJPC repository demonstrates multi-task control through several examples where tasks coexist in single files, each requiring specific residual parameters and weightings. While these examples are instructive, managing task transitions (like smoothly fading between residuals) remains challenging.

Has there been consideration of developing a higher-level task abstraction framework? For example, allowing users to define tasks via YAML configurations and implement clean switching logic between them? This abstraction would significantly enhance MJPC's deployability in real-world applications.

I'd appreciate any thoughts or suggestions on this architectural approach.

[yuvaltassa]

This goes back to the question of "what is a task". Right now we have the notion of "mode", as you can see in Quadruped Flat, which is basically a multi-task task.

There is another approach by @nkhosh (currently in an in-progress PR #341) that led to this paper.

Writing tasks in YAML is certainly not an option since you need to be able to write custom code in the residual...

So while I agree that enhancing task reusability is a valuable goal, I'm not sure there is a specific idea here. Would you like to propose something more concrete?

[huihuaNvidia2023]

Thanks for the pointers! I think the example is one-step closer to a generic task plan framework. The contact frame can be considered as an abstraction of task specification. Then the user only need to specify the needed parameters in the json file.

Similarly for other residuals, we can define them in a more generic way and then users only need to specify certain parameters. For example, there can be a residual of relative pose of two bodies. User need to input the names of two bodies and the dimensions of the 3D pose to be restricted.

I see this is the first step, as it allow users to reuse residuals and specify them with a configuration file.

The next step is how to handle tasks switch, which I am also looking for an answer :).

Coming back to task definition, it can be quite different depending on use cases. Within the MJPC context, a task represents a set of residuals and different tasks will have different set of residuals. For example, walking can be a task. User can update certain parameters to walk slower or faster or even turning. Walking with arms holding a object is a different task. Here is a list of tasks that I can think of.

• walk with free moving arms
• walk with carrying object
• pick object with single arm
• place object with single arm
• pick object with dual arms
• place object with dual arms
• run