Provide access to resource usage for processes and nodes

[rhc54]

Define two attributes by which applications and/or tools can request
resource usage of processes and nodes. Define a structure for each
case to contain the information, and associated macros for constructing
and destructing those structures.

Signed-off-by: Ralph Castain [email protected]

[rhc54]

Please use emoji reactions ON THIS COMMENT to indicate your position on this proposal.

• You do not need to vote on every proposal
• If you have no opinion, don't vote - that is also useful data
• If you've already commented on this issue, please still vote so
  we know your current thoughts
• Not all proposals solve exactly the same problem, so we may end
  up accepting proposals that appear to have some overlap
  This is not a binding majority-rule vote, but it will be a very
  significant input into the corresponding ASC decision.

Here are the meanings for the emojis:

• Hooray or Rocket: I support this so strongly that I
  want to be an advocate for it
• Heart: I think this is an ideal solution
• Thumbs up: I'd be happy with this solution
• Confused: I'd rather we not do this, but I can tolerate it
• Thumbs down: I'd be actively unhappy, and may even consider
  other technologies instead
  If you want to explain in more detail, feel free to add another
  comment, but please also vote on this comment.

[jjhursey]

Minor wording suggestions.

[jjhursey]

Similar to the PMIX_PROC_RESOURCE_USAGE should we point out that:

This attribute can be used with qualifiers such as \refattr{PMIX_SESSION_ID} to identify the session of the namespace whose statistics are being requested.

[jjhursey]

Do we have a link to more descriptions on each of these stats? We take them from Linux so a pointer to a good manpage description would be helpful to someone new to these fields.

[jjhursey]

We should add a Python type for these.

[jjhursey]

This PR adds some new structures. This PR is on the provisional track.

There are some minor notes above that need to be addressed before the presentation.

[jjhursey]

We need to update the pmix_value_t structure with these two :

        pmix_proc_stats_t *pstats;
        pmix_node_stats_t *ndstats;

I see quite a few other fields that need to be added to the pmix_value_t that are in OpenPMIx v4. I'll add those in a separate PR if you can add the above two to this PR.

[jjhursey]

Implementation Notes:

• Implemented in Shift the process/node stat support to PMIx openpmix/openpmix#2039
• With the exception of PMIX_PROC_RESOURCE_USAGE and PMIX_NODE_RESOURCE_USAGE which are not yet provided in OpenPMIx (easy enough to fix)

[jjhursey]

Notes from 2Q 2021 ASC Meeting (day 1)

• Comments need to be addressed. If they can be addressed before Day 2 then we can have a Revision Exception vote.
  • this comment regarding the need for documentation of the various fields in the pmix_proc_stats_t. Should also document the fields of pmix_node_stats_t (what if the value is not available - should it be set to 0?).
  • this comment implementation requirement for PMIX_PROC_RESOURCE_USAGE and PMIX_NODE_RESOURCE_USAGE missing in OpenPMIx.
  • this comment missing pmix_proc_stats_t and pmix_node_stats_t fields in the pmix_value_t
• Question regarding the choice of the data type for fields in pmix_node_stats_t - why float and not size_t/double/...? Is it better to have an accessor function instead of an explicit data structure?
• Question if there are users of this interface at the moment.

[jjhursey]

PMIx ASC 2Q 2021: May 11 & May 13

• This PR has been pushed back for the reasons summarized in this comment
• Once those comments have been addressed this PR can be brought back for presentation.

[jjhursey]

Triage file: https://github.com/openpmix/pmix-tests/blob/08705b12294fc5020f2af951b9bb10b36b6cec8d/check-standard/etc/openpmix_master-pmix-standard_master.txt#L277C1-L309