Add initial hwloc support

[philipmarshall21]

This PR is a subset of the changes PR #1107 and is intended to isolate the integration of hwloc as an optional dependency to SOS from the addition of multi-NIC functionality

The changes to the oac_* files reflect the current status of the main branch (at the time of this PR, that is commit c1cfc910d92af43f8c27807a9a84c9c13f4fbc65) as the upstream repo does not have any tags.

The changes to the opal_* files align with the v5.0.1 tag of the upstream repo.

[davidozog]

Are the upstream OPAL changes taken from a release or a particular commit of OMPI? Either way, perhaps we should leave a note on this PR in case it ends up being helpful later.

[philipmarshall21]

Are the upstream OPAL changes taken from a release or a particular commit of OMPI? Either way, perhaps we should leave a note on this PR in case it ends up being helpful later.

The changes to the oac_* files reflect the current status of the main branch (currently, that is commit c1cfc910d92af43f8c27807a9a84c9c13f4fbc65) as the upstream repo does not have any tags.

The changes to the opal_* files align with the v5.0.1 tag of the upstream repo.

[wrrobin]

LGTM. We can explore other use-cases for hwloc later.

[davidozog]

Side note: The UCX row with pmi-mpi looks slow in testing, but I think we can pretty safely ignore that...

[philipmarshall21]

Two quick questions @wrrobin @davidozog:

1. With the addition of hwloc support, should we expect to see the rpath for hwloc when doing something like oshcc -show (rpath for libfabric and libsma are being set correctly)? If so, this is currently not the case, and I'll need to look into why this is not happening.
2. In the changes to init.c that add hwloc API calls, should the application exit if the calls fail? Right now an ERROR message is printed but the application continues running.

[wrrobin]

Two quick questions @wrrobin @davidozog:

1. With the addition of hwloc support, should we expect to see the rpath for hwloc when doing something like oshcc -show (rpath for libfabric and libsma are being set correctly)? If so, this is currently not the case, and I'll need to look into why this is not happening.
2. In the changes to init.c that add hwloc API calls, should the application exit if the calls fail? Right now an ERROR message is printed but the application continues running.

For 1. I would guess we should see the rpath, considering your changes in the PR. Are you using any other flags that might skip the configury code?
For 2. my thought is we should not terminate. If one of the hwloc APIs fail or return error, it should print a warning that hwloc is not enabled for some reason. But, we should let the application proceed. Do other runtimes behave the same?

[philipmarshall21]

Two quick questions @wrrobin @davidozog:

1. With the addition of hwloc support, should we expect to see the rpath for hwloc when doing something like oshcc -show (rpath for libfabric and libsma are being set correctly)? If so, this is currently not the case, and I'll need to look into why this is not happening.
2. In the changes to init.c that add hwloc API calls, should the application exit if the calls fail? Right now an ERROR message is printed but the application continues running.

For 1. I would guess we should see the rpath, considering your changes in the PR. Are you using any other flags that might skip the configury code? For 2. my thought is we should not terminate. If one of the hwloc APIs fail or return error, it should print a warning that hwloc is not enabled for some reason. But, we should let the application proceed. Do other runtimes behave the same?

Ok, I've made the suggested changes (ensure rpath for hwloc is set in configure.ac, failed hwloc API calls now send warning message rather than error).

[davidozog]

@philipmarshall21 - I ran into a couple minor things but pending those, I think this PR is ready to merge. All the RPATH stuff is working on my end, thanks for patching that.

[davidozog]

It's a minor thing, but prob worth addressing if possible - we see this warning when building with hwloc but without enforcing single socket or numa node:

../../src/init.c:452:5: warning: label ‘hwloc_exit’ defined but not used [-Wunused-label]
  452 |     hwloc_exit:

One option might be to remove hwloc_issue_topology (it's not adding much value anyway), then we can remove hwloc_exit (and revive it if it's ever needed).

[davidozog]

Super minor, but looks like we lost a newline relative to upstream on this file.