Hello world distributed example and numa domain(s)

[erleria]

Hi,

Just a quick question. How do you define a NUMA domain (in terms of "terminology") and how is it related to locality?
Because when I use the hello world distributed example on a NUMA system (dual socket) it prints only messages from locality 0. Is the whole NUMA domain (and so the two sockets) considered as locality 0 or is locality 0 one of the two sockets?

• HPX Version: github version from today
• Platform (compiler, OS): Ubuntu Jammy,

[hkaiser]

An HPX locality corresponds to an OS process. A locality can span multiple NUMA domains on the same physical compute node. However, as you can run several localities on the same compute node, those may share a NUMA domain. Everything depends on how you run the application (i.e., what command line options you pass and how many process instances of the HPX application you launch).

[erleria]

Thank you @hkaiser. Also in HPX do you consider a socket and a numa node to be the same thing? This point is not clear, I just copy paste the documentation:

get_number_of_sockets() const
Return the number of available NUMA domains.

get_number_of_numa_nodes() const
Return the number of available NUMA domains.

Moreover I would like to know if there is a possibility in HPX to list the available GPUs (co_proc/osdev_gpu) per socket? Or is there any other way to access some hwloc data structures directly from HPX.

[hkaiser]

A NUMA domain and a socket in HPX corresponds to whatever hwloc reports. For the GPU's, please use whatever tools you prefer to access this information (e.g., directly using hwloc).