Feature macros for new features

[devreal]

Problem

Implementing new features included in a new version of MPI can take significant efforts for implementors and may happen over several releases. Users relying on the MPI version number to test for features may not be able to use a certain until all new features of the new MPI version are properly implemented, even if the desired feature is available in an earlier release of the implementation.

Proposal

New features should introduce a feature test macro, similar to C++. For example, __cpp_lib_atomic_ref can be used to test for std::atomic_ref introduced in C++20.

This will allow applications to conditionally use a feature as soon as they are available in one implementation without having to rely on MPI version numbers. It would also allow applications an easier path to testing features before they were voted into the standard.

Changes to the Text

A short introduction that explains the role of feature macros will be needed. This will mostly impact future features and would have to be enforced by the Forum.

Impact on Implementations

None (other than having to define the feature macro once the feature is supported).

Impact on Users

Earlier access to features, better support for conditional use of features.

References and Pull Requests

C++ feature macro documentation: https://en.cppreference.com/w/cpp/feature_test.html

[jeffhammond]

This should go in a side document. It should be lightweight and optional. It should be very easy to converge the two major implementations without requiring full forum activity.

[eschnett]

Feature macros are a good idea.

I want to mention that macros are a compile-time measure, and it is in many circumstances convenient to obtain information at run-time as well.

Feature macros are necessary to see whether a macro, a constant, or a function is declared. However, some features can be implemented without new syntax, and a compile-time check is then overly restrictive. For example, there could be various guarantees about multi-threading, or about efficiency of certain operations, or a new feature might be enabled via certain MPI_Info settings.

Many MPI implementations place great care at maintaining a backward-compatible ABI. This means that one could e.g. build against version 1.3 of an implementation and then run with version 1.4 which might offer the new feature. A compile-time check cannot describe this situation. This is particularly relevant for languages such as Python or Julia which are not compiled, and which only link against shared libraries.

[jeffhammond]

I think folks underestimate the challenge here. In practice, one needs to know the API support, the semantics of the API, and whether there are bugs in the implementation. I am dealing with this today in ARMCI-MPI, for example. I can test for the APIs but I cannot know from them whether MPI_Win_shared_query supports MPI_Win_allocate windows or not. I also cannot know whether there are bugs in MPI_Win_allocate or MPI_Accumulate. Today, I am testing the implementation version, which is the minimum necessary query, but still does not help me when RMA bugs are network-dependent.