Nonblocking Collectives

content/library_constants.tex

@@ -84,6 +84,15 @@
 See Section~\ref{subsec:shmem_ctx_create} for more detail about its use.
 \tabularnewline \hline
 %%
+\LibConstDecl{SHMEM\_REQ\_INVALID} &
+A value corresponding to an invalid request handle.
+This value can be used to initialize or update request handles to indicate
+that they do not reference a valid request.
+When managed in this way, applications can use an equality comparison
+to test whether a given request handle references a valid request.
+See Section~\ref{subsec:nb_coll} for more detail about its use.
+\tabularnewline \hline
+%%
 \LibConstDecl{SHMEM\_SIGNAL\_SET} &
 An integer constant expression corresponding to the signal update set operation.
 See Section~\ref{subsec:shmem_put_signal} and

content/nb_collectives_intro.tex

@@ -1,25 +1,26 @@
 An \openshmem nonblocking collective operation, like a blocking collective
 operation, is a group communication operation among the
-participants of the team. All participants of the team are required to call the
-collective operation.
+participants of the team. All \acp{PE} in the team are required to call the
+collective operation and each collective operation must be initiated in the same
+order across all \acp{PE} while the execution may be performed in any order.
 
 \begin{enumerate}
 
 \item Invocation semantics: Upon invocation of a nonblocking collective routine,
-the operation is initiated and the routine returns without ensuring completion. All participants of the Team
+the operation is initiated and the routine returns without ensuring completion. All \acp{PE} in the team
 must call this routine with identical arguments.
 
-\item Collective Types: The nonblocking variants supported include barrier all, alltoall,
-and broadcast collectives. Other collective operations such as
-reductions, collect, barrier, alltoalls, and sync will not have nonblocking variants.
+\item Collective Types: The nonblocking variants supported include the alltoall
+and broadcast collectives. All other collective operations such as
+reductions, collect, fcollect, barrier, barrier all, alltoalls, sync, and sync all will not have nonblocking variants.
 
-\item Completion semantics:  \openshmem programs can learn the status of the collective operations
+\item Completion semantics: \openshmem programs can learn the status of the collective operations
 using the \FUNC{shmem\_req\_test} routine. The operation is completed after
-at least one call to \FUNC{shmem\_req\_test} or a call to \FUNC{shmem\_req\_wait}.
+a call to \FUNC{shmem\_req\_test} or a call to \FUNC{shmem\_req\_wait}.
 
 \item Threads: While using SHMEM\_THREAD\_MULTIPLE, the \openshmem
 programs are not allowed to call multiple collective operations on different threads
-and the same Team.
+and the same team.
 
 \end{enumerate}
 

content/shmem_alltoall_nb.tex

@@ -55,7 +55,7 @@
     {\bf Invocation and completion}: A call to the nonblocking alltoall routine initiates the operation and returns
     immediately without necessarily completing the operation. On success,
     an opaque request handle is created and returned. The
-    operation is completed after at least one call to \FUNC{shmem\_req\_test} or
+    operation is completed after a call to \FUNC{shmem\_req\_test} or
     a call to \FUNC{shmem\_req\_wait}. When the operation is complete, the request handle
     is deallocated and cannot be reused.
 

content/shmem_broadcast_nb.tex

@@ -59,7 +59,7 @@
     A call to the nonblocking broadcast routine initiates the operation and returns
     immediately without necessarily completing the operation. On success,
     an opaque request handle is created and returned. The
-    operation is completed after at least one call to \FUNC{shmem\_req\_test} or a
+    operation is completed after a call to \FUNC{shmem\_req\_test} or a
     call to \FUNC{shmem\_req\_wait}. When the operation is complete, the request handle
     is deallocated and cannot be reused.
 
@@ -82,9 +82,9 @@
     Upon completion of a nonblocking broadcast routine, the following are true for the local
     \ac{PE}:
     \begin{itemize}
-    \item The \dest{} data object is
-      updated.
-    \item The \source{} data object may be safely reused.
+    \item The \dest{} data object is updated.
+    \item If the local \ac{PE} is \VAR{PE\_root}, the data has been copied
+          out of the \source{} data object.
     \end{itemize}
 }
 

content/shmem_collective_test.tex

@@ -5,7 +5,7 @@
 \begin{apidefinition}
 
 \begin{Csynopsis}
-int @\FuncDecl{shmem\_req\_test}@(shmem_req_h request);
+int @\FuncDecl{shmem\_req\_test}@(shmem_req_h *request);
 \end{Csynopsis}
 
 \begin{apiarguments}
@@ -17,9 +17,10 @@
 \apidescription{
     A call to \FUNC{shmem\_req\_test} returns immediately. If the
     operation identified by the request is completed, it returns
-    zero, and the request object is deallocated and cannot be reused.
-    If the operation is not completed, it returns a non-negative integer. If
-    the request object is not valid, behavior is undefined.
+    zero, and the request object is deallocated and set to \LibConstRef{SHMEM\_REQ\_INVALID}. 
+    If the operation is not completed, it returns a non-negative integer.
+    If the request object is not valid (i.e., it is set to \LibConstRef{SHMEM\_REQ\_INVALID}),
+    no operation is performed and a negative value is returned.
 
     In a multithreaded environment, \FUNC{shmem\_req\_test} can be called by
     different threads but on different request objects. It is the responsibility

content/shmem_collective_wait.tex

@@ -6,7 +6,7 @@
 \begin{apidefinition}
 
 \begin{Csynopsis}
-int @\FuncDecl{shmem\_req\_wait}@(shmem_req_h request);
+int @\FuncDecl{shmem\_req\_wait}@(shmem_req_h *request);
 \end{Csynopsis}
 
 \begin{apiarguments}
@@ -17,12 +17,13 @@
 
 \apidescription{
 
-The \FUNC{shmem\_req\_wait} function is a blocking operation used to determine whether an
-operation identified by the request object has been completed. If the operation
-is completed, \FUNC{shmem\_req\_wait} returns zero, and the request object is
-deallocated and cannot be reused. If the operation has not been completed,
-\FUNC{shmem\_req\_wait} blocks until the operation completes and then returns
-zero. If the request object is not valid, behavior is undefined.
+The \FUNC{shmem\_req\_wait} function is a blocking operation used to
+determine whether an operation identified by the request object has
+been completed. When the operation is completed, \FUNC{shmem\_req\_wait} returns
+zero, and the request object is deallocated and set to \LibConstRef{SHMEM\_REQ\_INVALID}.
+If the request object is not valid (i.e., it is set to
+\LibConstRef{SHMEM\_REQ\_INVALID}), no operation is performed and a negative
+value is returned.
 
 In a multithreaded environment, \FUNC{shmem\_req\_wait} can be called by different
 threads but on different request objects. It is the responsibility of the

main_spec.tex

@@ -387,9 +387,6 @@ \subsubsection{\textbf{SHMEM\_REDUCTIONS}}\label{subsec:shmem_reductions}
 \subsection{Nonblocking Collective Routines}\label{subsec:nb_coll}
 \input{content/nb_collectives_intro.tex}
 
-\subsubsection{\textbf{SHMEM\_BARRIER\_ALL\_NB}}\label{subsec:shmem_barrier_all_nb}
-\input{content/shmem_barrier_all_nb.tex}
-
 \subsubsection{\textbf{SHMEM\_BROADCAST\_NB}}\label{subsec:shmem_broadcast_nb}
 \input{content/shmem_broadcast_nb.tex}