Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce support for local complete #497

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions content/shmem_local_complete.tex
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
\apisummary{
Waits for data to be copied out of the \VAR{source} array on all
outstanding non-blocking \OPR{Put} and non-blocking \OPR{put-with-signal}
issued by a \ac{PE}.
}

\begin{apidefinition}

\begin{Csynopsis}
void @\FuncDecl{shmem\_local\_complete}@(void);
void @\FuncDecl{shmem\_ctx\_local\_complete}@(shmem_ctx_t ctx);
\end{Csynopsis}

\begin{apiarguments}
\apiargument{IN}{ctx}{A context handle specifying the context on which to
perform the operation. When this argument is not provided, the operation is
performed on the default context.}
\end{apiarguments}

\apidescription{
The \FUNC{shmem\_local\_complete} routine ensures local completion of all
non-blocking \OPR{Put} and non-blocking \OPR{put-with-signal} operations
issued by a \ac{PE}. Local completion guarantees the reusability of the
\VAR{source} buffers associated with a \ac{PE} issuing the operation.
Local completion does not guarantee any ordering and/or delivery of
completion with any visibility guarantees on all \acp{PE}.
Return from \FUNC{shmem\_local\_complete} just guarantees that the data has
been copied out of the \VAR{source} array on all previously posted
non-blocking \OPR{Put} and non-blocking \OPR{put-with-signal} operations
in the local \ac{PE}. Memory ordering routines as supported in
Section~\ref{subsec:memory_order} is still required to provide
mechanims to ensure ordering and/or delivery of completions on the
non-blocking \OPR{Put} and non-blocking \OPR{put-with-signal} operations.
}


\apireturnvalues{
None.
}

\begin{apiexamples}

\apicexample
{The following example uses \FUNC{shmem\_quiet} in a \Cstd[11] program: }
{./example_code/shmem_local_complete_example.c}
{\FUNC{shmem\_local\_complete} allows reusing the \VAR{source} buffer
without waiting for the completion and global visibility on target process
\VAR{tpe}}
\end{apiexamples}

\end{apidefinition}

40 changes: 40 additions & 0 deletions example_code/shmem_local_complete_example.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
#include <shmem.h>
#include <stdio.h>

#define SIZE 10

int main(void) {
int count = 0;
shmem_init();
static int64_t sig_addr = 0;
int *dest = shmem_malloc (SIZE * sizeof(int));
int *source = shmem_malloc (SIZE * sizeof(int));
int mype = shmem_my_pe();
int npes = shmem_n_pes();

for (int i = 0; i < SIZE; i++) {
source[i] = count;
dest[i] = 0;
}

if (mype == 0) {
for (int tpe = 1; tpe < npes; tpe++) {
shmem_put_signal_nbi(dest, source, SIZE, &sig_addr, 1, SHMEM_SIGNAL_SET, tpe);
shmem_local_complete();
count++;
for (int i = 0; i < SIZE; i++) {
source[i] = count;
}
}
} else {
shmem_wait_until(&sig_addr, SHMEM_CMP_EQ, 1);
for (int i = 0; i < SIZE; i++) {
if (dest[i] != mype) {
count++;
}
}
if (count) fprintf(stderr, "Program Error\n");
}
shmem_finalize();
return 0;
}
15 changes: 12 additions & 3 deletions main_spec.tex
Original file line number Diff line number Diff line change
Expand Up @@ -486,9 +486,18 @@ \subsubsection{Synchronization and Communication Ordering in OpenSHMEM}
\input{content/synchronization_model.tex}





\color{teal}
\subsection{Local Completion Operations}\label{sec:lcomplete}
This section specifies the OpenSHMEM support for \OPR{local-complete}
operation. The \OPR{local-complete} operation provide a method for reusing
the \VAR{source} buffers associated with the process initiating the
non-blocking remote memory access routines and signaling operations as
specified in Section~\ref{subsec:shmem_put_nbi} and
Section~\ref{subsec:shmem_put_signal_nbi} respectively.

\subsubsection{\textbf{SHMEM\_LOCAL\_COMPLETE}}\label{subsec:shmem_local_complete}
\input{content/shmem_local_complete.tex}
\color{black}

\subsection{Distributed Locking Routines}
The following section discusses \openshmem locks as a mechanism to provide
Expand Down