Skip to content

Commit

Permalink
Maintenance: Rename ginkgo dpcpp to sycl (#493)
Browse files Browse the repository at this point in the history
Ginkgo now refers to DPC++ as SYCL (inline with what Intel is doing), so
this just updates our interface to match the naming convention.

---------

Co-authored-by: David Gardner <[email protected]>
  • Loading branch information
balos1 and gardner48 authored May 24, 2024
1 parent 6ff1a00 commit 8cdbbec
Show file tree
Hide file tree
Showing 15 changed files with 42 additions and 32 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ Updated the CMake variable `HIP_PLATFORM` default to `amd` as the previous
default, `hcc`, is no longer recognized in ROCm 5.7.0 or newer. The new default
is also valid in older version of ROCm (at least back to version 4.3.1).

Renamed the DPCPP value for the `SUNDIALS_GINKGO_BACKENDS` CMake option to `SYCL`
to match Ginkgo's updated naming convention.

Changed the CMake version compatibility mode for SUNDIALS to `AnyNewerVersion`
instead of `SameMajorVersion`. This fixes the issue seen
[here](https://github.com/AMReX-Codes/amrex/pull/3835).
Expand Down
2 changes: 1 addition & 1 deletion cmake/SundialsTPLOptions.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ sundials_option(ENABLE_GINKGO BOOL "Enable Ginkgo support" OFF)
sundials_option(Ginkgo_DIR PATH "Path to the root of a Ginkgo installation" "${Ginkgo_DIR}"
DEPENDS_ON ENABLE_GINKGO)

sundials_option(SUNDIALS_GINKGO_BACKENDS STRING "Which Ginkgo backend(s) to build the SUNDIALS Ginkgo interfaces for (REF, OMP, CUDA, HIP, DPCPP)" "REF;OMP"
sundials_option(SUNDIALS_GINKGO_BACKENDS STRING "Which Ginkgo backend(s) to build the SUNDIALS Ginkgo interfaces for (REF, OMP, CUDA, HIP, SYCL)" "REF;OMP"
DEPENDS_ON ENABLE_GINKGO)

sundials_option(GINKGO_WORKS BOOL "Set to ON to force CMake to accept a given Ginkgo configuration" OFF
Expand Down
2 changes: 1 addition & 1 deletion cmake/macros/SundialsAddExamplesGinkgo.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ macro(sundials_add_examples_ginkgo EXAMPLES_VAR)
elseif(backend MATCHES "HIP")
set_source_files_properties(${example} PROPERTIES LANGUAGE CXX)
set(vector nvechip)
elseif(backend MATCHES "DPCPP")
elseif(backend MATCHES "SYCL")
set(vector nvecsycl)
elseif(backend MATCHES "OMP")
set(vector nvecopenmp)
Expand Down
4 changes: 2 additions & 2 deletions cmake/tpl/SundialsGinkgo.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ if(Ginkgo_FOUND AND (NOT GINKGO_WORKS))
print_error("SUNDIALS_GINKGO_BACKENDS includes HIP but HIP is not enabled. Set ENABLE_HIP=ON or change the backend.")
endif()

if(SUNDIALS_GINKGO_BACKENDS MATCHES "DPCPP" AND NOT ENABLE_SYCL)
print_error("SUNDIALS_GINKGO_BACKENDS includes DPC++ but SYCL/DPC++ is not enabled. Set ENABLE_SYCL=ON or change the backend.")
if(SUNDIALS_GINKGO_BACKENDS MATCHES "SYCL" AND NOT ENABLE_SYCL)
print_error("SUNDIALS_GINKGO_BACKENDS includes SYCL but SYCL is not enabled. Set ENABLE_SYCL=ON or change the backend.")
endif()

if(SUNDIALS_GINKGO_BACKENDS MATCHES "OMP" AND NOT ENABLE_OPENMP)
Expand Down
5 changes: 4 additions & 1 deletion doc/shared/RecentChanges.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ Updated the CMake variable ``HIP_PLATFORM`` default to ``amd`` as the previous
default, ``hcc``, is no longer recognized in ROCm 5.7.0 or newer. The new
default is also valid in older version of ROCm (at least back to version 4.3.1).

Renamed the DPCPP value for the :cmakeop:`SUNDIALS_GINKGO_BACKENDS` CMake option
to ``SYCL`` to match Ginkgo's updated naming convention.

Changed the CMake version compatibility mode for SUNDIALS to ``AnyNewerVersion``
instead of ``SameMajorVersion``. This fixes the issue seen `here
<https://github.com/AMReX-Codes/amrex/pull/3835>`_.
Expand Down Expand Up @@ -112,4 +115,4 @@ ARKODE-wide functions.

Deprecated the `ARKStepSetOptimalParams` function. Since this function does not have an
ARKODE-wide equivalent, instructions have been added to the user guide for how
to retain the current functionality using other user-callable functions.
to retain the current functionality using other user-callable functions.
14 changes: 9 additions & 5 deletions doc/shared/sundials/Install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -665,10 +665,14 @@ illustration only.
.. cmakeoption:: SUNDIALS_GINKGO_BACKENDS

Semi-colon separated list of Ginkgo target architecutres/executors to build for.
Options currenty supported are REF (the Ginkgo reference executor), OMP, CUDA, HIP, and DPC++.
Options currenty supported are REF (the Ginkgo reference executor), OMP, CUDA, HIP, and SYCL.

Default: "REF;OMP"

.. versionchanged: x.y.z
The ``DPCPP`` option was changed to ``SYCL`` to align with Ginkgo's naming convention.
.. cmakeoption:: ENABLE_KOKKOS

Enable the Kokkos based vector.
Expand Down Expand Up @@ -1480,13 +1484,13 @@ SUNDIALS has been tested with MAGMA version v2.6.1 and v2.6.2.

.. _Installation.CMake.ExternalLibraries.OneMKL:

Building with oneMKL
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Building with oneMKL for SYCL
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The Intel `oneAPI Math Kernel Library (oneMKL)
<https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl.html>`_
includes CPU and DPC++ interfaces for LAPACK dense linear algebra routines. The
SUNDIALS oneMKL interface targets the DPC++ routines, to utilize the CPU routine
includes CPU and SYCL/DPC++ interfaces for LAPACK dense linear algebra routines. The
SUNDIALS oneMKL interface targets the SYCL/DPC++ routines, to utilize the CPU routine
see :numref:`Installation.CMake.ExternalLibraries.LAPACK`.

To enable the SUNDIALS oneMKL interface set ``ENABLE_ONEMKL`` to ``ON`` and
Expand Down
4 changes: 2 additions & 2 deletions examples/cvode/ginkgo/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ set(cpu_gpu_examples

sundials_add_examples_ginkgo(cpu_gpu_examples
TARGETS sundials_cvode
BACKENDS REF OMP CUDA HIP DPCPP)
BACKENDS REF OMP CUDA HIP SYCL)

# Examples that only support CPU Ginkgo backends
set(cpu_examples
Expand All @@ -39,7 +39,7 @@ if(EXAMPLES_INSTALL)
if(SUNDIALS_GINKGO_BACKENDS MATCHES "HIP")
list(APPEND vectors nvechip)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "DPCPP")
if(SUNDIALS_GINKGO_BACKENDS MATCHES "SYCL")
list(APPEND vectors nvecsycl)
endif()
if((SUNDIALS_GINKGO_BACKENDS MATCHES "OMP") OR
Expand Down
10 changes: 5 additions & 5 deletions examples/cvode/ginkgo/cv_heat2D_ginkgo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ constexpr auto N_VNew = N_VNew_Cuda;
#include <nvector/nvector_hip.h>
#define HIP_OR_CUDA_OR_SYCL(a, b, c) a
constexpr auto N_VNew = N_VNew_Hip;
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#include <nvector/nvector_sycl.h>
#define HIP_OR_CUDA_OR_SYCL(a, b, c) c
constexpr auto N_VNew = N_VNew_Sycl;
Expand Down Expand Up @@ -122,7 +122,7 @@ int main(int argc, char* argv[])
#elif defined(USE_HIP)
auto gko_exec{gko::HipExecutor::create(0, gko::OmpExecutor::create(), false,
gko::allocation_mode::device)};
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
auto gko_exec{gko::DpcppExecutor::create(0, gko::ReferenceExecutor::create())};
#elif defined(USE_OMP)
auto gko_exec{gko::OmpExecutor::create()};
Expand All @@ -137,7 +137,7 @@ int main(int argc, char* argv[])
// ---------------

// Create solution vector
#if defined(USE_DPCPP)
#if defined(USE_SYCL)
N_Vector u = N_VNew(udata.nodes, gko_exec->get_queue(), sunctx);
#else
N_Vector u = N_VNew(udata.nodes, sunctx);
Expand Down Expand Up @@ -372,7 +372,7 @@ int f(sunrealtype t, N_Vector u, N_Vector f, void* user_data)

HIP_OR_CUDA_OR_SYCL(hipDeviceSynchronize(), cudaDeviceSynchronize(), );

#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
// Access device data arrays
sunrealtype* uarray = N_VGetDeviceArrayPointer(u);
if (check_ptr(uarray, "N_VGetDeviceArrayPointer")) return -1;
Expand Down Expand Up @@ -612,7 +612,7 @@ int J(sunrealtype t, N_Vector y, N_Vector fy, SUNMatrix J, void* user_data,
col_idxs, mat_data);

HIP_OR_CUDA_OR_SYCL(hipDeviceSynchronize(), cudaDeviceSynchronize(), );
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
auto queue =
std::dynamic_pointer_cast<const gko::DpcppExecutor>(udata->exec)->get_queue();
// J_sn_kernel
Expand Down
6 changes: 3 additions & 3 deletions examples/cvode/ginkgo/cv_heat2D_ginkgo.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
#include <nvector/nvector_cuda.h>
#elif defined(USE_HIP)
#include <nvector/nvector_hip.h>
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#include <nvector/nvector_sycl.h>
#endif

Expand Down Expand Up @@ -152,7 +152,7 @@ static int Solution(sunrealtype t, N_Vector u, UserData& udata)

solution_kernel<<<num_blocks, threads_per_block>>>(nx, ny, dx, dy, cos_sqr_t,
uarray);
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
sunrealtype* uarray = N_VGetDeviceArrayPointer(u);
if (check_ptr(uarray, "N_VGetDeviceArrayPointer")) return -1;
std::dynamic_pointer_cast<const gko::DpcppExecutor>(udata.exec)
Expand Down Expand Up @@ -366,7 +366,7 @@ static int WriteOutput(sunrealtype t, N_Vector u, N_Vector e, UserData& udata)
#elif defined(USE_HIP)
N_VCopyFromDevice_Hip(u);
N_VCopyFromDevice_Hip(e);
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
N_VCopyFromDevice_Sycl(u);
N_VCopyFromDevice_Sycl(e);
#endif
Expand Down
4 changes: 2 additions & 2 deletions examples/sunlinsol/ginkgo/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ include_directories(..)

sundials_add_examples_ginkgo(examples
TARGETS test_sunlinsol_obj
BACKENDS REF OMP CUDA HIP DPCPP
BACKENDS REF OMP CUDA HIP SYCL
UNIT_TEST)

# Install the targets
Expand All @@ -44,7 +44,7 @@ if(EXAMPLES_INSTALL)
if(SUNDIALS_GINKGO_BACKENDS MATCHES "HIP")
list(APPEND vectors nvechip)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "DPCPP")
if(SUNDIALS_GINKGO_BACKENDS MATCHES "SYCL")
list(APPEND vectors nvecsycl)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "OMP")
Expand Down
10 changes: 5 additions & 5 deletions examples/sunlinsol/ginkgo/test_sunlinsol_ginkgo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ constexpr auto N_VNew = N_VNew_Hip;
#include <nvector/nvector_cuda.h>
#define HIP_OR_CUDA_OR_SYCL(a, b, c) b
constexpr auto N_VNew = N_VNew_Cuda;
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#include <nvector/nvector_sycl.h>
#define HIP_OR_CUDA_OR_SYCL(a, b, c) c
constexpr auto N_VNew = N_VNew_Sycl;
Expand Down Expand Up @@ -163,7 +163,7 @@ static void fill_matrix(
fill_kernel<<<num_blocks, threads_per_block>>>(mat_rows, mat_cols, row_ptrs,
col_idxs, mat_data);
HIP_OR_CUDA_OR_SYCL(hipDeviceSynchronize(), cudaDeviceSynchronize(), );
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
std::dynamic_pointer_cast<const gko::DpcppExecutor>(matrix->get_executor())
->get_queue()
->submit(
Expand Down Expand Up @@ -250,7 +250,7 @@ static void fill_matrix(std::shared_ptr<gko::matrix::Dense<sunrealtype>> matrix)

fill_kernel<<<num_blocks, threads_per_block>>>(mat_rows, mat_cols, mat_data);
HIP_OR_CUDA_OR_SYCL(hipDeviceSynchronize(), cudaDeviceSynchronize(), );
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
std::dynamic_pointer_cast<const gko::DpcppExecutor>(matrix->get_executor())
->get_queue()
->submit(
Expand Down Expand Up @@ -355,7 +355,7 @@ int main(int argc, char* argv[])
#elif defined(USE_CUDA)
auto gko_exec{gko::CudaExecutor::create(0, gko::OmpExecutor::create(), false,
gko::allocation_mode::device)};
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
auto gko_exec{gko::DpcppExecutor::create(0, gko::ReferenceExecutor::create())};
#elif defined(USE_OMP)
auto gko_exec{gko::OmpExecutor::create()};
Expand Down Expand Up @@ -443,7 +443,7 @@ int main(int argc, char* argv[])
* Create solution and RHS vectors *
* ------------------------------- */

#if defined(USE_DPCPP)
#if defined(USE_SYCL)
N_Vector x{N_VNew(matcols, gko_exec->get_queue(), sunctx)};
#else
N_Vector x{N_VNew(matcols, sunctx)};
Expand Down
4 changes: 2 additions & 2 deletions examples/sunmatrix/ginkgo/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ include_directories(..)

sundials_add_examples_ginkgo(examples
TARGETS test_sunmatrix_obj sundials_sunmatrixdense
BACKENDS REF OMP CUDA HIP DPCPP
BACKENDS REF OMP CUDA HIP SYCL
UNIT_TEST)

# Install the targets
Expand All @@ -40,7 +40,7 @@ if(EXAMPLES_INSTALL)
if(SUNDIALS_GINKGO_BACKENDS MATCHES "HIP")
list(APPEND vectors nvechip)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "DPCPP")
if(SUNDIALS_GINKGO_BACKENDS MATCHES "SYCL")
list(APPEND vectors nvecsycl)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "OMP")
Expand Down
4 changes: 2 additions & 2 deletions examples/sunmatrix/ginkgo/test_sunmatrix_ginkgo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
#define REF_OR_OMP_OR_HIP_OR_CUDA_OR_SYCL(a, b, c, d, e) c
#elif defined(USE_CUDA)
#define REF_OR_OMP_OR_HIP_OR_CUDA_OR_SYCL(a, b, c, d, e) d
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#define REF_OR_OMP_OR_HIP_OR_CUDA_OR_SYCL(a, b, c, d, e) e
#elif defined(USE_OMP)
#define REF_OR_OMP_OR_HIP_OR_CUDA_OR_SYCL(a, b, c, d, e) b
Expand All @@ -42,7 +42,7 @@
#include <nvector/nvector_hip.h>
#elif defined(USE_OMP)
#include <nvector/nvector_openmp.h>
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#include <nvector/nvector_sycl.h>
#else
#include <nvector/nvector_serial.h>
Expand Down
2 changes: 1 addition & 1 deletion include/sundials/sundials_config.in
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@
#cmakedefine SUNDIALS_GINKGO_BACKENDS_HIP
#cmakedefine SUNDIALS_GINKGO_BACKENDS_OMP
#cmakedefine SUNDIALS_GINKGO_BACKENDS_REF
#cmakedefine SUNDIALS_GINKGO_BACKENDS_DPCPP
#cmakedefine SUNDIALS_GINKGO_BACKENDS_SYCL

/* MAGMA backends */
#cmakedefine SUNDIALS_MAGMA_BACKENDS_CUDA
Expand Down

0 comments on commit 8cdbbec

Please sign in to comment.