Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintenance/rename ginkgo dpcpp to sycl #493

Merged
merged 6 commits into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ Updated the CMake variable `HIP_PLATFORM` default to `amd` as the previous
default, `hcc`, is no longer recognized in ROCm 5.7.0 or newer. The new default
is also valid in older version of ROCm (at least back to version 4.3.1).

Renamed the DPCPP value for the `SUNDIALS_GINKGO_BACKENDS` CMake option to `SYCL`
to match Ginkgo's updated naming convention.

Changed the CMake version compatibility mode for SUNDIALS to `AnyNewerVersion`
instead of `SameMajorVersion`. This fixes the issue seen
[here](https://github.com/AMReX-Codes/amrex/pull/3835).
Expand Down
2 changes: 1 addition & 1 deletion cmake/SundialsTPLOptions.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ sundials_option(ENABLE_GINKGO BOOL "Enable Ginkgo support" OFF)
sundials_option(Ginkgo_DIR PATH "Path to the root of a Ginkgo installation" "${Ginkgo_DIR}"
DEPENDS_ON ENABLE_GINKGO)

sundials_option(SUNDIALS_GINKGO_BACKENDS STRING "Which Ginkgo backend(s) to build the SUNDIALS Ginkgo interfaces for (REF, OMP, CUDA, HIP, DPCPP)" "REF;OMP"
sundials_option(SUNDIALS_GINKGO_BACKENDS STRING "Which Ginkgo backend(s) to build the SUNDIALS Ginkgo interfaces for (REF, OMP, CUDA, HIP, SYCL)" "REF;OMP"
DEPENDS_ON ENABLE_GINKGO)

sundials_option(GINKGO_WORKS BOOL "Set to ON to force CMake to accept a given Ginkgo configuration" OFF
Expand Down
2 changes: 1 addition & 1 deletion cmake/macros/SundialsAddExamplesGinkgo.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ macro(sundials_add_examples_ginkgo EXAMPLES_VAR)
elseif(backend MATCHES "HIP")
set_source_files_properties(${example} PROPERTIES LANGUAGE CXX)
set(vector nvechip)
elseif(backend MATCHES "DPCPP")
elseif(backend MATCHES "SYCL")
set(vector nvecsycl)
elseif(backend MATCHES "OMP")
set(vector nvecopenmp)
Expand Down
4 changes: 2 additions & 2 deletions cmake/tpl/SundialsGinkgo.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ if(Ginkgo_FOUND AND (NOT GINKGO_WORKS))
print_error("SUNDIALS_GINKGO_BACKENDS includes HIP but HIP is not enabled. Set ENABLE_HIP=ON or change the backend.")
endif()

if(SUNDIALS_GINKGO_BACKENDS MATCHES "DPCPP" AND NOT ENABLE_SYCL)
print_error("SUNDIALS_GINKGO_BACKENDS includes DPC++ but SYCL/DPC++ is not enabled. Set ENABLE_SYCL=ON or change the backend.")
if(SUNDIALS_GINKGO_BACKENDS MATCHES "SYCL" AND NOT ENABLE_SYCL)
print_error("SUNDIALS_GINKGO_BACKENDS includes SYCL but SYCL is not enabled. Set ENABLE_SYCL=ON or change the backend.")
endif()

if(SUNDIALS_GINKGO_BACKENDS MATCHES "OMP" AND NOT ENABLE_OPENMP)
Expand Down
5 changes: 4 additions & 1 deletion doc/shared/RecentChanges.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ Updated the CMake variable ``HIP_PLATFORM`` default to ``amd`` as the previous
default, ``hcc``, is no longer recognized in ROCm 5.7.0 or newer. The new
default is also valid in older version of ROCm (at least back to version 4.3.1).

Renamed the DPCPP value for the :cmakeop:`SUNDIALS_GINKGO_BACKENDS` CMake option
to ``SYCL`` to match Ginkgo's updated naming convention.

Changed the CMake version compatibility mode for SUNDIALS to ``AnyNewerVersion``
instead of ``SameMajorVersion``. This fixes the issue seen `here
<https://github.com/AMReX-Codes/amrex/pull/3835>`_.
Expand Down Expand Up @@ -112,4 +115,4 @@ ARKODE-wide functions.

Deprecated the `ARKStepSetOptimalParams` function. Since this function does not have an
ARKODE-wide equivalent, instructions have been added to the user guide for how
to retain the current functionality using other user-callable functions.
to retain the current functionality using other user-callable functions.
14 changes: 9 additions & 5 deletions doc/shared/sundials/Install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -665,10 +665,14 @@ illustration only.
.. cmakeoption:: SUNDIALS_GINKGO_BACKENDS

Semi-colon separated list of Ginkgo target architecutres/executors to build for.
Options currenty supported are REF (the Ginkgo reference executor), OMP, CUDA, HIP, and DPC++.
Options currenty supported are REF (the Ginkgo reference executor), OMP, CUDA, HIP, and SYCL.

Default: "REF;OMP"
balos1 marked this conversation as resolved.
Show resolved Hide resolved

.. versionchanged: x.y.z

The ``DPCPP`` option was changed to ``SYCL`` to align with Ginkgo's naming convention.

.. cmakeoption:: ENABLE_KOKKOS

Enable the Kokkos based vector.
Expand Down Expand Up @@ -1480,13 +1484,13 @@ SUNDIALS has been tested with MAGMA version v2.6.1 and v2.6.2.

.. _Installation.CMake.ExternalLibraries.OneMKL:

Building with oneMKL
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Building with oneMKL for SYCL
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The Intel `oneAPI Math Kernel Library (oneMKL)
<https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl.html>`_
includes CPU and DPC++ interfaces for LAPACK dense linear algebra routines. The
SUNDIALS oneMKL interface targets the DPC++ routines, to utilize the CPU routine
includes CPU and SYCL/DPC++ interfaces for LAPACK dense linear algebra routines. The
SUNDIALS oneMKL interface targets the SYCL/DPC++ routines, to utilize the CPU routine
see :numref:`Installation.CMake.ExternalLibraries.LAPACK`.

To enable the SUNDIALS oneMKL interface set ``ENABLE_ONEMKL`` to ``ON`` and
Expand Down
4 changes: 2 additions & 2 deletions examples/cvode/ginkgo/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ set(cpu_gpu_examples

sundials_add_examples_ginkgo(cpu_gpu_examples
TARGETS sundials_cvode
BACKENDS REF OMP CUDA HIP DPCPP)
BACKENDS REF OMP CUDA HIP SYCL)

# Examples that only support CPU Ginkgo backends
set(cpu_examples
Expand All @@ -39,7 +39,7 @@ if(EXAMPLES_INSTALL)
if(SUNDIALS_GINKGO_BACKENDS MATCHES "HIP")
list(APPEND vectors nvechip)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "DPCPP")
if(SUNDIALS_GINKGO_BACKENDS MATCHES "SYCL")
list(APPEND vectors nvecsycl)
endif()
if((SUNDIALS_GINKGO_BACKENDS MATCHES "OMP") OR
Expand Down
10 changes: 5 additions & 5 deletions examples/cvode/ginkgo/cv_heat2D_ginkgo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ constexpr auto N_VNew = N_VNew_Cuda;
#include <nvector/nvector_hip.h>
#define HIP_OR_CUDA_OR_SYCL(a, b, c) a
constexpr auto N_VNew = N_VNew_Hip;
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#include <nvector/nvector_sycl.h>
#define HIP_OR_CUDA_OR_SYCL(a, b, c) c
constexpr auto N_VNew = N_VNew_Sycl;
Expand Down Expand Up @@ -122,7 +122,7 @@ int main(int argc, char* argv[])
#elif defined(USE_HIP)
auto gko_exec{gko::HipExecutor::create(0, gko::OmpExecutor::create(), false,
gko::allocation_mode::device)};
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
auto gko_exec{gko::DpcppExecutor::create(0, gko::ReferenceExecutor::create())};
#elif defined(USE_OMP)
auto gko_exec{gko::OmpExecutor::create()};
Expand All @@ -137,7 +137,7 @@ int main(int argc, char* argv[])
// ---------------

// Create solution vector
#if defined(USE_DPCPP)
#if defined(USE_SYCL)
N_Vector u = N_VNew(udata.nodes, gko_exec->get_queue(), sunctx);
#else
N_Vector u = N_VNew(udata.nodes, sunctx);
Expand Down Expand Up @@ -372,7 +372,7 @@ int f(sunrealtype t, N_Vector u, N_Vector f, void* user_data)

HIP_OR_CUDA_OR_SYCL(hipDeviceSynchronize(), cudaDeviceSynchronize(), );

#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
// Access device data arrays
sunrealtype* uarray = N_VGetDeviceArrayPointer(u);
if (check_ptr(uarray, "N_VGetDeviceArrayPointer")) return -1;
Expand Down Expand Up @@ -612,7 +612,7 @@ int J(sunrealtype t, N_Vector y, N_Vector fy, SUNMatrix J, void* user_data,
col_idxs, mat_data);

HIP_OR_CUDA_OR_SYCL(hipDeviceSynchronize(), cudaDeviceSynchronize(), );
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
auto queue =
std::dynamic_pointer_cast<const gko::DpcppExecutor>(udata->exec)->get_queue();
// J_sn_kernel
Expand Down
6 changes: 3 additions & 3 deletions examples/cvode/ginkgo/cv_heat2D_ginkgo.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
#include <nvector/nvector_cuda.h>
#elif defined(USE_HIP)
#include <nvector/nvector_hip.h>
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#include <nvector/nvector_sycl.h>
#endif

Expand Down Expand Up @@ -152,7 +152,7 @@ static int Solution(sunrealtype t, N_Vector u, UserData& udata)

solution_kernel<<<num_blocks, threads_per_block>>>(nx, ny, dx, dy, cos_sqr_t,
uarray);
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
sunrealtype* uarray = N_VGetDeviceArrayPointer(u);
if (check_ptr(uarray, "N_VGetDeviceArrayPointer")) return -1;
std::dynamic_pointer_cast<const gko::DpcppExecutor>(udata.exec)
Expand Down Expand Up @@ -366,7 +366,7 @@ static int WriteOutput(sunrealtype t, N_Vector u, N_Vector e, UserData& udata)
#elif defined(USE_HIP)
N_VCopyFromDevice_Hip(u);
N_VCopyFromDevice_Hip(e);
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
N_VCopyFromDevice_Sycl(u);
N_VCopyFromDevice_Sycl(e);
#endif
Expand Down
4 changes: 2 additions & 2 deletions examples/sunlinsol/ginkgo/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ include_directories(..)

sundials_add_examples_ginkgo(examples
TARGETS test_sunlinsol_obj
BACKENDS REF OMP CUDA HIP DPCPP
BACKENDS REF OMP CUDA HIP SYCL
UNIT_TEST)

# Install the targets
Expand All @@ -44,7 +44,7 @@ if(EXAMPLES_INSTALL)
if(SUNDIALS_GINKGO_BACKENDS MATCHES "HIP")
list(APPEND vectors nvechip)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "DPCPP")
if(SUNDIALS_GINKGO_BACKENDS MATCHES "SYCL")
list(APPEND vectors nvecsycl)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "OMP")
Expand Down
10 changes: 5 additions & 5 deletions examples/sunlinsol/ginkgo/test_sunlinsol_ginkgo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ constexpr auto N_VNew = N_VNew_Hip;
#include <nvector/nvector_cuda.h>
#define HIP_OR_CUDA_OR_SYCL(a, b, c) b
constexpr auto N_VNew = N_VNew_Cuda;
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#include <nvector/nvector_sycl.h>
#define HIP_OR_CUDA_OR_SYCL(a, b, c) c
constexpr auto N_VNew = N_VNew_Sycl;
Expand Down Expand Up @@ -163,7 +163,7 @@ static void fill_matrix(
fill_kernel<<<num_blocks, threads_per_block>>>(mat_rows, mat_cols, row_ptrs,
col_idxs, mat_data);
HIP_OR_CUDA_OR_SYCL(hipDeviceSynchronize(), cudaDeviceSynchronize(), );
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
std::dynamic_pointer_cast<const gko::DpcppExecutor>(matrix->get_executor())
->get_queue()
->submit(
Expand Down Expand Up @@ -250,7 +250,7 @@ static void fill_matrix(std::shared_ptr<gko::matrix::Dense<sunrealtype>> matrix)

fill_kernel<<<num_blocks, threads_per_block>>>(mat_rows, mat_cols, mat_data);
HIP_OR_CUDA_OR_SYCL(hipDeviceSynchronize(), cudaDeviceSynchronize(), );
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
std::dynamic_pointer_cast<const gko::DpcppExecutor>(matrix->get_executor())
->get_queue()
->submit(
Expand Down Expand Up @@ -355,7 +355,7 @@ int main(int argc, char* argv[])
#elif defined(USE_CUDA)
auto gko_exec{gko::CudaExecutor::create(0, gko::OmpExecutor::create(), false,
gko::allocation_mode::device)};
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
auto gko_exec{gko::DpcppExecutor::create(0, gko::ReferenceExecutor::create())};
#elif defined(USE_OMP)
auto gko_exec{gko::OmpExecutor::create()};
Expand Down Expand Up @@ -443,7 +443,7 @@ int main(int argc, char* argv[])
* Create solution and RHS vectors *
* ------------------------------- */

#if defined(USE_DPCPP)
#if defined(USE_SYCL)
N_Vector x{N_VNew(matcols, gko_exec->get_queue(), sunctx)};
#else
N_Vector x{N_VNew(matcols, sunctx)};
Expand Down
4 changes: 2 additions & 2 deletions examples/sunmatrix/ginkgo/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ include_directories(..)

sundials_add_examples_ginkgo(examples
TARGETS test_sunmatrix_obj sundials_sunmatrixdense
BACKENDS REF OMP CUDA HIP DPCPP
BACKENDS REF OMP CUDA HIP SYCL
UNIT_TEST)

# Install the targets
Expand All @@ -40,7 +40,7 @@ if(EXAMPLES_INSTALL)
if(SUNDIALS_GINKGO_BACKENDS MATCHES "HIP")
list(APPEND vectors nvechip)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "DPCPP")
if(SUNDIALS_GINKGO_BACKENDS MATCHES "SYCL")
list(APPEND vectors nvecsycl)
endif()
if(SUNDIALS_GINKGO_BACKENDS MATCHES "OMP")
Expand Down
4 changes: 2 additions & 2 deletions examples/sunmatrix/ginkgo/test_sunmatrix_ginkgo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
#define REF_OR_OMP_OR_HIP_OR_CUDA_OR_SYCL(a, b, c, d, e) c
#elif defined(USE_CUDA)
#define REF_OR_OMP_OR_HIP_OR_CUDA_OR_SYCL(a, b, c, d, e) d
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#define REF_OR_OMP_OR_HIP_OR_CUDA_OR_SYCL(a, b, c, d, e) e
#elif defined(USE_OMP)
#define REF_OR_OMP_OR_HIP_OR_CUDA_OR_SYCL(a, b, c, d, e) b
Expand All @@ -42,7 +42,7 @@
#include <nvector/nvector_hip.h>
#elif defined(USE_OMP)
#include <nvector/nvector_openmp.h>
#elif defined(USE_DPCPP)
#elif defined(USE_SYCL)
#include <nvector/nvector_sycl.h>
#else
#include <nvector/nvector_serial.h>
Expand Down
2 changes: 1 addition & 1 deletion include/sundials/sundials_config.in
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@
#cmakedefine SUNDIALS_GINKGO_BACKENDS_HIP
#cmakedefine SUNDIALS_GINKGO_BACKENDS_OMP
#cmakedefine SUNDIALS_GINKGO_BACKENDS_REF
#cmakedefine SUNDIALS_GINKGO_BACKENDS_DPCPP
#cmakedefine SUNDIALS_GINKGO_BACKENDS_SYCL

/* MAGMA backends */
#cmakedefine SUNDIALS_MAGMA_BACKENDS_CUDA
Expand Down
Loading