From 7340427f601a531f6e6e123596db6c0917c11138 Mon Sep 17 00:00:00 2001 From: Marcel Koch Date: Fri, 14 Jun 2024 16:44:05 +0000 Subject: [PATCH] [doc] add build/install section --- .../user-guide/build-install.cmake-options.md | 121 ++++++++++++++++++ doc/sphinx/user-guide/build-install.hip.md | 42 ++++++ doc/sphinx/user-guide/build-install.md | 98 ++++++++++++++ .../build-install.system-requirements.md | 78 +++++++++++ doc/sphinx/user-guide/build-install.tpl.md | 41 ++++++ .../user-guide/build-install.windows.md | 11 ++ 6 files changed, 391 insertions(+) create mode 100644 doc/sphinx/user-guide/build-install.cmake-options.md create mode 100644 doc/sphinx/user-guide/build-install.hip.md create mode 100644 doc/sphinx/user-guide/build-install.md create mode 100644 doc/sphinx/user-guide/build-install.system-requirements.md create mode 100644 doc/sphinx/user-guide/build-install.tpl.md create mode 100644 doc/sphinx/user-guide/build-install.windows.md diff --git a/doc/sphinx/user-guide/build-install.cmake-options.md b/doc/sphinx/user-guide/build-install.cmake-options.md new file mode 100644 index 00000000000..1dc8c027cda --- /dev/null +++ b/doc/sphinx/user-guide/build-install.cmake-options.md @@ -0,0 +1,121 @@ +# CMake Option List + +The following discusses CMake options that are relevant to building Ginkgo. +They are used by passing them to the configuration step by: + +```shell +cmake -S <...> -B <...> -DOPTION1=VAL1 -DOPTION2=VAL2 ... +``` + +## Ginkgo-Specific Options + +All Ginkgo specific options are prefixed with `GINKGO_`. + +* `-DGINKGO_DEVEL_TOOLS={ON, OFF}` sets up the build system for development + (requires pre-commit, will also download the clang-format pre-commit hook), + default is `OFF`. The default behavior installs a pre-commit hook, which + disables git commits. If it is set to `ON`, a new pre-commit hook for + formatting will be installed (enabling commits again). In both cases the + hook may overwrite a user defined pre-commit hook when Ginkgo is used as + a submodule. +* `-DGINKGO_MIXED_PRECISION={ON, OFF}` compiles true mixed-precision kernels + instead of converting data on the fly, default is `OFF`. + Enabling this flag increases the library size, but improves performance of + mixed-precision kernels. +* `-DGINKGO_BUILD_TESTS={ON, OFF}` builds Ginkgo's tests + (will download googletest), default is `ON`. +* `-DGINKGO_FAST_TESTS={ON, OFF}` reduces the input sizes for a few slow tests + to speed them up, default is `OFF`. +* `-DGINKGO_BUILD_BENCHMARKS={ON, OFF}` builds Ginkgo's benchmarks + (will download gflags and nlohmann-json), default is `ON`. +* `-DGINKGO_BUILD_EXAMPLES={ON, OFF}` builds Ginkgo's examples, default is `ON` +* `-DGINKGO_BUILD_EXTLIB_EXAMPLE={ON, OFF}` builds the interfacing example + with deal.II, default is `OFF`. +* `-DGINKGO_BUILD_REFERENCE={ON, OFF}` build reference implementations of the + kernels, useful for testing, default is `ON` +* `-DGINKGO_BUILD_OMP={ON, OFF}` builds optimized OpenMP versions of the kernels, + default is `ON` if the selected C++ compiler supports OpenMP, `OFF` otherwise. +* `-DGINKGO_BUILD_CUDA={ON, OFF}` builds optimized cuda versions of the kernels + (requires CUDA), default is `ON` if a CUDA compiler could be detected, + `OFF` otherwise. +* `-DGINKGO_BUILD_DPCPP={ON, OFF}` is deprecated. Please use `GINKGO_BUILD_SYCL` instead. +* `-DGINKGO_BUILD_SYCL={ON, OFF}` builds optimized SYCL versions of the + kernels (requires `CMAKE_CXX_COMPILER` to be set to the `dpcpp` or `icpx` compiler). + The default is `ON` if `CMAKE_CXX_COMPILER` is a SYCL compiler, `OFF` + otherwise. Due to some differences in IEEE 754 floating point numberhandling in the Intel + SYCL compilers, Ginkgo tests may fail unless compiled with + `-DCMAKE_CXX_FLAGS=-ffp-model=precise` +* `-DGINKGO_BUILD_HIP={ON, OFF}` builds optimized HIP versions of the kernels + (requires HIP), default is `ON` if an installation of HIP could be detected, + `OFF` otherwise. +* `-DGINKGO_BUILD_HWLOC={ON, OFF}` builds Ginkgo with HWLOC. Default is `OFF`. +* `-DGINKGO_BUILD_DOC={ON, OFF}` creates an HTML version of Ginkgo's documentation + from inline comments in the code. The default is `OFF`. +* `-DGINKGO_DOC_GENERATE_EXAMPLES={ON, OFF}` generates the documentation of examples + in Ginkgo. The default is `ON`. +* `-DGINKGO_DOC_GENERATE_PDF={ON, OFF}` generates a PDF version of Ginkgo's + documentation from inline comments in the code. The default is `OFF`. +* `-DGINKGO_DOC_GENERATE_DEV={ON, OFF}` generates the developer version of + Ginkgo's documentation. The default is `OFF`. +* `-DGINKGO_WITH_CLANG_TIDY={ON, OFF}` makes Ginkgo call `clang-tidy` to find + programming issues. The path can be manually controlled with the CMake + variable `-DGINKGO_CLANG_TIDY_PATH=`. The default is `OFF`. +* `-DGINKGO_WITH_IWYU={ON, OFF}` makes Ginkgo call `iwyu` to find include + issues. The path can be manually controlled with the CMake variable + `-DGINKGO_IWYU_PATH=`. The default is `OFF`. +* `-DGINKGO_CHECK_CIRCULAR_DEPS={ON, OFF}` enables compile-time checks for + circular dependencies between different Ginkgo libraries and self-sufficient + headers. Should only be used for development purposes. The default is `OFF`. +* `-DGINKGO_VERBOSE_LEVEL={0, 1}` sets the verbosity of Ginkgo. + * `0` disables all output in the main libraries, + * `1` enables a few important messages related to unexpected behavior (default). +* `GINKGO_INSTALL_RPATH` allows setting any RPATH information when installing + the Ginkgo libraries. If this is `OFF`, the behavior is the same as if all + other RPATH flags are set to `OFF` as well. The default is `ON`. +* `GINKGO_INSTALL_RPATH_ORIGIN` adds $ORIGIN (Linux) or @loader_path (MacOS) + to the installation RPATH. The default is `ON`. +* `GINKGO_INSTALL_RPATH_DEPENDENCIES` adds the dependencies to the + installation RPATH. The default is `OFF`. +* `-DGINKGO_JACOBI_FULL_OPTIMIZATIONS={ON, OFF}` use all the optimizations + for the CUDA Jacobi algorithm. `OFF` by default. Setting this option to `ON` + may lead to very slow compile time (>20 minutes) for the + `jacobi_generate_kernels.cu` file and high memory usage. +* `-DGINKGO_CUDA_ARCHITECTURES=` where `` is a semicolon (`;`) separated + list of architectures. Supported values are: + + * `Auto` + * `Kepler`, `Maxwell`, `Pascal`, `Volta`, `Turing`, `Ampere` + + `Auto` will automatically detect the present CUDA-enabled GPU architectures + in the system. `Kepler`, `Maxwell`, `Pascal`, `Volta` and `Ampere` will add flags for + all architectures of that particular NVIDIA GPU generation. It is advised to use + the non-Ginkgo option `CMAKE_CUDA_ARCHITECTURES` as described below instead. + + +## Important Non-Ginkgo-Specific Options + +These are options that as defined by CMake. +Only a selection of the most relevant options is given here, the full list is available +[here](https://cmake.org/cmake/help/latest/manual/cmake-variables.7.html). + +* `-DCMAKE_BUILD_TYPE=type` specifies which configuration will be used for + this build of Ginkgo. The default is `RELEASE`. Supported values are CMake's + standard build types such as `DEBUG` and `RELEASE` and the Ginkgo specific + `COVERAGE`, `ASAN` (AddressSanitizer), `LSAN` (LeakSanitizer), `TSAN` + (ThreadSanitizer) and `UBSAN` (undefined behavior sanitizer) types. [link](https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html#variable:CMAKE_BUILD_TYPE) +* `-DCMAKE_CXX_COMPILER=path`, `-DCMAKE_CUDA_COMPILER=path`, `-DCMAKE_HIP_COMPILER=path` set + the compiler for the respective language. [link](https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER.html) +* `-DCMAKE__HOST_COMPILER=path` instructs the build system to explicitly + set 's (either CUDA or HIP) host compiler to the path given as argument. By default, + uses its toolchain's host compiler. Setting this option may help if you're + experiencing linking errors due to ABI incompatibilities. [link](https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_HOST_COMPILER.html#variable:CMAKE_%3CLANG%3E_HOST_COMPILER) +* `-DCMAKE_CUDA_ARCHITECTURES="gpuarch1;gpuarch2"` the NVIDIA targets to be passed to the compiler. + If empty, compiler chooses based on the available GPUs. [link](https://cmake.org/cmake/help/latest/variable/CMAKE_CUDA_ARCHITECTURES.html#cmake-cuda-architectures) +* `-DCMAKE_HIP_ARCHITECTURES="gpuarch1;gpuarch2"` the AMDGPU targets to be passed to the compiler. + If empty, compiler chooses based on the available GPUs. [link](https://cmake.org/cmake/help/latest/variable/CMAKE_HIP_ARCHITECTURES.html#cmake-hip-architectures) +* `-DCMAKE_INSTALL_PREFIX=path` sets the installation path for `make install`. + The default value is usually something like `/usr/local`. [link](https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html#variable:CMAKE_INSTALL_PREFIX) +* `-DBUILD_SHARED_LIBS={ON, OFF}` builds ginkgo as shared libraries (`OFF`) + or as dynamic libraries (`ON`), default is `ON`. [link](https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html#variable:BUILD_SHARED_LIBS) +* `-DCMAKE_EXPORT_PACKAGE_REGISTRY={ON,OFF}` if set to `ON` the build directory will + be stored in the current user's CMake package registry. [link](https://cmake.org/cmake/help/latest/variable/CMAKE_EXPORT_PACKAGE_REGISTRY.html#cmake-export-package-registry) diff --git a/doc/sphinx/user-guide/build-install.hip.md b/doc/sphinx/user-guide/build-install.hip.md new file mode 100644 index 00000000000..9bf2f42cb7a --- /dev/null +++ b/doc/sphinx/user-guide/build-install.hip.md @@ -0,0 +1,42 @@ +# Building Ginkgo with HIP support + +Ginkgo provides a [HIP](https://github.com/ROCm-Developer-Tools/HIP) backend. +This allows to compile optimized versions of the kernels for either AMD or +NVIDIA GPUs. The CMake configuration step will try to auto-detect the presence +of HIP either at `/opt/rocm/hip` or at the path specified by `HIP_PATH` as a +CMake parameter (`-DHIP_PATH=`) or environment variable (`export HIP_PATH=`), +unless `-DGINKGO_BUILD_HIP=ON/OFF` is set explicitly. + +## Changing the paths to search for HIP and other packages +All HIP installation paths can be configured through the use of environment +variables or CMake variables. This way of configuring the paths is currently +imposed by the `HIP` tool suite. The variables are the following: ++ CMake `-DROCM_PATH=` or environment `export ROCM_PATH=`: sets the `ROCM` + installation path. The default value is `/opt/rocm/`. ++ CMake `-DHIP_CLANG_PATH` or environment `export HIP_CLANG_PATH=`: sets the + `HIP` compatible `clang` binary path. The default value is + `${ROCM_PATH}/llvm/bin`. ++ CMake `-DHIP_PATH=` or environment `export HIP_PATH=`: sets the `HIP` + installation path. The default value is `${ROCM_PATH}/hip`. ++ CMake `-DHIPBLAS_PATH=` or environment `export HIPBLAS_PATH=`: sets the + `hipBLAS` installation path. The default value is `${ROCM_PATH}/hipblas`. ++ CMake `-DHIPSPARSE_PATH=` or environment `export HIPSPARSE_PATH=`: sets the + `hipSPARSE` installation path. The default value is `${ROCM_PATH}/hipsparse`. ++ CMake `-DHIPFFT_PATH=` or environment `export HIPFFT_PATH=`: sets the + `hipFFT` installation path. The default value is `${ROCM_PATH}/hipfft`. ++ CMake `-DROCRAND_PATH=` or environment `export ROCRAND_PATH=`: sets the + `rocRAND` installation path. The default value is `${ROCM_PATH}/rocrand`. ++ CMake `-DHIPRAND_PATH=` or environment `export HIPRAND_PATH=`: sets the + `hipRAND` installation path. The default value is `${ROCM_PATH}/hiprand`. ++ environment `export CUDA_PATH=`: where `hipcc` can find `CUDA` if it is not in + the default `/usr/local/cuda` path. + + +## HIP platform detection of AMD and NVIDIA +Ginkgo relies on CMake to decide which compiler to use for HIP. +To choose `nvcc` instead of the default ROCm `clang++`, set the corresponding +environment variable: +```bash +export HIPCXX=nvcc +``` +Note that this option is currently not being tested in our CI pipelines. diff --git a/doc/sphinx/user-guide/build-install.md b/doc/sphinx/user-guide/build-install.md new file mode 100644 index 00000000000..8d24613cfee --- /dev/null +++ b/doc/sphinx/user-guide/build-install.md @@ -0,0 +1,98 @@ +# Configuring, Building, and Installing + +Ginkgo follows the standard CMake procedure. +A default configuration of Ginkgo can be configured, build, and installed with the following three commands: + +```shell +cmake -S -B [options] +cmake --build +cmake --install +``` + +Ginkgo requires both CMake 3.16+ and a C++17 compiler to be available. +Each backend might have additional requirements. +A full list of the requirements can be found in [](build-install.system-requirements.md). + +## Selecting Backends + +The default configuration will try to detect and enable all available backends. +Each backend can also be explicitly enabled via the following CMake options: + +- `GINKGO_BUILD_REFERENCE`: enable the reference backend +- `GINKGO_BUILD_OMP`: enable the OpenMP backend +- `GINKGO_BUILD_CUDA`: enable the CUDA backend +- `GINKGO_BUILD_HIP`: enable the HIP backend +- `GINKGO_BUILD_SYCL`: enable the SYCL backend + +:::{note} +It is valid to enable multiple backends, even GPU ones, at the same time, +as long as the system supports those backends. +::: + + +Using these, Ginkgo can be told to explicitly enable the CUDA backend and explicitly disable the OpenMP backend +with the following command: + +```shell +cmake -DGINKGO_BUILD_CUDA=ON -DGINKGO_BUILD_HIP=OFF -S -B +``` + +If any backend is explicitly enabled, the required compilers, libraries, etc. must be available on the system. +In the above example, CMake must be able to find a CUDA capable compiler. +This can be helped by setting the CMake option `-DCMAKE_CUDA_COMPILER=/path/to/compiler`. +The same fix can be applied to the HIP backend. +CMake will return an error, if the requirements for enabling a backend are not satisfied. + + +:::{attention} +If using the SYCL backend, the `CMAKE_CXX_COMPILER` option must be set to a SYCL compatible compiler. +::: + +## Configuration Options + +Ginkgo has more options than the backend selection to configure its build. +The full list of options is available in [](build-install.cmake-options.md) + + +## Linking Ginkgo + +After installing, Ginkgo may be linked against by using: + +```cmake +find_package(Ginkgo VERSION 1.9.0 REQUIRED) + +target_link_libraries(target Ginkgo::ginkgo) +``` + +If Ginkgo can't be found by CMake, either add the install directory to the CMake option `CMAKE_PREFIX_PATH`, +or passing `Ginkgo_ROOT=` to the CMake configuration of the consumer library. +The default install directory is system dependent. +To explicitly set this directory either provide a prefix path to the install step: + +```shell +cmake --install --prefix +``` + +Alternatively, the same path might be set during the configuration step: + +```shell +cmake -DCMAKE_INSTALL_PREFIX= -S -B +``` + +## Advanced Topics + +Details on more advanced topics can be found here: + +- [](build-install.tpl.md) +- [](build-install.hip.md) +- [](build-install.windows.md) + +```{toctree} +:hidden: + +build-install.system-requirements +build-install.cmake-options +build-install.tpl +build-install.hip +build-install.windows +``` diff --git a/doc/sphinx/user-guide/build-install.system-requirements.md b/doc/sphinx/user-guide/build-install.system-requirements.md new file mode 100644 index 00000000000..f55c39641ba --- /dev/null +++ b/doc/sphinx/user-guide/build-install.system-requirements.md @@ -0,0 +1,78 @@ +# System Requirements + +## Linux and Mac OS + +For Ginkgo core library: + +* _cmake 3.16+_ +* C++14 compliant compiler, one of: + * _gcc 5.5+_ + * _clang 3.9+_ + * _Intel compiler 2019+_ + * _Apple Clang 14.0_ is tested. Earlier versions might also work. + * _Cray Compiler 14.0.1+_ + * _NVHPC Compiler 22.7+_ + +The Ginkgo CUDA module has the following __additional__ requirements: + +* _cmake 3.18+_ (If CUDA was installed through the NVIDIA HPC Toolkit, we require _cmake 3.22+_) +* _CUDA 10.1+_ or _NVHPC Package 22.7+_ +* Any host compiler restrictions your version of CUDA may impose also apply + here. For the newest CUDA version, this information can be found in the + [CUDA installation guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html) + or [CUDA installation guide for Mac Os X](https://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html) + +The Ginkgo HIP module has the following __additional__ requirements: + +* _ROCm 4.5+_ +* the HIP, hipBLAS, hipSPARSE, hip/rocRAND and rocThrust packages compiled with either: + * _AMD_ backend (using the `clang` compiler) + * _10.1 <= CUDA < 11_ backend +* if the hipFFT package is available, it is used to implement the FFT LinOps. +* _cmake 3.21+_ + +The Ginkgo DPC++(SYCL) module has the following __additional__ requirements: + +* _oneAPI 2023.1+_ +* Set `dpcpp` or `icpx` as the `CMAKE_CXX_COMPILER` +* `c++17` is used to compile Ginkgo +* The following oneAPI packages should be available: + * oneMKL + * oneDPL + +The Ginkgo MPI module has the following __additional__ requirements: + +* MPI 3.1+, ideally GPU-Aware, for best performance + +In addition, if you want to contribute code to Ginkgo, you will also need the +following: + +* _clang-format 8.0.0+_ (ships as part of _clang_) +* _clang-tidy_ (optional, when setting the flag `-DGINKGO_WITH_CLANG_TIDY=ON`) +* _iwyu_ (Include What You Use, optional, when setting the flag `-DGINKGO_WITH_IWYU=ON`) + +## Windows + +* _cmake 3.16+_ +* C++14 compliant 64-bit compiler: + * _MinGW : gcc 5.5+_ + * _Microsoft Visual Studio : VS 2019+_ + +The Ginkgo CUDA module has the following __additional__ requirements: + +* _CUDA 10.1+_ +* _Microsoft Visual Studio_ +* Any host compiler restrictions your version of CUDA may impose also apply + here. For the newest CUDA version, this information can be found in the + [CUDA installation guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) + +The Ginkgo OMP module has the following __additional__ requirements: +* _MinGW_ + +In these environments, two problems can be encountered, the solution for which is described in the +[windows section in INSTALL.md](INSTALL.md#building-ginkgo-in-windows): +* `ld: error: export ordinal too large` needs the compilation flag `-O1` +* `cc1plus.exe: out of memory allocating 65536 bytes` requires a modification of the environment + +__NOTE:__ Some restrictions will also apply on the version of C and C++ standard +libraries installed on the system. This needs further investigation. diff --git a/doc/sphinx/user-guide/build-install.tpl.md b/doc/sphinx/user-guide/build-install.tpl.md new file mode 100644 index 00000000000..e8d21443433 --- /dev/null +++ b/doc/sphinx/user-guide/build-install.tpl.md @@ -0,0 +1,41 @@ +# Third Party Libraries and Packages + +Ginkgo relies on third party packages in different cases. These third party +packages can be turned off by disabling the relevant options. + ++ `-DGINKGO_BUILD_TESTS=ON`: Our tests are implemented with [Google + Test](https://github.com/google/googletest); ++ `-DGINKGO_BUILD_BENCHMARKS=ON`: For argument management we use + [gflags](https://github.com/gflags/gflags) and for JSON parsing we use + [nlohmann-json](https://github.com/nlohmann/json); ++ `-DGINKGO_BUILD_HWLOC=ON`: + [hwloc](https://www.open-mpi.org/projects/hwloc) to detect and control cores + and devices. ++ `-DGINKGO_BUILD_HWLOC=ON` and `-DGINKGO_BUILD_TESTS=ON`: + [libnuma](https://www.man7.org/linux/man-pages/man3/numa.3.html) is required + when testing the functions provided through MachineTopology. ++ `-DGINKGO_BUILD_EXAMPLES=ON`: + [OpenCV](https://opencv.org/) is required for some examples, they are disabled when OpenCV is not available. ++ `-DGINKGO_BUILD_DOC=ON`: + [doxygen](https://www.doxygen.nl/) is required to build the documentation and + additionally [graphviz](https://graphviz.org/) is required to build the class hierarchy graphs. ++ [METIS](http://glaros.dtc.umn.edu/gkhome/metis/metis/overview) is required + when using the `NestedDissection` reordering functionality. + If METIS is not found, the functionality is disabled. ++ [PAPI](https://icl.utk.edu/papi/) (>= 7.1.0) is required when using the `Papi` logger. + If PAPI is not found, the functionality is disabled. + +Ginkgo attempts to use pre-installed versions of these package if they match +version requirements using `find_package`. Otherwise, the configuration step +will download the files for each of the packages `GTest`, `gflags`, and +`nlohmann-json` and build them internally. + +Note that, if the external packages were not installed to the default location, +the CMake option `-DCMAKE_PREFIX_PATH=` needs to be set to the +semicolon (`;`) separated list of install paths of these external packages. For +more Information, see the [CMake documentation for +CMAKE_PREFIX_PATH](https://cmake.org/cmake/help/v3.9/variable/CMAKE_PREFIX_PATH.html) +for details. + +For convenience, the options `GINKGO_INSTALL_RPATH[_.*]` can be used +to bind the installed Ginkgo shared libraries to the path of its dependencies. diff --git a/doc/sphinx/user-guide/build-install.windows.md b/doc/sphinx/user-guide/build-install.windows.md new file mode 100644 index 00000000000..36701922f17 --- /dev/null +++ b/doc/sphinx/user-guide/build-install.windows.md @@ -0,0 +1,11 @@ +# Building Ginkgo in Windows + +Depending on the configuration settings, some manual work might be required: +* Build Ginkgo with Debug mode: + Some Debug build specific issues can appear depending on the machine and environment: + When you encounter the error message `ld: error: export ordinal too large`, add the compilation flag `-O1` + by adding `-DCMAKE_CXX_FLAGS=-O1` to the CMake invocation. +* Build Ginkgo in _MinGW_:\ + If encountering the issue `cc1plus.exe: out of memory allocating 65536 bytes`, please follow the workaround in + [reference](https://www.intel.com/content/www/us/en/programmable/support/support-resources/knowledge-base/embedded/2016/cc1plus-exe--out-of-memory-allocating-65536-bytes.html), + or trying to compile ginkgo again might work.