Skip to content

Commit

Permalink
Merge Release 1.6.0
Browse files Browse the repository at this point in the history
Release 1.6.0 of Ginkgo.

The Ginkgo team is proud to announce the new Ginkgo minor release 1.6.0. This release brings new features such as:
- Several building blocks for GPU-resident sparse direct solvers like symbolic
  and numerical LU and Cholesky factorization, ...,
- A distributed Schwarz preconditioner,
- New FGMRES and GCR solvers,
- Distributed benchmarks for the SpMV operation, solvers, ...
- Support for non-default streams in the CUDA and HIP backends,
- Mixed precision support for the CSR SpMV,
- A new profiling logger which integrates with NVTX, ROCTX, TAU and VTune to
  provide internal Ginkgo knowledge to most HPC profilers!

and much more.

If you face an issue, please first check our [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues) and the [open issues list](https://github.com/ginkgo-project/ginkgo/issues) and if you do not find a solution, feel free to [open a new issue](https://github.com/ginkgo-project/ginkgo/issues/new/choose) or ask a question using the [github discussions](https://github.com/ginkgo-project/ginkgo/discussions).

Supported systems and requirements:
+ For all platforms, CMake 3.13+
+ C++14 compliant compiler
+ Linux and macOS
  + GCC: 5.5+
  + clang: 3.9+
  + Intel compiler: 2018+
  + Apple Clang: 14.0 is tested. Earlier versions might also work.
  + NVHPC: 22.7+
  + Cray Compiler: 14.0.1+
  + CUDA module: CUDA 9.2+ or NVHPC 22.7+
  + HIP module: ROCm 4.5+
  + DPC++ module: Intel OneAPI 2021.3+ with oneMKL and oneDPL. Set the CXX compiler to `dpcpp`.
+ Windows
  + MinGW: GCC 5.5+
  + Microsoft Visual Studio: VS 2019+
  + CUDA module: CUDA 9.2+, Microsoft Visual Studio
  + OpenMP module: MinGW.

### Version Support Changes
+ ROCm 4.0+ -> 4.5+ after [#1303](#1303)
+ Removed Cygwin pipeline and support [#1283](#1283)

### Interface Changes
+ Due to internal changes, `ConcreteExecutor::run` will now always throw if the corresponding module for the `ConcreteExecutor` is not build [#1234](#1234)
+ The constructor of `experimental::distributed::Vector` was changed to only accept local vectors as `std::unique_ptr` [#1284](#1284)
+ The default parameters for the `solver::MultiGrid` were improved. In particular, the smoother defaults to one iteration of `Ir` with `Jacobi` preconditioner, and the coarse grid solver uses the new direct solver with LU factorization. [#1291](#1291) [#1327](#1327)
+ The `iteration_complete` event gained a more expressive overload with additional parameters, the old overloads were deprecated. [#1288](#1288) [#1327](#1327)

### Deprecations
+ Deprecated less expressive `iteration_complete` event. Users are advised to now implement the function `void iteration_complete(const LinOp* solver, const LinOp* b, const LinOp* x, const size_type& it, const LinOp* r, const LinOp* tau, const LinOp* implicit_tau_sq, const array<stopping_status>* status, bool stopped)` [#1288](#1288)

### Added Features
+ A distributed Schwarz preconditioner. [#1248](#1248)
+ A GCR solver [#1239](#1239)
+ Flexible Gmres solver [#1244](#1244)
+ Enable Gmres solver for distributed matrices and vectors [#1201](#1201)
+ An example that uses Kokkos to assemble the system matrix [#1216](#1216)
+ A symbolic LU factorization allowing the `gko::experimental::factorization::Lu` and `gko::experimental::solver::Direct` classes to be used for matrices with non-symmetric sparsity pattern [#1210](#1210)
+ A numerical Cholesky factorization [#1215](#1215)
+ Symbolic factorizations in host-side operations are now wrapped in a host-side `Operation` to make their execution visible to loggers. This means that profiling loggers and benchmarks are no longer missing a separate entry for their runtime [#1232](#1232)
+ Symbolic factorization benchmark [#1302](#1302)
+ The `ProfilerHook` logger allows annotating the Ginkgo execution (apply, operations, ...) for profiling frameworks like NVTX, ROCTX and TAU. [#1055](#1055)
+ `ProfilerHook::created_(nested_)summary` allows the generation of a lightweight runtime profile over all Ginkgo functions written to a user-defined stream [#1270](#1270) for both host and device timing functionality [#1313](#1313)
+ It is now possible to enable host buffers for MPI communications at runtime even if the compile option `GINKGO_FORCE_GPU_AWARE_MPI` is set. [#1228](#1228)
+ A stencil matrices generator (5-pt, 7-pt, 9-pt, and 27-pt) for benchmarks [#1204](#1204)
+ Distributed benchmarks (multi-vector blas, SpMV, solver) [#1204](#1204)
+ Benchmarks for CSR sorting and lookup [#1219](#1219)
+ A timer for MPI benchmarks that reports the longest time [#1217](#1217)
+ A `timer_method=min|max|average|median` flag for benchmark timing summary [#1294](#1294)
+ Support for non-default streams in CUDA and HIP executors [#1236](#1236)
+ METIS integration for nested dissection reordering [#1296](#1296)
+ SuiteSparse AMD integration for fillin-reducing reordering [#1328](#1328)
+ Csr mixed-precision SpMV support [#1319](#1319)
+ A `with_loggers` function for all `Factory` parameters [#1337](#1337)

### Improvements
+ Improve naming of kernel operations for loggers [#1277](#1277)
+ Annotate solver iterations in `ProfilerHook` [#1290](#1290)
+ Allow using the profiler hooks and inline input strings in benchmarks [#1342](#1342)
+ Allow passing smart pointers in place of raw pointers to most matrix functions. This means that things like `vec->compute_norm2(x.get())` or `vec->compute_norm2(lend(x))` can be simplified to `vec->compute_norm2(x)` [#1279](#1279) [#1261](#1261)
+ Catch overflows in prefix sum operations, which makes Ginkgo's operations much less likely to crash. This also improves the performance of the prefix sum kernel [#1303](#1303)
+ Make the installed GinkgoConfig.cmake file relocatable and follow more best practices [#1325](#1325)

### Fixes
+ Fix OpenMPI version check [#1200](#1200)
+ Fix the mpi cxx type binding by c binding [#1306](#1306)
+ Fix runtime failures for one-sided MPI wrapper functions observed on some OpenMPI versions [#1249](#1249)
+ Disable thread pinning with GPU executors due to poor performance [#1230](#1230)
+ Fix hwloc version detection [#1266](#1266)
+ Fix PAPI detection in non-implicit include directories [#1268](#1268)
+ Fix PAPI support for newer PAPI versions: [#1321](#1321)
+ Fix pkg-config file generation for library paths outside prefix [#1271](#1271)
+ Fix various build failures with ROCm 5.4, CUDA 12, and OneAPI 6 [#1214](#1214), [#1235](#1235), [#1251](#1251)
+ Fix incorrect read for skew-symmetric MatrixMarket files with explicit diagonal entries [#1272](#1272)
+ Fix handling of missing diagonal entries in symbolic factorizations [#1263](#1263)
+ Fix segmentation fault in benchmark matrix construction [#1299](#1299)
+ Fix the stencil matrix creation for benchmarking [#1305](#1305)
+ Fix the additional residual check in IR [#1307](#1307)
+ Fix the cuSPARSE CSR SpMM issue on single strided vector when cuda >= 11.6 [#1322](#1322) [#1331](#1331)
+ Fix Isai generation for large sparsity powers [#1327](#1327)
+ Fix Ginkgo compilation and test with NVHPC >= 22.7 [#1331](#1331)
+ Fix Ginkgo compilation of 32 bit binaries with MSVC [#1349](#1349)
  • Loading branch information
tcojean authored Jun 16, 2023
2 parents 234594c + be2115d commit 1f1ed46
Show file tree
Hide file tree
Showing 1,296 changed files with 291,986 additions and 17,325 deletions.
2 changes: 1 addition & 1 deletion .github/abidiff.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ popd

bot_delete_comments_matching "Note: This PR changes the Ginkgo ABI"

abidiff build-old/lib/libginkgod.so build-new/lib/libginkgod.so &> abi.diff || (bot_comment "Note: This PR changes the Ginkgo ABI:\n\`\`\`\n$(head -n2 abi.diff | tr '\n' ';' | sed 's/;/\\n/g')\`\`\`\nFor details check the full ABI diff under **Artifacts** [here]($JOB_URL)")
abidiff build-old/lib/libginkgod.so build-new/lib/libginkgod.so &> abi.diff || true # (bot_comment "Note: This PR changes the Ginkgo ABI:\n\`\`\`\n$(head -n2 abi.diff | tr '\n' ';' | sed 's/;/\\n/g')\`\`\`\nFor details check the full ABI diff under **Artifacts** [here]($JOB_URL)")
4 changes: 3 additions & 1 deletion .github/bot-pr-base.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ source .github/bot-base.sh
EXTENSION_REGEX='\.(cuh?|hpp|hpp\.inc?|cpp)$'
FORMAT_HEADER_REGEX='^(benchmark|core|cuda|hip|include/ginkgo/core|omp|reference|dpcpp|common/unified|test)/'
FORMAT_REGEX='^(common|examples)/'
CLANG_FORMAT=clang-format-9
CLANG_FORMAT=clang-format-14

echo -n "Collecting information on triggering PR"
PR_URL=$(jq -r .pull_request.url "$GITHUB_EVENT_PATH")
Expand All @@ -30,6 +30,8 @@ PR_JSON=$(api_get $PR_URL)
echo -n .
PR_MERGED=$(echo "$PR_JSON" | jq -r .merged)
echo -n .
PR_NUMBER=$(echo "$PR_JSON" | jq -r .number)
echo -n .
ISSUE_URL=$(echo "$PR_JSON" | jq -er ".issue_url")
echo -n .
BASE_REPO=$(echo "$PR_JSON" | jq -er .base.repo.full_name)
Expand Down
25 changes: 25 additions & 0 deletions .github/check-wiki-changelog.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env bash

source .github/bot-pr-base.sh

echo "Checking whether the PR contains the 1:ST:no-changelog-entry label"
SKIP_CHECK=$(api_get "$PR_URL" | jq -r 'any( [.labels | .[] | .name ] | .[] ; . == "1:ST:no-changelog-entry")')

if [[ "$SKIP_CHECK" == "true" ]]; then
echo "The PR contains the 1:ST:no-changelog-entry label. Skipping the wiki check."
elif [[ "$HEAD_BRANCH" =~ ^release-.* ]]; then
echo "The PR branch name starts with release. Skipping the wiki check."
else
curl https://raw.githubusercontent.com/wiki/ginkgo-project/ginkgo/Changelog.md > Changelog.md
PR="\[\#${PR_NUMBER}\]\(https://github.com/ginkgo-project/ginkgo/(issues|pull)/${PR_NUMBER}\)"
HAS_WIKI="$(cat Changelog.md | grep -E ${PR} || echo false)"
if [[ "${HAS_WIKI}" == "false" ]]; then
echo "This PR does not create the corresponding entry in wiki/Changelog"
echo "Please add [#${PR_NUMBER}](https://github.com/ginkgo-project/ginkgo/pull/${PR_NUMBER})"
echo "to the changelog at https://github.com/ginkgo-project/ginkgo/wiki/Changelog"
echo "Alternatively, use the label 1:ST:no-changelog-entry to skip this check."
exit 1
else
echo "wiki/Changelog already has this PR entry."
fi
fi
20 changes: 10 additions & 10 deletions .github/workflows/bot-pr-comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
if: github.event.issue.pull_request != '' && github.event.comment.body == 'label!' && (github.event.comment.author_association == 'COLLABORATOR' || github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'OWNER')
steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: develop
- name: Add appropriate labels
Expand All @@ -17,11 +17,11 @@ jobs:
run: cp --preserve .github/label.sh /tmp && /tmp/label.sh
check_format:
name: check-format
runs-on: ubuntu-18.04
runs-on: ubuntu-22.04
if: github.event.issue.pull_request != '' && github.event.comment.body == 'check-format!' && (github.event.comment.author_association == 'COLLABORATOR' || github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'OWNER')
steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: develop
- name: Check for formatting changes
Expand All @@ -30,17 +30,17 @@ jobs:
run: cp --preserve .github/check-format.sh /tmp && /tmp/check-format.sh
- name: Upload code formatting patch
if: failure()
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: patch
path: format.patch
format:
name: format
runs-on: ubuntu-18.04
runs-on: ubuntu-22.04
if: github.event.issue.pull_request != '' && github.event.comment.body == 'format!' && (github.event.comment.author_association == 'COLLABORATOR' || github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'OWNER')
steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: develop
persist-credentials: false
Expand All @@ -54,7 +54,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout the latest code
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: develop
fetch-depth: 0
Expand All @@ -66,10 +66,10 @@ jobs:
format-rebase:
name: format-rebase
if: github.event.issue.pull_request != '' && github.event.comment.body == 'format-rebase!' && (github.event.comment.author_association == 'COLLABORATOR' || github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'OWNER')
runs-on: ubuntu-18.04
runs-on: ubuntu-22.04
steps:
- name: Checkout the latest code
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: develop
fetch-depth: 0
Expand All @@ -80,7 +80,7 @@ jobs:
run: cp --preserve .github/format-rebase.sh /tmp && /tmp/format-rebase.sh
- name: Upload code formatting diffs
if: success()
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: patch
path: diff.patch
2 changes: 1 addition & 1 deletion .github/workflows/bot-pr-created.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
if: github.event.pull_request.author_association == 'COLLABORATOR' || github.event.pull_request.author_association == 'MEMBER' || github.event.pull_request.author_association == 'OWNER'
steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: develop
- name: Add appropriate labels
Expand Down
24 changes: 18 additions & 6 deletions .github/workflows/bot-pr-updated.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ concurrency:
cancel-in-progress: true
jobs:
check-format:
runs-on: ubuntu-18.04
runs-on: ubuntu-22.04
if: github.event.pull_request.author_association == 'COLLABORATOR' || github.event.pull_request.author_association == 'MEMBER' || github.event.pull_request.author_association == 'OWNER'
steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: develop
- name: Check for formatting changes
Expand All @@ -20,7 +20,7 @@ jobs:
run: cp .github/check-format.sh /tmp && /tmp/check-format.sh
- name: Upload code formatting patch
if: failure()
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: patch
path: format.patch
Expand All @@ -31,12 +31,12 @@ jobs:
CMAKE_FLAGS: -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=DEBUG -DGINKGO_BUILD_TESTS=OFF -DGINKGO_BUILD_EXAMPLES=OFF -DGINKGO_BUILD_BENCHMARKS=OFF -DGINKGO_BUILD_HWLOC=OFF -DGINKGO_BUILD_REFERENCE=OFF -DGINKGO_BUILD_OMP=OFF -DGINKGO_BUILD_CUDA=OFF -DGINKGO_BUILD_HIP=OFF -DGINKGO_BUILD_DPCPP=OFF
steps:
- name: Checkout the new code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
path: new
ref: ${{ github.event.pull_request.head.ref }}
- name: Checkout the old code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
path: old
ref: ${{ github.event.pull_request.base.ref }}
Expand All @@ -56,7 +56,19 @@ jobs:
run: old/.github/abidiff.sh
- name: Upload ABI diff
if: failure()
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: abi
path: abi.diff
check-wiki-changelog:
runs-on: ubuntu-latest
if: github.event.pull_request.author_association == 'COLLABORATOR' || github.event.pull_request.author_association == 'MEMBER' || github.event.pull_request.author_association == 'OWNER'
steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v3
with:
ref: develop
- name: Check if PR number exists in wiki/Changelog
env:
GITHUB_TOKEN: ${{ secrets.BOT_TOKEN }}
run: .github/check-wiki-changelog.sh
50 changes: 50 additions & 0 deletions .github/workflows/intel.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Intel-build

on:
push:
branches:
- 'master'
- 'develop'
- 'release/**'
tags:
- '**'
pull_request:
types: [opened,synchronize]

concurrency:
group: ${{ github.workflow }}-${{ (github.head_ref && github.ref) || github.run_id }}
cancel-in-progress: true

jobs:
intel:
strategy:
fail-fast: false
matrix:
config:
- {build_type: "Release", name: "intel/release/shared", "mixed": "ON"}
name: ${{ matrix.config.name }}
runs-on: [gpu_intel]

steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v3

- name: configure
run: |
spack load intel-oneapi-compilers intel-oneapi-dpl intel-oneapi-mkl intel-oneapi-tbb cmake
spack find --loaded
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=install_ginkgo -DCMAKE_CXX_COMPILER=dpcpp -DCMAKE_BUILD_TYPE=${{ matrix.config.build_type }} -DGINKGO_MIXED_PRECISION=${{ matrix.config.mixed }} -DGINKGO_DPCPP_SINGLE_MODE=ON
make -j8
SYCL_DEVICE_FILTER=level_zero ctest -j10 --output-on-failure
- name: install
run: |
spack load intel-oneapi-compilers intel-oneapi-dpl intel-oneapi-mkl intel-oneapi-tbb cmake
cd build
SYCL_DEVICE_FILTER=level_zero make install
export GINKGO_PATH="$(pwd)/install_ginkgo/lib"
export LIBRARY_PATH=${GINKGO_PATH}:$LIBRARY_PATH
export LD_LIBRARY_PATH=${GINKGO_PATH}:$LD_LIBRARY_PATH
SYCL_DEVICE_FILTER=level_zero make test_install
2 changes: 1 addition & 1 deletion .github/workflows/joss.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:

steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3

- name: setup
run: sudo apt-get install texlive-xetex pandoc pandoc-citeproc
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/mirror.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: develop
fetch-depth: 0
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/osx.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:

steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3

- name: setup
run: |
Expand Down
66 changes: 0 additions & 66 deletions .github/workflows/windows-cygwin.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/windows-mingw.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
runs-on: [windows-latest]
steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3

- name: Debug over SSH (tmate)
uses: mxschmitt/[email protected]
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/windows-msvc-cuda.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:

steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3
- name: setup (versioned)
if: matrix.config.version != 'latest'
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/windows-msvc-ref.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
runs-on: [windows-latest]
steps:
- name: Checkout the latest code (shallow clone)
uses: actions/checkout@v2
uses: actions/checkout@v3

- name: Debug over SSH (tmate)
uses: mxschmitt/[email protected]
Expand Down
Loading

0 comments on commit 1f1ed46

Please sign in to comment.