Enables building Partition from local ranges #1227

MarcelKoch · 2022-12-05T12:29:00Z

This PR adds a freestanding function to create a Partition from local ranges. This has to be a freestanding function, because adding it to Partition would require including MPI, which leads to our device libraries requiring MPI (or at least the headers).

upsj

The main thing that comes to mind is the issue of correctness: The current model doesn't allow feedback on whether the ranges are actually consecutive, right? If they aren't, we just pick a handful of values from the range array, which may or may not be in ascending order. What you could do is check via run_kernel_reduce over a boolean whether the input was consecutive with little overhead, or do the check on the host side after a host-side allgather.

include/ginkgo/core/distributed/partition_helpers.hpp

core/distributed/partition.cpp

test/mpi/distributed/partition_helpers.cpp

include/ginkgo/core/distributed/partition_helpers.hpp

core/distributed/partition_helpers.cpp

greole

Some quick comments.

One additional thing, this would be a good oportunity to mark Partition<LocalIndexType, GlobalIndexType>::has_connected_parts() and Partition<LocalIndexType, GlobalIndexType>::has_ordered_parts() as const

common/cuda_hip/distributed/partition_helpers_kernels.hpp.inc

greole · 2023-01-11T08:01:26Z

include/ginkgo/core/distributed/partition_helpers.hpp

+ * @param local_range the start and end indices of the local range.
+ * @param comm  the communicator used to determine the global partition.


Suggested change

* @param local_range the start and end indices of the local range.

* @param comm the communicator used to determine the global partition.

* @param comm the communicator used to determine the global partition.

* @param local_range the start and end indices of the local range.

To reflect the order of the arguments.

greole · 2023-01-11T08:03:14Z

include/ginkgo/core/distributed/partition_helpers.hpp

+ * @param local_range the number of the locally owned indices
+ * @param comm  the communicator used to determine the global partition.


Suggested change

* @param local_range the number of the locally owned indices

* @param comm the communicator used to determine the global partition.

* @param comm the communicator used to determine the global partition.

* @param local_range the number of the locally owned indices

core/distributed/partition_helpers.cpp

include/ginkgo/core/distributed/partition.hpp

upsj

First look, still need to go over the tests

upsj · 2023-02-08T15:43:26Z

core/distributed/partition_helpers_kernels.hpp

+
+#define GKO_DECLARE_PARTITION_HELPERS_CHECK_CONSECUTIVE_RANGES(_type)          \
+    void check_consecutive_ranges(std::shared_ptr<const DefaultExecutor> exec, \
+                                  array<_type>& range_start_ends,              \


Suggested change

array<_type>& range_start_ends, \

const array<_type>& range_start_ends, \

upsj · 2023-02-08T15:44:43Z

common/cuda_hip/distributed/partition_helpers_kernels.hpp.inc

+    auto starts = thrust::device_pointer_cast(range_start_ends.get_data());
+    auto ends = starts + num_ranges;
+    auto zip_it = thrust::make_zip_iterator(thrust::make_tuple(
+        ends, thrust::device_pointer_cast(part_ids.get_data())));
+    thrust::sort_by_key(thrust::device, starts, starts + num_ranges, zip_it);


So the data is laid out as | starts | ends |, and not interleaved? Maybe note that somewhere

Also related to #1236

nit: remove device_pointer_cast

Suggested change

auto starts = thrust::device_pointer_cast(range_start_ends.get_data());

auto ends = starts + num_ranges;

auto zip_it = thrust::make_zip_iterator(thrust::make_tuple(

ends, thrust::device_pointer_cast(part_ids.get_data())));

thrust::sort_by_key(thrust::device, starts, starts + num_ranges, zip_it);

auto starts = range_start_ends.get_data();

auto ends = starts + num_ranges;

auto zip_it = thrust::make_zip_iterator(thrust::make_tuple(ends, part_ids.get_data()));

thrust::sort_by_key(thrust::device, starts, starts + num_ranges, zip_it);

upsj · 2023-02-08T15:46:02Z

common/unified/distributed/partition_helpers_kernels.cpp

+
+template <typename GlobalIndexType>
+void check_consecutive_ranges(std::shared_ptr<const DefaultExecutor> exec,
+                              array<GlobalIndexType>& range_start_ends,


Suggested change

array<GlobalIndexType>& range_start_ends,

const array<GlobalIndexType>& range_start_ends,

upsj · 2023-02-08T15:48:00Z

common/unified/distributed/partition_kernels.cpp

+        ranges.get_num_elems() - 1, ranges, part_id_mapping, range_bounds,
+        part_ids, part_id_mapping.get_num_elems() > 0);


just an observation: Maybe we should ensure that empty arrays always point to nullptr, then we can use the pointer directly to check whether to use it

upsj · 2023-02-08T15:56:45Z

include/ginkgo/core/distributed/partition.hpp

-     *                Part i contains the indices [ranges[i], ranges[i + 1]).
-     *                Has to contain at least one element.
-     *                The first element has to be 0.
+     *                Part parti_id[i] contains the indices


typo

Suggested change

* Part parti_id[i] contains the indices

* Part part_id[i] contains the indices

upsj · 2023-02-08T16:00:04Z

core/distributed/partition_helpers.cpp

+    reqs.push_back(comm.i_all_gather(mpi_exec, &range[0], 1,
+                                     ranges_start_end.get_data(), 1));
+    reqs.push_back(comm.i_all_gather(
+        mpi_exec, &range[1], 1, ranges_start_end.get_data() + comm.size(), 1));


why this layout and not communicate the two entries together by interleaving?

I would like to have it interleaved, but then I would need iterators with stride=2, which I don't think thrust support. Or is there some way to do that with thrust?

Something like a counting_iterator + transform_iterator? That's my go-to solution for non-trivial access patterns

upsj · 2023-02-08T16:00:46Z

core/distributed/partition_helpers.cpp

+    exec->run(partition_helpers::make_check_consecutive_ranges(
+        ranges_start_end, &consecutive_ranges));
+    if (!consecutive_ranges) {
+        throw Error(__FILE__, __LINE__, "The partition contains gaps.");


We need a more specific error for invalid user data, I think

I agree, but I would rather put that into another PR.

I think that is not a lot of code and can be added in this PR. Just an additional class in exception.hpp and macro in exception_helpers.hpp and most of it boiler plate ?

upsj · 2023-02-08T16:01:57Z

core/distributed/partition_helpers.cpp

+        throw Error(__FILE__, __LINE__, "The partition contains gaps.");
+    }
+
+    // remove duplicates


To me, this description of what happens below could be improved, e.g.

Suggested change

// remove duplicates

// join (now consecutive) starts and ends into combined array

stale

- constness - documentation Co-authored-by: Tobias Ribizel <[email protected]>

- use stable sort Co-authored-by: Gregor Olenik <[email protected]>

Co-authored-by: Tobias Ribizel <[email protected]>

- removes more reinterpret casts - makes permute_iterator copy assignable Co-authored-by: Thomas Grützmacher <[email protected]> Co-authored-by: Tobias Ribizel <[email protected]>

- use gko::array instead of gko::vector Co-authored-by: Thomas Grützmacher <[email protected]>

this allows to rely on default implementation of the constructors and assignment operators in the `permute_iterator`. Except for the default constructor due to AppleClang issues Co-authored-by: Thomas Grützmacher <[email protected]> Co-authored-by: Tobias Ribizel <[email protected]>

Co-authored-by: Pratik Nayak <[email protected]> Co-authored-by: Tobias Ribizel <[email protected]>

- documentation - formatting Co-authored-by: Pratik Nayak <[email protected]>

Co-authored-by: Marcel Koch <[email protected]>

Co-authored-by: Tobias Ribizel <[email protected]>

Co-authored-by: Marcel Koch <[email protected]>

- initialize pointer - dereference pointer instead of array access - use `bool&` as return type instead of `bool*` Co-authored-by: Tobias Ribizel <[email protected]>

sonarcloud · 2023-08-17T17:06:51Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
22 Code Smells

98.9% Coverage
2.5% Duplication

Release 1.7.0 to master The Ginkgo team is proud to announce the new Ginkgo minor release 1.7.0. This release brings new features such as: - Complete GPU-resident sparse direct solvers feature set and interfaces, - Improved Cholesky factorization performance, - A new MC64 reordering, - Batched iterative solver support with the BiCGSTAB solver with batched Dense and ELL matrix types, - MPI support for the SYCL backend, - Improved ParILU(T)/ParIC(T) preconditioner convergence, and more! If you face an issue, please first check our [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues) and the [open issues list](https://github.com/ginkgo-project/ginkgo/issues) and if you do not find a solution, feel free to [open a new issue](https://github.com/ginkgo-project/ginkgo/issues/new/choose) or ask a question using the [github discussions](https://github.com/ginkgo-project/ginkgo/discussions). Supported systems and requirements: + For all platforms, CMake 3.16+ + C++14 compliant compiler + Linux and macOS + GCC: 5.5+ + clang: 3.9+ + Intel compiler: 2019+ + Apple Clang: 14.0 is tested. Earlier versions might also work. + NVHPC: 22.7+ + Cray Compiler: 14.0.1+ + CUDA module: CMake 3.18+, and CUDA 10.1+ or NVHPC 22.7+ + HIP module: ROCm 4.5+ + DPC++ module: Intel oneAPI 2022.1+ with oneMKL and oneDPL. Set the CXX compiler to `dpcpp` or `icpx`. + MPI: standard version 3.1+, ideally GPU Aware, for best performance + Windows + MinGW: GCC 5.5+ + Microsoft Visual Studio: VS 2019+ + CUDA module: CUDA 10.1+, Microsoft Visual Studio + OpenMP module: MinGW. ### Version support changes + CUDA 9.2 is no longer supported and 10.0 is untested [#1382](#1382) + Ginkgo now requires CMake version 3.16 (and 3.18 for CUDA) [#1368](#1368) ### Interface changes + `const` Factory parameters can no longer be modified through `with_*` functions, as this breaks const-correctness [#1336](#1336) [#1439](#1439) ### New Deprecations + The `device_reset` parameter of CUDA and HIP executors no longer has an effect, and its `allocation_mode` parameters have been deprecated in favor of the `Allocator` interface. [#1315](#1315) + The CMake parameter `GINKGO_BUILD_DPCPP` has been deprecated in favor of `GINKGO_BUILD_SYCL`. [#1350](#1350) + The `gko::reorder::Rcm` interface has been deprecated in favor of `gko::experimental::reorder::Rcm` based on `Permutation`. [#1418](#1418) + The Permutation class' `permute_mask` functionality. [#1415](#1415) + Multiple functions with typos (`set_complex_subpsace()`, range functions such as `conj_operaton` etc). [#1348](#1348) ### Summary of previous deprecations + `gko::lend()` is not necessary anymore. + The classes `RelativeResidualNorm` and `AbsoluteResidualNorm` are deprecated in favor of `ResidualNorm`. + The class `AmgxPgm` is deprecated in favor of `Pgm`. + Default constructors for the CSR `load_balance` and `automatical` strategies + The PolymorphicObject's move-semantic `copy_from` variant + The templated `SolverBase` class. + The class `MachineTopology` is deprecated in favor of `machine_topology`. + Logger constructors and create functions with the `executor` parameter. + The virtual, protected, Dense functions `compute_norm1_impl`, `add_scaled_impl`, etc. + Logger events for solvers and criterion without the additional `implicit_tau_sq` parameter. + The global `gko::solver::default_krylov_dim`, use instead `gko::solver::gmres_default_krylov_dim`. ### Added features + Adds a batch::BatchLinOp class that forms a base class for batched linear operators such as batched matrix formats, solver and preconditioners [#1379](#1379) + Adds a batch::MultiVector class that enables operations such as dot, norm, scale on batched vectors [#1371](#1371) + Adds a batch::Dense matrix format that stores batched dense matrices and provides gemv operations for these dense matrices. [#1413](#1413) + Adds a batch::Ell matrix format that stores batched Ell matrices and provides spmv operations for these batched Ell matrices. [#1416](#1416) [#1437](#1437) + Add a batch::Bicgstab solver (class, core, and reference kernels) that enables iterative solution of batched linear systems [#1438](#1438). + Add device kernels (CUDA, HIP, and DPCPP) for batch::Bicgstab solver. [#1443](#1443). + New MC64 reordering algorithm which optimizes the diagonal product or sum of a matrix by permuting the rows, and computes additional scaling factors for equilibriation [#1120](#1120) + New interface for (non-symmetric) permutation and scaled permutation of Dense and Csr matrices [#1415](#1415) + LU and Cholesky Factorizations can now be separated into their factors [#1432](#1432) + New symbolic LU factorization algorithm that is optimized for matrices with an almost-symmetric sparsity pattern [#1445](#1445) + Sorting kernels for SparsityCsr on all backends [#1343](#1343) + Allow passing pre-generated local solver as factory parameter for the distributed Schwarz preconditioner [#1426](#1426) + Add DPCPP kernels for Partition [#1034](#1034), and CSR's `check_diagonal_entries` and `add_scaled_identity` functionality [#1436](#1436) + Adds a helper function to create a partition based on either local sizes, or local ranges [#1227](#1227) + Add function to compute arithmetic mean of dense and distributed vectors [#1275](#1275) + Adds `icpx` compiler supports [#1350](#1350) + All backends can be built simultaneously [#1333](#1333) + Emits a CMake warning in downstream projects that use different compilers than the installed Ginkgo [#1372](#1372) + Reordering algorithms in sparse_blas benchmark [#1354](#1354) + Benchmarks gained an `-allocator` parameter to specify device allocators [#1385](#1385) + Benchmarks gained an `-input_matrix` parameter that initializes the input JSON based on the filename [#1387](#1387) + Benchmark inputs can now be reordered as a preprocessing step [#1408](#1408) ### Improvements + Significantly improve Cholesky factorization performance [#1366](#1366) + Improve parallel build performance [#1378](#1378) + Allow constrained parallel test execution using CTest resources [#1373](#1373) + Use arithmetic type more inside mixed precision ELL [#1414](#1414) + Most factory parameters of factory type no longer need to be constructed explicitly via `.on(exec)` [#1336](#1336) [#1439](#1439) + Improve ParILU(T)/ParIC(T) convergence by using more appropriate atomic operations [#1434](#1434) ### Fixes + Fix an over-allocation for OpenMP reductions [#1369](#1369) + Fix DPCPP's common-kernel reduction for empty input sizes [#1362](#1362) + Fix several typos in the API and documentation [#1348](#1348) + Fix inconsistent `Threads` between generations [#1388](#1388) + Fix benchmark median condition [#1398](#1398) + Fix HIP 5.6.0 compilation [#1411](#1411) + Fix missing destruction of rand_generator from cuda/hip [#1417](#1417) + Fix PAPI logger destruction order [#1419](#1419) + Fix TAU logger compilation [#1422](#1422) + Fix relative criterion to not iterate if the residual is already zero [#1079](#1079) + Fix memory_order invocations with C++20 changes [#1402](#1402) + Fix `check_diagonal_entries_exist` report correctly when only missing diagonal value in the last rows. [#1440](#1440) + Fix checking OpenMPI version in cross-compilation settings [#1446](#1446) + Fix false-positive deprecation warnings in Ginkgo, especially for the old Rcm (it doesn't emit deprecation warnings anymore as a result but is still considered deprecated) [#1444](#1444) ### Related PR: #1451

Release 1.7.0 to develop The Ginkgo team is proud to announce the new Ginkgo minor release 1.7.0. This release brings new features such as: - Complete GPU-resident sparse direct solvers feature set and interfaces, - Improved Cholesky factorization performance, - A new MC64 reordering, - Batched iterative solver support with the BiCGSTAB solver with batched Dense and ELL matrix types, - MPI support for the SYCL backend, - Improved ParILU(T)/ParIC(T) preconditioner convergence, and more! If you face an issue, please first check our [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues) and the [open issues list](https://github.com/ginkgo-project/ginkgo/issues) and if you do not find a solution, feel free to [open a new issue](https://github.com/ginkgo-project/ginkgo/issues/new/choose) or ask a question using the [github discussions](https://github.com/ginkgo-project/ginkgo/discussions). Supported systems and requirements: + For all platforms, CMake 3.16+ + C++14 compliant compiler + Linux and macOS + GCC: 5.5+ + clang: 3.9+ + Intel compiler: 2019+ + Apple Clang: 14.0 is tested. Earlier versions might also work. + NVHPC: 22.7+ + Cray Compiler: 14.0.1+ + CUDA module: CMake 3.18+, and CUDA 10.1+ or NVHPC 22.7+ + HIP module: ROCm 4.5+ + DPC++ module: Intel oneAPI 2022.1+ with oneMKL and oneDPL. Set the CXX compiler to `dpcpp` or `icpx`. + MPI: standard version 3.1+, ideally GPU Aware, for best performance + Windows + MinGW: GCC 5.5+ + Microsoft Visual Studio: VS 2019+ + CUDA module: CUDA 10.1+, Microsoft Visual Studio + OpenMP module: MinGW. ### Version support changes + CUDA 9.2 is no longer supported and 10.0 is untested [#1382](#1382) + Ginkgo now requires CMake version 3.16 (and 3.18 for CUDA) [#1368](#1368) ### Interface changes + `const` Factory parameters can no longer be modified through `with_*` functions, as this breaks const-correctness [#1336](#1336) [#1439](#1439) ### New Deprecations + The `device_reset` parameter of CUDA and HIP executors no longer has an effect, and its `allocation_mode` parameters have been deprecated in favor of the `Allocator` interface. [#1315](#1315) + The CMake parameter `GINKGO_BUILD_DPCPP` has been deprecated in favor of `GINKGO_BUILD_SYCL`. [#1350](#1350) + The `gko::reorder::Rcm` interface has been deprecated in favor of `gko::experimental::reorder::Rcm` based on `Permutation`. [#1418](#1418) + The Permutation class' `permute_mask` functionality. [#1415](#1415) + Multiple functions with typos (`set_complex_subpsace()`, range functions such as `conj_operaton` etc). [#1348](#1348) ### Summary of previous deprecations + `gko::lend()` is not necessary anymore. + The classes `RelativeResidualNorm` and `AbsoluteResidualNorm` are deprecated in favor of `ResidualNorm`. + The class `AmgxPgm` is deprecated in favor of `Pgm`. + Default constructors for the CSR `load_balance` and `automatical` strategies + The PolymorphicObject's move-semantic `copy_from` variant + The templated `SolverBase` class. + The class `MachineTopology` is deprecated in favor of `machine_topology`. + Logger constructors and create functions with the `executor` parameter. + The virtual, protected, Dense functions `compute_norm1_impl`, `add_scaled_impl`, etc. + Logger events for solvers and criterion without the additional `implicit_tau_sq` parameter. + The global `gko::solver::default_krylov_dim`, use instead `gko::solver::gmres_default_krylov_dim`. ### Added features + Adds a batch::BatchLinOp class that forms a base class for batched linear operators such as batched matrix formats, solver and preconditioners [#1379](#1379) + Adds a batch::MultiVector class that enables operations such as dot, norm, scale on batched vectors [#1371](#1371) + Adds a batch::Dense matrix format that stores batched dense matrices and provides gemv operations for these dense matrices. [#1413](#1413) + Adds a batch::Ell matrix format that stores batched Ell matrices and provides spmv operations for these batched Ell matrices. [#1416](#1416) [#1437](#1437) + Add a batch::Bicgstab solver (class, core, and reference kernels) that enables iterative solution of batched linear systems [#1438](#1438). + Add device kernels (CUDA, HIP, and DPCPP) for batch::Bicgstab solver. [#1443](#1443). + New MC64 reordering algorithm which optimizes the diagonal product or sum of a matrix by permuting the rows, and computes additional scaling factors for equilibriation [#1120](#1120) + New interface for (non-symmetric) permutation and scaled permutation of Dense and Csr matrices [#1415](#1415) + LU and Cholesky Factorizations can now be separated into their factors [#1432](#1432) + New symbolic LU factorization algorithm that is optimized for matrices with an almost-symmetric sparsity pattern [#1445](#1445) + Sorting kernels for SparsityCsr on all backends [#1343](#1343) + Allow passing pre-generated local solver as factory parameter for the distributed Schwarz preconditioner [#1426](#1426) + Add DPCPP kernels for Partition [#1034](#1034), and CSR's `check_diagonal_entries` and `add_scaled_identity` functionality [#1436](#1436) + Adds a helper function to create a partition based on either local sizes, or local ranges [#1227](#1227) + Add function to compute arithmetic mean of dense and distributed vectors [#1275](#1275) + Adds `icpx` compiler supports [#1350](#1350) + All backends can be built simultaneously [#1333](#1333) + Emits a CMake warning in downstream projects that use different compilers than the installed Ginkgo [#1372](#1372) + Reordering algorithms in sparse_blas benchmark [#1354](#1354) + Benchmarks gained an `-allocator` parameter to specify device allocators [#1385](#1385) + Benchmarks gained an `-input_matrix` parameter that initializes the input JSON based on the filename [#1387](#1387) + Benchmark inputs can now be reordered as a preprocessing step [#1408](#1408) ### Improvements + Significantly improve Cholesky factorization performance [#1366](#1366) + Improve parallel build performance [#1378](#1378) + Allow constrained parallel test execution using CTest resources [#1373](#1373) + Use arithmetic type more inside mixed precision ELL [#1414](#1414) + Most factory parameters of factory type no longer need to be constructed explicitly via `.on(exec)` [#1336](#1336) [#1439](#1439) + Improve ParILU(T)/ParIC(T) convergence by using more appropriate atomic operations [#1434](#1434) ### Fixes + Fix an over-allocation for OpenMP reductions [#1369](#1369) + Fix DPCPP's common-kernel reduction for empty input sizes [#1362](#1362) + Fix several typos in the API and documentation [#1348](#1348) + Fix inconsistent `Threads` between generations [#1388](#1388) + Fix benchmark median condition [#1398](#1398) + Fix HIP 5.6.0 compilation [#1411](#1411) + Fix missing destruction of rand_generator from cuda/hip [#1417](#1417) + Fix PAPI logger destruction order [#1419](#1419) + Fix TAU logger compilation [#1422](#1422) + Fix relative criterion to not iterate if the residual is already zero [#1079](#1079) + Fix memory_order invocations with C++20 changes [#1402](#1402) + Fix `check_diagonal_entries_exist` report correctly when only missing diagonal value in the last rows. [#1440](#1440) + Fix checking OpenMPI version in cross-compilation settings [#1446](#1446) + Fix false-positive deprecation warnings in Ginkgo, especially for the old Rcm (it doesn't emit deprecation warnings anymore as a result but is still considered deprecated) [#1444](#1444) ### Related PR: #1454

MarcelKoch added the 1:ST:ready-for-review This PR is ready for review label Dec 5, 2022

MarcelKoch self-assigned this Dec 5, 2022

ginkgo-bot added mod:core This is related to the core module. mod:cuda This is related to the CUDA module. mod:hip This is related to the HIP module. mod:reference This is related to the reference module. reg:build This is related to the build system. labels Dec 5, 2022

MarcelKoch requested a review from a team December 5, 2022 13:49

upsj previously requested changes Dec 7, 2022

View reviewed changes

MarcelKoch force-pushed the partition-from-local-range branch 6 times, most recently from 9416e79 to 6e6eb5d Compare December 20, 2022 12:45

greole reviewed Jan 11, 2023

View reviewed changes

MarcelKoch force-pushed the partition-from-local-range branch 2 times, most recently from b83eeb4 to cbb11c8 Compare January 12, 2023 09:04

MarcelKoch force-pushed the partition-from-local-range branch 2 times, most recently from 8e9aa2d to 4903f20 Compare January 27, 2023 09:11

pratikvn reviewed Jan 27, 2023

View reviewed changes

include/ginkgo/core/distributed/partition.hpp Outdated Show resolved Hide resolved

MarcelKoch force-pushed the partition-from-local-range branch from 4903f20 to c351f58 Compare January 30, 2023 08:18

upsj reviewed Feb 8, 2023

View reviewed changes

MarcelKoch requested review from upsj, greole and pratikvn February 10, 2023 09:30

greole mentioned this pull request Feb 16, 2023

Ogl rebase 1.5.0 #1273

Closed

MarcelKoch and others added 22 commits August 16, 2023 12:44

adapts sorting to changed ranges layout

457b7f0

adapts consecutive check to changed ranges layout

0d68caf

adds kernel to compress ranges

3752a5b

review updates:

4be22f9

- constness - documentation Co-authored-by: Tobias Ribizel <[email protected]>

review update:

9ef64f2

- use stable sort Co-authored-by: Gregor Olenik <[email protected]>

fixing dpcpp

29a5a8a

don't mix host and device buffers for MPI

f8d3a8c

adds permutation iterator

31ae609

Co-authored-by: Tobias Ribizel <[email protected]>

use permute iterator for STL algorithms

3b7d008

review updates:

e6e3958

- removes more reinterpret casts - makes permute_iterator copy assignable Co-authored-by: Thomas Grützmacher <[email protected]> Co-authored-by: Tobias Ribizel <[email protected]>

review updates:

8edb0e5

- use gko::array instead of gko::vector Co-authored-by: Thomas Grützmacher <[email protected]>

adds invalid state exception

348b067

Co-authored-by: Pratik Nayak <[email protected]> Co-authored-by: Tobias Ribizel <[email protected]>

review updates:

af51e52

- documentation - formatting Co-authored-by: Pratik Nayak <[email protected]>

Format files

5fb31ba

Co-authored-by: Marcel Koch <[email protected]>

uses placement-new for copy-assignable wrapper

0383f94

Co-authored-by: Tobias Ribizel <[email protected]>

Format files

e90d154

Co-authored-by: Marcel Koch <[email protected]>

remove undefined lambda default constructor

a8c9900

use workaround for old dpcpp version

e3308ff

fixup after rebase

8cdb658

review updates:

5285e41

- initialize pointer - dereference pointer instead of array access - use `bool&` as return type instead of `bool*` Co-authored-by: Tobias Ribizel <[email protected]>

use custom stream for thrust policy

09bfa7f

ginkgo-bot force-pushed the partition-from-local-range branch from 9e007b7 to 09bfa7f Compare August 16, 2023 12:44

correctly define permutation map

8393dca

MarcelKoch merged commit 437e3ba into develop Aug 17, 2023
11 of 12 checks passed

MarcelKoch deleted the partition-from-local-range branch August 17, 2023 15:28

tcojean mentioned this pull request Nov 6, 2023

Release 1.7.0 to master #1451

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enables building Partition from local ranges #1227

Enables building Partition from local ranges #1227

MarcelKoch commented Dec 5, 2022

upsj left a comment •

edited

Loading

greole left a comment

greole Jan 11, 2023

greole Jan 11, 2023

upsj left a comment

upsj Feb 8, 2023

upsj Feb 8, 2023 •

edited

Loading

upsj Feb 8, 2023

upsj Feb 8, 2023

upsj Feb 8, 2023

upsj Feb 8, 2023

MarcelKoch Feb 9, 2023

upsj Feb 9, 2023

upsj Feb 8, 2023

MarcelKoch Feb 10, 2023

pratikvn Apr 26, 2023

upsj Feb 8, 2023

sonarcloud bot commented Aug 17, 2023 •

edited

Loading

		* @param local_range the start and end indices of the local range.
		* @param comm the communicator used to determine the global partition.

		* @param local_range the number of the locally owned indices
		* @param comm the communicator used to determine the global partition.

	array<_type>& range_start_ends, \
	const array<_type>& range_start_ends, \

	array<GlobalIndexType>& range_start_ends,
	const array<GlobalIndexType>& range_start_ends,

		ranges.get_num_elems() - 1, ranges, part_id_mapping, range_bounds,
		part_ids, part_id_mapping.get_num_elems() > 0);

	* Part parti_id[i] contains the indices
	* Part part_id[i] contains the indices

	// remove duplicates
	// join (now consecutive) starts and ends into combined array

Enables building Partition from local ranges #1227

Enables building Partition from local ranges #1227

Conversation

MarcelKoch commented Dec 5, 2022

upsj left a comment • edited Loading

Choose a reason for hiding this comment

greole left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

upsj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

upsj Feb 8, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarcloud bot commented Aug 17, 2023 • edited Loading

upsj left a comment •

edited

Loading

upsj Feb 8, 2023 •

edited

Loading

sonarcloud bot commented Aug 17, 2023 •

edited

Loading