Add GPU support for extruded 1D spaces #2172
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds GPU support for extruded 1D spaces, with the following modifications:
Ni == Nj
frommapreduce_cuda_kernel
,columnwise_partition
,columnwise_universal_index
, andmultiple_field_solve_universal_index
operator_shmem
operator_fill_shmem!
, andoperator_evaluate
corresponding to the 1D versions of all SEM operatorsDSSDataTypes
toDSSTypes2D
, and define a new constantDSSTypes1D
dss_1d!
corresponding toCUDADevice
, and extenddss!
to support both 1D and 2D data typeslocal_geometry
anddss_weights
of aSpectralElementSpace1D
fromArray
s toCuArray
sThese changes are also tested through CI:
unit_2d.jl
andconvergence_2d.jl
IFH
,IHF
,VIFH
, andVIHF
tounit_mapreduce.jl
, which already has a CUDA jobTo fix several test failues, I had to make a few additional changes:
Gradient
by directly differentiating vectors instead of breaking them up into components, as the original design could not distinguish the gradient of a scalar from the gradient of a single-component vector, and one of the tests inunit_2d.jl
takes the gradient of aUVector
fieldstrip_space
toFiniteDifferenceOperator
boundary conditions, as one of the tests inunit_2d.jl
uses a 1D field for theCurlC2F
boundary conditionlevel(space::ExtrudedFiniteDifferenceSpace, v)
from 4 to 1 to avoid a GPU inference failure inreconstruct_placeholder_space
when evaluating boundary conditions (we hit some sort of compiler heuristic when we use 2 or 4 specialized methods, so we need to just have one method with union-splitting)CuArray
s toArray
s inunit_2d.jl
andunit_spaces.jl
to avoid scalar indexing errorsSpectralElementSpace1D
inopt_spaces.jl
from 141 to 825 on GPUs and from 137 to 150 on CPUs