ParallelRandomFields.jl

Efficient multi-XPU parallel random field generator for large 2D and 3D problems

ParallelRandomFields enables to sample spatial realisations of 2D or 3D random fields with given power spectrum. The method allows for fast and accurate generation of Gaussian random fields with anisotropic exponential (left figure pane) and isotropic Gaussian (right figure pane) covariance functions. The method is fast, accurate and fully local. We use ParallelStencil.jl to provide an architecture-agnostic high-level CPU and GPU implementation, and ImplicitGlobalGrid.jl for multi-XPU support (distributed memory parallelisation).

The implementation builds upon an approach proposed in [1] and employs a parallel implementation of the method based on spectral representation described in [2]. Advantages of this method are the possibility of random field simulation on an arbitrary grid and the simplicity of parallel implementation of the algorithm. The method is flexible and is also applicable for arbitrary anisotropic spectrum.

ParallelRandomFields.jl is the Julia version with additional multi-XPU support of the original CUDA C implementation accessible here, published in Computers & Geosciences:

@article{raess2019grf,
    title = {Efficient parallel random field generator for large 3-D geophysical problems},
    journal = {Computers & Geosciences},
    volume = {131},
    pages = {158-169},
    year = {2019},
    issn = {0098-3004},
    doi = {https://doi.org/10.1016/j.cageo.2019.06.007},
    url = {https://www.sciencedirect.com/science/article/pii/S0098300418309944},
    author = {Ludovic Räss and Dmitriy Kolyukhin and Alexander Minakov},
    keywords = {Geophysics, Geostatistics, Seismology, Computational methods, Parallel and high-performance computing}
}

Content

• Module documentation callable from the Julia REPL / IJulia
• Usage
• Supported backends
• Multi-XPU implementation
• Dependencies
• Installation
• Development status
• Questions, comments and discussions
• References

Module documentation callable from the Julia REPL / IJulia

The module documentation can be called from the Julia REPL or in IJulia:

julia> using ParallelRandomFields

julia>?

help?> ParallelRandomFields
search: ParallelRandomFields

  Module ParallelRandomFields

  Enables to sample spatial realisations of 2D and 3D Gaussian random fields with given power spectrum using following
  covariance functions:

    •    anisotropic exponential

    •    isotropic Gaussian

  ParallelRandomFields can be deployed on both (multi-) GPUs and CPUs.

  General overview and examples
  ≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡

  https://github.com/luraess/ParallelRandomFields.jl

  Functions are defined in the submodules.

  Submodules
  ≡≡≡≡≡≡≡≡≡≡≡≡

    •    ParallelRandomFields.grf2D_Threads

    •    ParallelRandomFields.grf3D_Threads

    •    ParallelRandomFields.grf2D_CUDA

    •    ParallelRandomFields.grf3D_CUDA

  To see a description of a function type ?<functionname>.

Usage

Parallel random fields can be interactively generated within the Julia REPL using the 2D or 3D generator functions generate_grf2D(), generate_grf3D() available from the selected submodule depending on the number of dimensions (2D, 3D) and backend (Threads, CUDA). Consult the submodule and function documentation callable from the Julia REPL or in IJulia for further details.

To generate a 2D Gaussian random field with exponential anisotropic covariance function, type following in the Julia REPL:

julia> using ParallelRandomFields

julia> using ParallelRandomFields.grf2D_Threads

julia> generate_grf2D();

Main functionalities

• The 2D generator function generate_grf2D() calls random field realisation functions grf2D_expon!() (exponential covariance) and grf2D_gauss!() (Gaussian covariance) on the chosen backend (CPU/GPU).

• The 3D generator function generate_grf3D() calls random field realisation functions grf3D_expon!() (exponential covariance) and grf3D_gauss!() (Gaussian covariance) on the chosen backend (CPU/GPU).

• The following random field realisation functions can be used within user-specific external code using ParallelStencil.jl to generate random fields:

  • grf2D_expon!()
  • grf2D_gauss!()
  • grf3D_expon!()
  • grf3D_gauss!()

---

💡 Besides the module functionalities, the scripts folder contains:

• a runme2D_Threads.jl script that provides an example of implementing the 2D random field realisation functions (grf2D_expon!(), grf2D_gauss!()) on the default Threads backend;
• a runme2D_multixpu.jl script that provides an example of a multi-XPU implementation of the 2D random field realisation functions (grf2D_expon!(), grf2D_gauss!()) on both the Threads and CUDA backend;
• a standalone_scripts directory about standalone "monolithic" random field generator single and multi-XPU scripts (originally used prior to creating the ParallelRandomFields module).

Supported backends

ParallelRandomFields builds upon ParallelStencil.jl and thus supports the default Threads backend as well as the CUDA backend for GPU acceleration. Multi-threading is enabled upon export of the JULIA_NUM_THREADS variable.

Multi-XPU implementation

The advantage of the iterative random field generation algorithm is the trivial distributed memory parallelisation. The generation algorithm does not perform any communication among neighbouring cells nor global operations making it highly suitable for an efficient distributed memory parallelisation implementation using ImplicitGlobalGrid.jl (relying on MPI.jl and CUDA.jl for CUDA-aware MPI features).

The random field realisation functions are multi-XPU ready and can be called in codes relying on ParallelStencil and IMplicitGlobalGrid. See this example for a 2D multi-XPU implementation. The runme2D_multixpu.jl script can be launched as following upon installing the mpiexecjl wrapper from MPI.jl (install infos):

mpiexecjl -n 2 -host localhost julia --project runme2D_multixpu.jl

Note: refer to the documentation of your Supercomputing Centre for instructions to run Julia at scale. Instructions for running on the Piz Daint GPU supercomputer at the Swiss National Supercomputing Centre can be found here and for running on the octopus GPU supercomputer at the Swiss Geocomputing Centre can be found here. (The julia_setup_multixpu.sh script provides a multi-GPU node config, to then launch the runme2D.sh script which will call submit_julia.sh to launch runme2D_multixpu.jl.)

Note also that previous standalone multi-XPU scripts can be found in the ParallelRandomFields_multixpu directory.

Dependencies

ParallelRandomFields relies on:

• ParallelStencil.jl which builds upon CUDA.jl for GPU support
• ImplicitGlobalGrid.jl which builds upon MPI.jl
• MAT.jl and Plots.jl are required for saving and visualisation purposes.

Installation

ParallelRandomFields may be installed directly with the Julia package manager from the Julia REPL:

julia> ]

pkg> add https://github.com/luraess/ParallelRandomFields.jl

Development status

This section lists the current development status of the module.

🚧 TODOs

• Enhance documentation
• Add tests

Questions, comments and discussions

To discuss technical (multi-) GPU issues, please post on Julia Discourse in the GPU topic or the Julia at Scale topic. To discuss numerical/domain-science issues, please post on Julia Discourse in the Numerics topic or the Modelling & Simulations topic or whichever other topic fits best your issue.

References

[1] Räss, L., Kolyukhin D., and Minakov, A., 2019. Efficient parallel random field generator for large 3-D geophysical problems. Computers & Geosciences, 131, 158-169.

[2] Sabelfeld, K.K., 1991. Monte Carlo Methods in Boundary Value Problems. Springer.