Building and running the RFMIP examples on a GPU enable desktop with Spack

Compiler Availability

This assumes that you have the PGI compiler installed on your desktop.

Install and Set-up Spack

Follow the instructions here to install Spack: https://spack.readthedocs.io/en/latest/getting_started.html

Use the bash shell script for setting up your environment. In general, C shell scripts appear not to be as well tested in HPC environment so you save yourself a lot of trouble but just switching completely over to bash.

Add the PGI compiler to your Spack environment: spack compiler add <PGI installation path>

You will also need a GCC compiler, because some Spack packages will only compile with GCC. spack compiler add <GCC installation path>

It is handy to have modules install to load/unload multiple environment, to install modules and have the entire Spack software stack available follow the instructions here: https://spack.readthedocs.io/en/latest/module_file_support.html

Install NETCDF-Fortran via Spack

Many Spack packages will not build with PGI, but NETCDF-Fortran and NETCDF will. This command should install what you need: spack install netcdf-fortran%[email protected] ^mpich ^pkgconf%gcc ^perl%gcc ^findutils%gcc ^libpciaccess%gcc

Build CPU Script

unset RTE_KERNELS

module load mpich-3.3.1-pgi-19.3-24ryljg
module load netcdf-4.7.0-pgi-19.3-kpr2eoe
module load netcdf-fortran-4.4.5-pgi-19.3-27jvsnj 

RRTMGP_HOME=/homes/naromero/rte-rrtmgp

cd $RRTMGP_HOME/build
export FC=mpif90
export FCFLAGS=-O3
make clean
make

cd $RRTMGP_HOME/examples/rfmip-clear-sky
export NCHOME=`spack location -i netcdf%pgi
export NFHOME=`spack location -i netcdf-fortran%pgi
export RRTMGP_DIR=$RRTMGP_HOME/build
rm -rf rrtmgp_rfmip_sw rrtmgp_rfmip_lw *.o *.mod *.optrpt
make

Build GPU Script

Same as the build CPU script except: RTE_KERNELS=openacc and export FCFLAGS="-O3 -g -ta=tesla -Minfo=accel"