Skip to content

Commit

Permalink
Docs: Update README.md in compile-notes for frontera
Browse files Browse the repository at this point in the history
  • Loading branch information
lwJi committed Mar 24, 2024
1 parent 2224a84 commit fd08bd4
Showing 1 changed file with 56 additions and 42 deletions.
98 changes: 56 additions & 42 deletions Docs/compile-notes/frontera/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,32 +34,37 @@

### CUDA version (`[email protected]`)

* Load `intel/19.1.1`

## The Long Way
- `module reset` or `module load intel/19.1.1`

* Download spack
* Install SpacetimeX

- `git clone -c feature.manyFiles=true https://github.com/spack/spack.git`

- use `develop` branch: (maybe `git checkout 141c7de5`)
```bash
. /work2/08708/liwei/frontera/SpackSource/spack/share/spack/setup-env.sh
spack load [email protected]
spack load [email protected]
- fix error with `krb5%oneapi` temporary:
* modify `var/spack/repos/builtin/packages/bison/package.py` line 68
```bash
+ conflicts(
+ "%oneapi",
+ msg="bison may have unexpected behaviours with oneapi, \
+ see https://github.com/spack/spack/issues/37172",
+ )
```
cd Cactus
gmake SpacetimeX-cuda options=repos/SpacetimeX/Docs/compile-notes/frontera/configs/config-frontera-cuda-intel.cfg
cp repos/SpacetimeX/Docs/thornlist/spacetimex.th configs/SpacetimeX-cuda/ThornList
gmake -j16 SpacetimeX-cuda
```


## The Long Way

### Intel-Oneapi version (`[email protected]`)

* Load `intel/23.1.0`

- `module load intel/23.1.0`

* Setup spack
* Download spack

- `git clone -c feature.manyFiles=true https://github.com/spack/spack.git`

- use `develop` branch: (maybe `git checkout 141c7de5`)

- `. share/spack/setup-env.sh`

Expand All @@ -69,7 +74,7 @@

* Replace the last line of `oneapi-23.1.0/spack.yaml` with your own dir (say `/work2/.../username/frontera/SpackView/oneapi/view`)

* Replace the dir `/work2/08708/liwei/frontera/SpackView/oneapi/view` in `configs/config_frontera_oneapi.cfg` (with say `/work2/.../username/frontera/SpackView/oneapi/view`)
* Replace the dir `/work2/08708/liwei/frontera/SpackView/oneapi/view` (with say `/work2/.../username/frontera/SpackView/oneapi/view`) in `configs/config-frontera-oneapi.cfg`

* Install other required packages

Expand All @@ -88,43 +93,52 @@
gmake -j24 SpacetimeX-oneapi
```

* More tricks:

* Make `[email protected]` work with `[email protected]`:

modify `/var/spack/repos/builtin/packages/silo/package.py`:
### Cuda version (`[email protected]` with `[email protected]`)

```bash
- depends_on("[email protected]:1.10", when="@:4.10+hdf5")
+ depends_on("[email protected]:", when="@:4.10+hdf5")
```
* Load `intel/19.1.1`

- `module reset` or `module load intel/19.1.1`

### CUDA version (`[email protected]`)
* Download spack

- `git clone -c feature.manyFiles=true https://github.com/spack/spack.git`

- `git checkout relesases/v0.21`

- `. share/spack/setup-env.sh`

- `spack compiler find`

* Make sure rerun `spack install [email protected] %[email protected]` again on `rtx-dev` or `rtx`
* Install [email protected]

* Create a dir where you want put `view` in (say `/work2/.../username/frontera/SpackView/cuda`)
```bash
spack install [email protected] %[email protected]
spack compiler add ... # ... is the last line of previous command
```

* Replace the last line of `cuda-11.8.0/spack_yaml` with your dir (say `/work2/.../username/frontera/SpackView/cuda/view`)
* Create a dir where you want put `view` in (say `/work2/.../username/frontera/SpackView/cuda-intel`)

* Replace the dir `/work2/08708/liwei/frontera/SpackView/cuda/view` in `configs/config_frontera_cuda.cfg` (with say `/work2/.../username/frontera/SpackView/cuda/view`)
* Replace the last line of `cuda-11.8.0-intel/spack_yaml` with your dir (say `/work2/.../username/frontera/SpackView/cuda-intel/view`)

* Replace the dir `/work2/08708/liwei/frontera/SpackView/cuda-intel/view` (with say `/work2/.../username/frontera/SpackView/cuda-intel/view`)
in `config-frontera-cuda-intel.cfg`

* Install other required packages

```bash
env TMPDIR=$WORK/tmp spack --env-dir ./cuda-11.8.0 compiler find
env TMPDIR=$WORK/tmp spack --env-dir ./cuda-11.8.0 concretize --force
env TMPDIR=$WORK/tmp spack --env-dir ./cuda-11.8.0 install --fail-fast
```
```bash
env TMPDIR=$WORK/tmp spack --env-dir ./cuda-11.8.0-intel compiler find view-cuda-compilers
env TMPDIR=$WORK/tmp spack --env-dir ./cuda-11.8.0-intel concretize --force
env TMPDIR=$WORK/tmp spack --env-dir ./cuda-11.8.0-intel install --fail-fast
```

* Install SpacetimeX

```bash
spack load [email protected]
spack load [email protected]
cd Cactus
gmake SpacetimeX-cuda options=repos/SpacetimeX/Docs/compile-notes/frontera/configs/config-frontera-cuda.cfg
cp repos/SpacetimeX/Docs/thornlist/spacetimex.th configs/SpacetimeX-cuda/ThornList
gmake -j16 SpacetimeX-cuda
```
```bash
spack load [email protected]
spack load [email protected]
cd Cactus
gmake SpacetimeX-cuda options=config-frontera-cuda-intel.cfg
cp repos/SpacetimeX/Docs/thornlist/spacetimex.th configs/SpacetimeX-cuda/ThornList
gmake -j16 SpacetimeX-cuda
```

0 comments on commit fd08bd4

Please sign in to comment.