Skip to content

Commit

Permalink
getting_started/spack.md: incorporate feedback
Browse files Browse the repository at this point in the history 
* Thanks to Claire Carouge, Chris Bull and Davide Marchegiani.
  • Loading branch information
@harshula
harshula committed Oct 16, 2024
1 parent a610064 commit 51df1a8
Showing 1 changed file with 155 additions and 10 deletions.
165 changes: 155 additions & 10 deletions docs/getting_started/spack.md
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
# Set up Spack for building ACCESS models

!!! danger
This page is tailored to experienced users and collaborators developing ACCESS models.
This page is tailored to experienced users and collaborators developing ACCESS models. This step is *not* required if you *only* want to run a model.

[Spack](https://spack.io/about/) is a build-from-source package manager, specifically designed to simplify the installation of scientific software on supercomputers.

To use _Spack_, please familiarise yourself with the [Basic Usage instructions](https://spack.readthedocs.io/en/latest/basic_usage.html) and [Environments](https://spack.readthedocs.io/en/latest/environments.html).

We also recommend that you refer to the [Spack 101 Tutorial](https://spack-tutorial.readthedocs.io/en/latest/).


## Prerequisites
These instructions are tailored specifically for _Gadi_. To use _Spack_ on _Gadi_, you must have an NCI account. For instructions on how to set up an account, refer to [Set Up your NCI Account](/getting_started/set_up_nci_account).

Expand All @@ -20,11 +19,11 @@ These instructions are tailored specifically for _Gadi_. To use _Spack_ on _Gadi

### Create a directory for Spack

Create a directory on the filesystem where _Spack_ will be installed (e.g. `/g/data/$PROJECT/$USER/spack/0.22`).
Create a directory on the filesystem where _Spack_ will be installed (e.g. `/g/data/$PROJECT/$USER/spack/0.22`). Use the `/g/data` filesystem if you wish to run the binaries on the compute nodes.

```
mkdir -p spack/0.22
cd spack/0.22
mkdir -p /g/data/$PROJECT/$USER/spack/0.22
cd /g/data/$PROJECT/$USER/spack/0.22
```

### Clone the relevant git repositories
Expand All @@ -47,21 +46,167 @@ ln -s -r -v spack-config/v0.22/gadi/* spack/etc/spack/
!!! success
Your _Spack_ setup is complete!

## Test Spack (OPTIONAL)
## Enable Spack

To test _Spack_ we will create an `ACCESS-TEST` environment and build the relevant packages. Then, we will uninstall all the packages and remove the environment.
!!! warning
This step needs to be carried out for any new login or new shell environment.

```
cd /g/data/$PROJECT/$USER/spack/0.22
module purge
cd spack/0.22
. spack-config/spack-enable.bash
```

!!! warning
There is a space between the `.` and the path to the file, as we are sourcing the file.

## Test Spack (OPTIONAL)

To test _Spack_ we will create an `ACCESS-TEST` environment and build the relevant packages. It takes approximately 30 minutes to compile. Then, we will uninstall all the packages and remove the environment.


### Create the Spack managed environment

```
git clone https://github.com/ACCESS-NRI/ACCESS-TEST.git
spack env create test ACCESS-TEST/spack.yaml
```

<terminal-window>
<terminal-line data="input">spack env create test ACCESS-TEST/spack.yaml</terminal-line>
<terminal-line><span class="indigo bold">\==></span> Created environment <span class="cyan">test</span> in: <span class="cyan">/g/data/$PROJECT/$USER/spack/0.22/environments/test</span></terminal-line>
<terminal-line><span class="indigo bold">\==></span> Activate with: <span class="cyan">spack env activate test</span></terminal-line>
</terminal-window>

### Compile packages

```
spack env activate -p test
spack find
```

<terminal-window>
<terminal-line data="input">spack env activate -p test</terminal-line>
<terminal-line data="input">[test]$ spack find</terminal-line>
<terminal-line>\==> In environment test</terminal-line>
<terminal-line>\==> 1 root specs</terminal-line>
<terminal-line> - [email protected]=2024.09.20</terminal-line>
<terminal-line></terminal-line>
<terminal-line>\==> 0 installed packages</terminal-line>
</terminal-window>

```
spack concretize -f --fresh
spack install --verbose
spack install
```

<terminal-window>
<terminal-line data="input">[test]$ spack concretize -f --fresh</terminal-line>
<terminal-line>\==> Concretized [email protected]=2024.09.20</terminal-line>
<terminal-line> - hmy75yl [email protected]=2024.09.20%[email protected]~deterministic build_system=bundle arch=linux-rocky8-x86_64</terminal-line>
<terminal-line>[e] 5elnsoi ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - kok5n7h ^[email protected]=2023.11.09%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - doeoclg ^[email protected]%[email protected]~guile build_system=generic arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - ntfunrm ^[email protected]%[email protected]~doc+pic+shared build_system=autotools arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - vob7om3 ^[email protected]%[email protected]+blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared+szip+zstd build_system=autotools patches=0161eb8 arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - 357vng5 ^[email protected]%[email protected]~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - y7n7vkn ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - 55ipnye ^[email protected]%[email protected] build_system=autotools libs=shared,static arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - zrjfo56 ^[email protected]%[email protected]+avx2~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line>
<terminal-line>[e] rldyvqn ^[email protected]%[email protected]~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - 7a5olrr ^[email protected]%[email protected]+pic build_system=makefile libs=shared,static arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - wk4pvru ^[email protected]%[email protected]~ipo+pic+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - aumsrgz ^[email protected]%[email protected]~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8 arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - vrupasu ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - x4hw5jq ^[email protected]%[email protected]~ipo+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - wa2x7rh ^[email protected]%[email protected]+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64</terminal-line>
<terminal-line> - kzc7pcv ^[email protected]%[email protected]~programs build_system=makefile libs=shared,static arch=linux-rocky8-x86_64</terminal-line>
<terminal-line>[e] uvea7q2 ^[email protected]%[email protected] cppflags='-diag-disable=10441' ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64</terminal-line>
<terminal-line data="input">[test]$ spack install</terminal-line>
<terminal-line>[+] /usr (external glibc-2.28-5elnsoiqgcg5k5zmmwsp33bmnmaa3g5p)</terminal-line>
<terminal-line>[+] /apps/cmake/3.24.2 (external cmake-3.24.2-rldyvqnwmmg4cepzstrlmu75vx2wf6cj)</terminal-line>
<terminal-line>\==> [email protected] : has external module in ['openmpi/4.1.5']</terminal-line>
<terminal-line>[+] /apps/openmpi/4.1.5 (external openmpi-4.1.5-uvea7q2exkjwz2v7azix3x4zfcddnc23)</terminal-line>
<terminal-line>\==> Installing gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4 [4/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4</terminal-line>
<terminal-line>\==> Installing libaec-1.0.6-x4hw5jqq3zvnrgjicgweicomeaelulqq [5/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libaec-1.0.6-x4hw5jqq3zvnrgjicgweicomeaelulqq</terminal-line>
<terminal-line>\==> Installing zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6 [6/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6</terminal-line>
<terminal-line>\==> Installing snappy-1.1.10-wk4pvrufyvy7v3hxn5nwa3i3fncp3azm [7/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/snappy-1.1.10-wk4pvrufyvy7v3hxn5nwa3i3fncp3azm</terminal-line>
<terminal-line>\==> Installing pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa [8/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa</terminal-line>
<terminal-line>\==> Installing zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4 [9/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4</terminal-line>
<terminal-line>\==> Installing lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4 [10/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4</terminal-line>
<terminal-line>\==> Installing libiconv-1.17-55ipnyeeqcpbfgaqfanu36viaqqni3sx [11/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libiconv-1.17-55ipnyeeqcpbfgaqfanu36viaqqni3sx</terminal-line>
<terminal-line>\==> Installing hdf5-1.14.3-aumsrgzvbh6grtyyegzuufilnqa7ftm7 [12/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/hdf5-1.14.3-aumsrgzvbh6grtyyegzuufilnqa7ftm7</terminal-line>
<terminal-line>\==> Installing c-blosc-1.21.5-zrjfo567d2n6ctwayae77z3b54mf23yc [13/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/c-blosc-1.21.5-zrjfo567d2n6ctwayae77z3b54mf23yc</terminal-line>
<terminal-line>\==> Installing diffutils-3.10-y7n7vkngczu47neysm3retisvlsmw53l [14/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/diffutils-3.10-y7n7vkngczu47neysm3retisvlsmw53l</terminal-line>
<terminal-line>\==> Installing bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt [15/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt</terminal-line>
<terminal-line>\==> Installing netcdf-c-4.9.2-vob7om32jopqwss5jilrdtqqogjvcmzb [16/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-c-4.9.2-vob7om32jopqwss5jilrdtqqogjvcmzb</terminal-line>
<terminal-line>\==> Installing netcdf-fortran-4.6.1-ntfunrmysxanqqu7sqfmf66zdkd2xemy [17/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-fortran-4.6.1-ntfunrmysxanqqu7sqfmf66zdkd2xemy</terminal-line>
<terminal-line>\==> Installing oasis3-mct-git.2023.11.09=2023.11.09-kok5n7hvm374eicnidcedxhgxmmytc2p [18/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/oasis3-mct-git.2023.11.09_2023.11.09-kok5n7hvm374eicnidcedxhgxmmytc2p</terminal-line>
<terminal-line>\==> Installing access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc [19/19]</terminal-line>
<terminal-line>...</terminal-line>
<terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-test-git.2024.09.20_2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc</terminal-line>
<terminal-line>\==> Updating view at /g/data/$PROJECT/$USER/spack/0.22/environments/test/.spack-env/view</terminal-line>
</terminal-window>

!!! info
The full output has been truncated for brevity.

```
spack find
```

<terminal-window>
<terminal-line data="input">[test]$ spack find</terminal-line>
<terminal-line>\==> In environment test</terminal-line>
<terminal-line>\==> 1 root specs</terminal-line>
<terminal-line>[+] [email protected]=2024.09.20</terminal-line>
<terminal-line></terminal-line>
<terminal-line>\==> Installed packages</terminal-line>
<terminal-line>-- linux-rocky8-x86_64 / [email protected] ------------------------</terminal-line>
<terminal-line>[email protected]=2024.09.20 [email protected]</terminal-line>
<terminal-line>[email protected] [email protected]</terminal-line>
<terminal-line>[email protected] [email protected]</terminal-line>
<terminal-line>[email protected] [email protected]=2023.11.09</terminal-line>
<terminal-line>[email protected] [email protected]</terminal-line>
<terminal-line>[email protected] [email protected]</terminal-line>
<terminal-line>[email protected] [email protected]</terminal-line>
<terminal-line>[email protected] [email protected]</terminal-line>
<terminal-line>[email protected] [email protected]</terminal-line>
<terminal-line>[email protected]</terminal-line>
<terminal-line>\==> 19 installed packages</terminal-line>
</terminal-window>

```
spack uninstall --remove --all
spack env deactivate
spack env rm test
Expand All @@ -73,7 +218,7 @@ rm -rf ACCESS-TEST
Keep your Spack instance up-to-date by doing the following:

```
cd spack/0.22
cd /g/data/$PROJECT/$USER/spack/0.22
git -C spack fetch --all -Pp
git -C spack reset --hard origin/releases/v0.22
git -C spack-config pull
Expand Down

0 comments on commit 51df1a8

Please sign in to comment.