diff --git a/docs/getting_started/spack.md b/docs/getting_started/spack.md
index d2a9462bf..d3ec3d5e1 100644
--- a/docs/getting_started/spack.md
+++ b/docs/getting_started/spack.md
@@ -1,7 +1,7 @@
# Set up Spack for building ACCESS models
!!! danger
- This page is tailored to experienced users and collaborators developing ACCESS models.
+ This page is tailored to experienced users and collaborators developing ACCESS models. This step is *not* required if you *only* want to run a model.
[Spack](https://spack.io/about/) is a build-from-source package manager, specifically designed to simplify the installation of scientific software on supercomputers.
@@ -9,7 +9,6 @@ To use _Spack_, please familiarise yourself with the [Basic Usage instructions](
We also recommend that you refer to the [Spack 101 Tutorial](https://spack-tutorial.readthedocs.io/en/latest/).
-
## Prerequisites
These instructions are tailored specifically for _Gadi_. To use _Spack_ on _Gadi_, you must have an NCI account. For instructions on how to set up an account, refer to [Set Up your NCI Account](/getting_started/set_up_nci_account).
@@ -20,11 +19,11 @@ These instructions are tailored specifically for _Gadi_. To use _Spack_ on _Gadi
### Create a directory for Spack
-Create a directory on the filesystem where _Spack_ will be installed (e.g. `/g/data/$PROJECT/$USER/spack/0.22`).
+Create a directory on the filesystem where _Spack_ will be installed (e.g. `/g/data/$PROJECT/$USER/spack/0.22`). Use the `/g/data` filesystem if you wish to run the binaries on the compute nodes.
```
-mkdir -p spack/0.22
-cd spack/0.22
+mkdir -p /g/data/$PROJECT/$USER/spack/0.22
+cd /g/data/$PROJECT/$USER/spack/0.22
```
### Clone the relevant git repositories
@@ -47,21 +46,167 @@ ln -s -r -v spack-config/v0.22/gadi/* spack/etc/spack/
!!! success
Your _Spack_ setup is complete!
-## Test Spack (OPTIONAL)
+## Enable Spack
-To test _Spack_ we will create an `ACCESS-TEST` environment and build the relevant packages. Then, we will uninstall all the packages and remove the environment.
+!!! warning
+ This step needs to be carried out for any new login or new shell environment.
```
+cd /g/data/$PROJECT/$USER/spack/0.22
module purge
-cd spack/0.22
. spack-config/spack-enable.bash
+```
+
+!!! warning
+ There is a space between the `.` and the path to the file, as we are sourcing the file.
+
+## Test Spack (OPTIONAL)
+
+To test _Spack_ we will create an `ACCESS-TEST` environment and build the relevant packages. It takes approximately 30 minutes to compile. Then, we will uninstall all the packages and remove the environment.
+
+
+### Create the Spack managed environment
+
+```
git clone https://github.com/ACCESS-NRI/ACCESS-TEST.git
spack env create test ACCESS-TEST/spack.yaml
+```
+
+
+ spack env create test ACCESS-TEST/spack.yaml
+ \==> Created environment test in: /g/data/$PROJECT/$USER/spack/0.22/environments/test
+ \==> Activate with: spack env activate test
+
+
+### Compile packages
+
+```
spack env activate -p test
spack find
+```
+
+
+ spack env activate -p test
+ [test]$ spack find
+ \==> In environment test
+ \==> 1 root specs
+ - access-test@git.2024.09.20=2024.09.20
+
+ \==> 0 installed packages
+
+
+```
spack concretize -f --fresh
-spack install --verbose
+spack install
+```
+
+
+ [test]$ spack concretize -f --fresh
+ \==> Concretized access-test@git.2024.09.20=2024.09.20
+ - hmy75yl access-test@git.2024.09.20=2024.09.20%intel@2021.10.0~deterministic build_system=bundle arch=linux-rocky8-x86_64
+ [e] 5elnsoi ^glibc@2.28%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
+ - kok5n7h ^oasis3-mct@git.2023.11.09=2023.11.09%intel@2021.10.0~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
+ - doeoclg ^gmake@4.4.1%intel@2021.10.0~guile build_system=generic arch=linux-rocky8-x86_64
+ - ntfunrm ^netcdf-fortran@4.6.1%intel@2021.10.0~doc+pic+shared build_system=autotools arch=linux-rocky8-x86_64
+ - vob7om3 ^netcdf-c@4.9.2%intel@2021.10.0+blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared+szip+zstd build_system=autotools patches=0161eb8 arch=linux-rocky8-x86_64
+ - 357vng5 ^bzip2@1.0.8%intel@2021.10.0~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64
+ - y7n7vkn ^diffutils@3.10%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
+ - 55ipnye ^libiconv@1.17%intel@2021.10.0 build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
+ - zrjfo56 ^c-blosc@1.21.5%intel@2021.10.0+avx2~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
+ [e] rldyvqn ^cmake@3.24.2%intel@2021.10.0~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64
+ - 7a5olrr ^lz4@1.9.4%intel@2021.10.0+pic build_system=makefile libs=shared,static arch=linux-rocky8-x86_64
+ - wk4pvru ^snappy@1.1.10%intel@2021.10.0~ipo+pic+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
+ - aumsrgz ^hdf5@1.14.3%intel@2021.10.0~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8 arch=linux-rocky8-x86_64
+ - vrupasu ^pkgconf@2.2.0%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64
+ - x4hw5jq ^libaec@1.0.6%intel@2021.10.0~ipo+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
+ - wa2x7rh ^zlib-ng@2.1.6%intel@2021.10.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64
+ - kzc7pcv ^zstd@1.5.6%intel@2021.10.0~programs build_system=makefile libs=shared,static arch=linux-rocky8-x86_64
+ [e] uvea7q2 ^openmpi@4.1.5%intel@2021.10.0 cppflags='-diag-disable=10441' ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64
+ [test]$ spack install
+ [+] /usr (external glibc-2.28-5elnsoiqgcg5k5zmmwsp33bmnmaa3g5p)
+ [+] /apps/cmake/3.24.2 (external cmake-3.24.2-rldyvqnwmmg4cepzstrlmu75vx2wf6cj)
+ \==> openmpi@4.1.5 : has external module in ['openmpi/4.1.5']
+ [+] /apps/openmpi/4.1.5 (external openmpi-4.1.5-uvea7q2exkjwz2v7azix3x4zfcddnc23)
+ \==> Installing gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4 [4/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4
+ \==> Installing libaec-1.0.6-x4hw5jqq3zvnrgjicgweicomeaelulqq [5/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libaec-1.0.6-x4hw5jqq3zvnrgjicgweicomeaelulqq
+ \==> Installing zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6 [6/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6
+ \==> Installing snappy-1.1.10-wk4pvrufyvy7v3hxn5nwa3i3fncp3azm [7/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/snappy-1.1.10-wk4pvrufyvy7v3hxn5nwa3i3fncp3azm
+ \==> Installing pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa [8/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa
+ \==> Installing zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4 [9/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4
+ \==> Installing lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4 [10/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4
+ \==> Installing libiconv-1.17-55ipnyeeqcpbfgaqfanu36viaqqni3sx [11/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libiconv-1.17-55ipnyeeqcpbfgaqfanu36viaqqni3sx
+ \==> Installing hdf5-1.14.3-aumsrgzvbh6grtyyegzuufilnqa7ftm7 [12/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/hdf5-1.14.3-aumsrgzvbh6grtyyegzuufilnqa7ftm7
+ \==> Installing c-blosc-1.21.5-zrjfo567d2n6ctwayae77z3b54mf23yc [13/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/c-blosc-1.21.5-zrjfo567d2n6ctwayae77z3b54mf23yc
+ \==> Installing diffutils-3.10-y7n7vkngczu47neysm3retisvlsmw53l [14/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/diffutils-3.10-y7n7vkngczu47neysm3retisvlsmw53l
+ \==> Installing bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt [15/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt
+ \==> Installing netcdf-c-4.9.2-vob7om32jopqwss5jilrdtqqogjvcmzb [16/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-c-4.9.2-vob7om32jopqwss5jilrdtqqogjvcmzb
+ \==> Installing netcdf-fortran-4.6.1-ntfunrmysxanqqu7sqfmf66zdkd2xemy [17/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-fortran-4.6.1-ntfunrmysxanqqu7sqfmf66zdkd2xemy
+ \==> Installing oasis3-mct-git.2023.11.09=2023.11.09-kok5n7hvm374eicnidcedxhgxmmytc2p [18/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/oasis3-mct-git.2023.11.09_2023.11.09-kok5n7hvm374eicnidcedxhgxmmytc2p
+ \==> Installing access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc [19/19]
+ ...
+ [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-test-git.2024.09.20_2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc
+ \==> Updating view at /g/data/$PROJECT/$USER/spack/0.22/environments/test/.spack-env/view
+
+
+!!! info
+ The full output has been truncated for brevity.
+
+```
spack find
+```
+
+
+ [test]$ spack find
+ \==> In environment test
+ \==> 1 root specs
+ [+] access-test@git.2024.09.20=2024.09.20
+
+ \==> Installed packages
+ -- linux-rocky8-x86_64 / intel@2021.10.0 ------------------------
+ access-test@git.2024.09.20=2024.09.20 lz4@1.9.4
+ bzip2@1.0.8 netcdf-c@4.9.2
+ c-blosc@1.21.5 netcdf-fortran@4.6.1
+ cmake@3.24.2 oasis3-mct@git.2023.11.09=2023.11.09
+ diffutils@3.10 openmpi@4.1.5
+ glibc@2.28 pkgconf@2.2.0
+ gmake@4.4.1 snappy@1.1.10
+ hdf5@1.14.3 zlib-ng@2.1.6
+ libaec@1.0.6 zstd@1.5.6
+ libiconv@1.17
+ \==> 19 installed packages
+
+
+```
spack uninstall --remove --all
spack env deactivate
spack env rm test
@@ -73,7 +218,7 @@ rm -rf ACCESS-TEST
Keep your Spack instance up-to-date by doing the following:
```
-cd spack/0.22
+cd /g/data/$PROJECT/$USER/spack/0.22
git -C spack fetch --all -Pp
git -C spack reset --hard origin/releases/v0.22
git -C spack-config pull