-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
getting_started/spack.md: incorporate feedback
- Loading branch information
Showing
1 changed file
with
126 additions
and
9 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,14 @@ | ||
| # Set up Spack for building ACCESS models | ||
|
|
||
| !!! danger | ||
| This page is tailored to experienced users and collaborators developing ACCESS models. | ||
| This page is tailored to experienced users and collaborators developing ACCESS models. This step is *not* required if you *only* want to run a model. | ||
|
|
||
| [Spack](https://spack.io/about/) is a build-from-source package manager, specifically designed to simplify the installation of scientific software on supercomputers. | ||
|
|
||
| To use _Spack_, please familiarise yourself with the [Basic Usage instructions](https://spack.readthedocs.io/en/latest/basic_usage.html) and [Environments](https://spack.readthedocs.io/en/latest/environments.html). | ||
|
|
||
| We also recommend that you refer to the [Spack 101 Tutorial](https://spack-tutorial.readthedocs.io/en/latest/). | ||
|
|
||
|
|
||
| ## Prerequisites | ||
| These instructions are tailored specifically for _Gadi_. To use _Spack_ on _Gadi_, you must have an NCI account. For instructions on how to set up an account, refer to [Set Up your NCI Account](/getting_started/set_up_nci_account). | ||
|
|
||
|
|
@@ -20,11 +19,11 @@ These instructions are tailored specifically for _Gadi_. To use _Spack_ on _Gadi | |
|
|
||
| ### Create a directory for Spack | ||
|
|
||
| Create a directory on the filesystem where _Spack_ will be installed (e.g. `/g/data/$PROJECT/$USER/spack/0.22`). | ||
| Create a directory on the filesystem where _Spack_ will be installed (e.g. `/g/data/$PROJECT/$USER/spack/0.22`). Use the `/g/data` filesystem if you wish to run the binaries on the compute nodes. | ||
|
|
||
| ``` | ||
| mkdir -p spack/0.22 | ||
| cd spack/0.22 | ||
| mkdir -p /g/data/$PROJECT/$USER/spack/0.22 | ||
| cd /g/data/$PROJECT/$USER/spack/0.22 | ||
| ``` | ||
|
|
||
| ### Clone the relevant git repositories | ||
|
|
@@ -49,19 +48,137 @@ ln -s -r -v spack-config/v0.22/gadi/* spack/etc/spack/ | |
|
|
||
| ## Test Spack (OPTIONAL) | ||
|
|
||
| To test _Spack_ we will create an `ACCESS-TEST` environment and build the relevant packages. Then, we will uninstall all the packages and remove the environment. | ||
| To test _Spack_ we will create an `ACCESS-TEST` environment and build the relevant packages. It takes approximately 30 minutes to compile. Then, we will uninstall all the packages and remove the environment. | ||
|
|
||
|
|
||
| ### Change to the Spack directory | ||
| ``` | ||
| cd /g/data/$PROJECT/$USER/spack/0.22 | ||
| ``` | ||
|
|
||
| ### Enable Spack | ||
|
|
||
| !!! warning | ||
| This step needs to be carried out for any new login or new shell environment. | ||
|
|
||
| ``` | ||
| module purge | ||
| cd spack/0.22 | ||
| . spack-config/spack-enable.bash | ||
| ``` | ||
|
|
||
| !!! warning | ||
| There is a space between the `.` and the path to the file, as we are sourcing the file. | ||
|
|
||
| ### Create the Spack managed environment | ||
|
|
||
| ``` | ||
| git clone https://github.com/ACCESS-NRI/ACCESS-TEST.git | ||
| spack env create test ACCESS-TEST/spack.yaml | ||
| ``` | ||
|
|
||
| <terminal-window> | ||
| <terminal-line data="input">spack env create test ACCESS-TEST/spack.yaml</terminal-line> | ||
| <terminal-line><span class="indigo bold">\==></span> Created environment <span class="cyan">test</span> in: <span class="cyan">/g/data/$PROJECT/$USER/spack/0.22/environments/test</span></terminal-line> | ||
| <terminal-line><span class="indigo bold">\==></span> Activate with: <span class="cyan">spack env activate test</span></terminal-line> | ||
| </terminal-window> | ||
|
|
||
| ### Start compiling | ||
|
|
||
| ``` | ||
| spack env activate -p test | ||
| spack find | ||
| ``` | ||
|
|
||
| <terminal-window> | ||
| <terminal-line data="input">spack env activate -p test</terminal-line> | ||
| <terminal-line data="input">[test]$ spack find</terminal-line> | ||
| <terminal-line>==> In environment test</terminal-line> | ||
| <terminal-line>==> 1 root specs</terminal-line> | ||
| <terminal-line> - [email protected]=2024.09.20</terminal-line> | ||
| <terminal-line></terminal-line> | ||
| <terminal-line>==> 0 installed packages</terminal-line> | ||
| </terminal-window> | ||
|
|
||
| ``` | ||
| spack concretize -f --fresh | ||
| spack install --verbose | ||
| spack install | ||
| ``` | ||
|
|
||
| <terminal-window> | ||
| <terminal-line data="input">[test]$ spack concretize -f --fresh</terminal-line> | ||
| <terminal-line>==> Concretized [email protected]=2024.09.20</terminal-line> | ||
| <terminal-line> - hmy75yl [email protected]=2024.09.20%[email protected]~deterministic build_system=bundle arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line>[e] 5elnsoi ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - kok5n7h ^[email protected]=2023.11.09%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - doeoclg ^[email protected]%[email protected]~guile build_system=generic arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - ntfunrm ^[email protected]%[email protected]~doc+pic+shared build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - vob7om3 ^[email protected]%[email protected]+blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared+szip+zstd build_system=autotools patches=0161eb8 arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - 357vng5 ^[email protected]%[email protected]~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - y7n7vkn ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - 55ipnye ^[email protected]%[email protected] build_system=autotools libs=shared,static arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - zrjfo56 ^[email protected]%[email protected]+avx2~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line>[e] rldyvqn ^[email protected]%[email protected]~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - 7a5olrr ^[email protected]%[email protected]+pic build_system=makefile libs=shared,static arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - wk4pvru ^[email protected]%[email protected]~ipo+pic+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - aumsrgz ^[email protected]%[email protected]~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8 arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - vrupasu ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - x4hw5jq ^[email protected]%[email protected]~ipo+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - wa2x7rh ^[email protected]%[email protected]+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - kzc7pcv ^[email protected]%[email protected]~programs build_system=makefile libs=shared,static arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line>[e] uvea7q2 ^[email protected]%[email protected] cppflags='-diag-disable=10441' ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line data="input">[test]$ spack install</terminal-line> | ||
| <terminal-line>[+] /usr (external glibc-2.28-5elnsoiqgcg5k5zmmwsp33bmnmaa3g5p)</terminal-line> | ||
| <terminal-line>[+] /apps/cmake/3.24.2 (external cmake-3.24.2-rldyvqnwmmg4cepzstrlmu75vx2wf6cj)</terminal-line> | ||
| <terminal-line>==> [email protected] : has external module in ['openmpi/4.1.5']</terminal-line> | ||
| <terminal-line>[+] /apps/openmpi/4.1.5 (external openmpi-4.1.5-uvea7q2exkjwz2v7azix3x4zfcddnc23)</terminal-line> | ||
| <terminal-line>==> Installing gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4 [4/19]</terminal-line> | ||
| <terminal-line>==> No binary for gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4 found: installing from source</terminal-line> | ||
| <terminal-line>==> Fetching https://mirror.spack.io/_source-cache/archive/dd/dd16fb1d67bfab79a72f5e8390735c49e3e8e70b4945a15ab1f81ddb78658fb3.tar.gz</terminal-line> | ||
| <terminal-line>==> No patches needed for gmake</terminal-line> | ||
| <terminal-line>==> gmake: Executing phase: 'install'</terminal-line> | ||
| <terminal-line>==> gmake: Successfully installed gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4</terminal-line> | ||
| <terminal-line> Stage: 3.54s. Install: 2m 15.82s. Post-install: 0.31s. Total: 2m 20.90s</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>==> Waiting for access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjg==> Installing access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc [19/19]</terminal-line> | ||
| <terminal-line>==> No binary for access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc found: installing from source</terminal-line> | ||
| <terminal-line>==> No patches needed for access-test</terminal-line> | ||
| <terminal-line>==> access-test: Executing phase: 'install'</terminal-line> | ||
| <terminal-line>==> access-test: Successfully installed access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc</terminal-line> | ||
| <terminal-line> Stage: 0.00s. Install: 0.00s. Post-install: 0.37s. Total: 1.77s</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-test-git.2024.09.20_2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc</terminal-line> | ||
| <terminal-line>==> Updating view at /g/data/$PROJECT/$USER/spack/0.22/environments/test/.spack-env/view</terminal-line> | ||
| </terminal-window> | ||
|
|
||
| !!! info | ||
| The full output has been truncated for brevity. | ||
|
|
||
| ``` | ||
| spack find | ||
| ``` | ||
|
|
||
| <terminal-window> | ||
| <terminal-line data="input">[test]$ spack find</terminal-line> | ||
| <terminal-line>==> In environment test</terminal-line> | ||
| <terminal-line>==> 1 root specs</terminal-line> | ||
| <terminal-line>[+] [email protected]=2024.09.20</terminal-line> | ||
| <terminal-line></terminal-line> | ||
| <terminal-line>==> Installed packages</terminal-line> | ||
| <terminal-line>-- linux-rocky8-x86_64 / [email protected] ------------------------</terminal-line> | ||
| <terminal-line>[email protected]=2024.09.20 [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]=2023.11.09</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected]</terminal-line> | ||
| <terminal-line>==> 19 installed packages</terminal-line> | ||
| </terminal-window> | ||
|
|
||
| ``` | ||
| spack uninstall --remove --all | ||
| spack env deactivate | ||
| spack env rm test | ||
|
|
@@ -73,7 +190,7 @@ rm -rf ACCESS-TEST | |
| Keep your Spack instance up-to-date by doing the following: | ||
|
|
||
| ``` | ||
| cd spack/0.22 | ||
| cd /g/data/$PROJECT/$USER/spack/0.22 | ||
| git -C spack fetch --all -Pp | ||
| git -C spack reset --hard origin/releases/v0.22 | ||
| git -C spack-config pull | ||
|
|
||