-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Updated content of 'How to build an ACCESS model'
- Loading branch information
1 parent
6d73976
commit a5b5c5c
Showing
1 changed file
with
60 additions
and
41 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,10 @@ | ||
| !!! danger | ||
| This page is tailored to experienced users and collaborators developing ACCESS models. | ||
|
|
||
| {% set om2_build_config = "https://github.com/ACCESS-NRI/ACCESS-OM2" %} | ||
| {% set esm1_5_build_config = "https://github.com/ACCESS-NRI/ACCESS-ESM1.5" %} | ||
| [gadi]: https://opus.nci.org.au/display/Help/0.+Welcome+to+Gadi#id-0.WelcometoGadi-Overview | ||
|
|
||
| # <div tabcontentfor="access-esm1.5">Build ACCESS-ESM1.5</div><div tabcontentfor="access-om2">Build ACCESS-OM2</div> | ||
|
|
||
| !!! tip | ||
| To tailor the following instructions to your desired model, please select it from the tabs below. | ||
|
|
||
|
|
@@ -12,6 +13,8 @@ | |
| <button id="access-om2">ACCESS-OM2</button> | ||
| </div> | ||
|
|
||
| # <div tabcontentfor="access-esm1.5">Build ACCESS-ESM1.5</div><div tabcontentfor="access-om2">Build ACCESS-OM2</div> | ||
|
|
||
| <div tabcontentfor="access-esm1.5" markdown> | ||
| [:fontawesome-brands-github:{: class="twemoji icon-before-text"} ACCESS-ESM1.5 build]({{esm1_5_build_config}}){: class="text-card"} | ||
| </div> | ||
|
|
@@ -22,6 +25,7 @@ | |
| ## About | ||
|
|
||
| The instructions below outline how to build ACCESS-<span tabcontentfor="access-esm1.5">ESM1.5</span><span tabcontentfor="access-om2">OM2</span> using the build-from-source package manager [Spack](https://spack.readthedocs.io).<br> | ||
|
|
||
| This build workflow is specifically designed to run on the [National Computating Infrastructure (NCI)](https://nci.org.au/about-us/who-we-are) supercomputer [_Gadi_][gadi]. | ||
|
|
||
| <div tabcontentfor="access-esm1.5" markdown> | ||
|
|
@@ -31,6 +35,8 @@ If you are looking for information on how to run ACCESS-ESM1.5 instead, refer to | |
| If you are looking for information on how to run ACCESS-OM2 instead, refer to [Run ACCESS-OM2](/models/run-a-model/run-access-om). | ||
| </div> | ||
|
|
||
| Portions of the following instructions are adapted from the [spack Developer Workflows Tutorial](https://spack-tutorial.readthedocs.io/en/latest/tutorial_developer_workflows.html). | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - **NCI account**<br> | ||
|
|
@@ -39,19 +45,20 @@ If you are looking for information on how to run ACCESS-OM2 instead, refer to [R | |
| - **_Spack_**<br> | ||
| To set up _Spack_ on _Gadi_, refer to [Set up Spack for building ACCESS models](/getting_started/spack/). | ||
|
|
||
| ## Spack | ||
|
|
||
| _Spack_ supports model development using a modified environment where the components that are being actively developed are compiled in a local source directory, and the rest built in the normal spack manner. | ||
|
|
||
| This guide is to assist developers who wish to modify one or more of the model components of ACCESS-OM2 and compile the modified code. Typically there is a development cycle, where the code is modified, compiled, tested and further modified based on testing. | ||
|
|
||
| It is assumed the modifications and compilation will be done on the [NCI HPC system `gadi`](https://opus.nci.org.au/display/Help/Gadi+User+Guide). | ||
|
|
||
| Much of this is adapted from the [spack Developer Workflows Tutorial](https://spack-tutorial.readthedocs.io/en/latest/tutorial_developer_workflows.html). | ||
| ## Navigate into your Spack directory | ||
| Navigate into the directory where you cloned the _Spack_ repositories during [_Spack_'s setup](/getting_started/spack/#create-a-directory-for-spack).<br> | ||
| This directory will be referred to as `<myspack>`. | ||
| To navigate into it run: | ||
| ``` | ||
| cd <myspack> | ||
| ``` | ||
|
|
||
| ## Enable spack | ||
|
|
||
| For every new login or new shell it is necessary to add your local copy of `spack` to your shell, as well as some other settings. | ||
| !!! warning | ||
| This step needs to be carried out for any new login or new shell | ||
|
|
||
| To add the `spack` command to your shell, as well as other settings, run: | ||
| ``` | ||
| . spack-config/spack-enable.bash | ||
| ``` | ||
|
|
@@ -61,27 +68,32 @@ For every new login or new shell it is necessary to add your local copy of `spac | |
|
|
||
| ## Development environment | ||
|
|
||
| Spack [has environments](https://spack.readthedocs.io/en/latest/environments.html), which is similar to many other related implementations such as conda environments and python virtual environments. | ||
| Spack has [environments](https://spack.readthedocs.io/en/latest/environments.html) that, in some ways, are similar to _Conda_ environments or _Python_ virtual environments. | ||
|
|
||
| Environments create an isolated operating environment within which `spack` can only see and access packages that are added to the environment. In this repository, the model to be built is defined using the `spack.yaml` environment file. | ||
| _Spack_ environments create an isolated operating environment where _Spack_ can only see and access specific packages that are added to that environment. This allows to build and deploy new packages in a coherent fashion. | ||
|
|
||
| ### Source code | ||
| The environment configuration is defined using the `spack.yaml` file, located in the _Spack_ environment directory. | ||
|
|
||
| There are two options for local source directory location: | ||
| ### Source code directory | ||
|
|
||
| 1. Use a path to an existing source code repository | ||
| 2. Let spack clone the code for you and place it in the environment directory | ||
| There are two options for local source directory location: | ||
|
|
||
| If option 1 is preferred the source code for the component(s) to be modified has to be available on the filesystem, e.g. using `git clone`, before the next steps. For more details, see the [Notes](#notes) section below. | ||
| - Use a path to an existing source code `git` repository | ||
| - Let _Spack_ clone the code for you and place it in the environment directory ??? | ||
|
|
||
| > [!Note] | ||
| > The choice of where the source code for an environment should reside depends on use case and personal prefrence. If an existing source code repository location is used in more than one environment it requires the code repository be kept in sync with the purpose of the development environment, e.g. two separate feature branches utilising two different development environments would require the developer to check out the correct branch when switching between development environments. It would also preclude building both environments simultaneously. | ||
| If option 1 is preferred the source code for the component(s) to be modified has to be available on the filesystem, e.g. using `git clone`, before the next steps. | ||
|
|
||
| ### Creating an environment | ||
| !!! info | ||
| The choice of where the source code ???for an environment should??? reside depends on use case and personal preference. If an existing source code repository location is used in more than one environment, it requires the code repository be kept in sync with the purpose of the development environment, e.g. two separate feature branches utilising two different development environments would require the developer to check out the correct branch when switching between development environments. It would also preclude building both environments simultaneously. ??? | ||
|
|
||
| These instructions use [independent environments](https://spack-tutorial.readthedocs.io/en/latest/tutorial_environments.html#creating-an-independent-environment) rather than managed environments. The [main difference](https://spack-tutorial.readthedocs.io/en/latest/tutorial_environments.html#managed-versus-independent-environments) is that managed environments live within the spack directory structure, independent environments are outside it, and must be referenced directly by their path. | ||
| ### Create a development environment | ||
|
|
||
| First step is choose a location for your development environment directory: it will only contain text files and possibly source code repositories. All compiled packages will be put in the directories defined in `spack-config` (defined with [`install_tree`](https://spack.readthedocs.io/en/latest/config_yaml.html#install-tree-root) in [`config.yaml`](https://github.com/ACCESS-NRI/spack-config/blob/main/common/config.yaml#L4)). | ||
| These instructions use [independent environments](https://spack-tutorial.readthedocs.io/en/latest/tutorial_environments.html#creating-an-independent-environment) rather than managed environments. The [main difference](https://spack-tutorial.readthedocs.io/en/latest/tutorial_environments.html#managed-versus-independent-environments) is that managed environments live within the _Spack_ directory ???(myspack?) and can be referenced by their names, whereas independent environments can be in any directory and must be referenced directly by their path. | ||
|
|
||
| The first step is to choose a location for your development environment directory. This directory will contain text files and possibly source code repositories.<br> | ||
| The development environment directory will be referred to as `<myspackdevenv>`. | ||
|
|
||
| All compiled packages will be put in the directories defined in `spack-config` (defined with [`install_tree`](https://spack.readthedocs.io/en/latest/config_yaml.html#install-tree-root) in [`config.yaml`](https://github.com/ACCESS-NRI/spack-config/blob/main/common/config.yaml#L4)). ??? | ||
|
|
||
| Choose a name that suits the aim of the work. Make a directory with that name, `cd` into that directory then type the command `spack env create -d .`. This creates a `spack` independent environment in the current directory. | ||
|
|
||
|
|
@@ -99,33 +111,39 @@ It is necessary after `spack develop` to call `spack concretize -f` to force spa | |
|
|
||
| Now the source code can be modified, and then compiled by invoking `spack install` | ||
|
|
||
| ## Example | ||
| ## Example: Developing a single model component | ||
|
|
||
| The instructions above are best understood by example. | ||
| In this example we will only change the [mom5 component](/models/model_components/ocean/#mom5), whereas all other ACCESS-<span tabcontentfor="access-esm1.5">ESM1.5</span><span tabcontentfor="access-om2">OM2</span> components and dependencies will remain the same as the official ACCESS-NRI release. | ||
|
|
||
| ### Developing a single model component | ||
| ### Create the Spack development environment | ||
|
|
||
| In this example the `mom5` component will be modified but all other components and dependencies remain the same. | ||
| To ensure we keep all dependecies the same as the ACCESS-NRI official release, we will create our _Spack_ development environment as a copy of the released <span tabcontentfor="access-esm1.5" markdown> | ||
| [ACCESS-ESM1.5 Spack environment](https://github.com/ACCESS-NRI/ACCESS-ESM1.5/blob/main/spack.yaml)</span><span tabcontentfor="access-om2" markdown>[ACCESS-OM2 Spack environment](https://github.com/ACCESS-NRI/ACCESS-OM2/blob/main/spack.yaml)</span>: | ||
|
|
||
| This example also uses the [`ACCESS-OM2` `spack.yaml` environment file](https://github.com/ACCESS-NRI/ACCESS-OM2/blob/main/spack.yaml) as an argument to the [`spack env create` command](https://spack.readthedocs.io/en/latest/command_index.html#spack-env-create). This makes the development environment the same as that for the released ACCESS-OM2 model that is referenced in the local copy of the repo. So it is necessary to clone [this repository](https://github.com/ACCESS-NRI/ACCESS-OM2/). | ||
| <pre><code>git clone <span tabcontentfor="access-esm1.5">{{esm1_5_build_config}}</span><span tabcontentfor="access-om2">{{om2_build_config}}</span>.git | ||
| spack env create -d <myspackdevenv> ACCESS-OM2/spack.yaml | ||
| spack env activate -p <myspackdevenv> | ||
| spack env status | ||
| spack develop [email protected]</code></pre> | ||
|
|
||
| 1. Create an environment with a name that reflects the purpose, activate, and set the `mom5` package to be a development environment: | ||
| ```bash | ||
| $ git clone https://github.com/ACCESS-NRI/ACCESS-OM2.git | ||
| $ mkdir mom5-dev | ||
| $ cd mom5-dev | ||
| $ spack env create -d . ../ACCESS-OM2/spack.yaml | ||
| <terminal-window> | ||
| <terminal-line data="input">git clone <span tabcontentfor="access-esm1.5">{{esm1_5_build_config}}</span><span tabcontentfor="access-om2">{{om2_build_config}}</span>.git</terminal-line> | ||
| <terminal-line>Cloning into 'ACCESS-OM2'...</terminal-line> | ||
| <terminal-line>remote: Enumerating objects: 375, done.</terminal-line> | ||
| <terminal-line>remote: Counting objects: 100% (200/200), done.</terminal-line> | ||
| <terminal-line>remote: Compressing objects: 100% (87/87), done.</terminal-line> | ||
| <terminal-line>remote: Total 375 (delta 137), reused 151 (delta 107), pack-reused 175 (from 1)</terminal-line> | ||
| <terminal-line>Receiving objects: 100% (375/375), 128.37 KiB | 2.38 MiB/s, done.</terminal-line> | ||
| <terminal-line>Resolving deltas: 100% (177/177), done.</terminal-line> | ||
|
|
||
| </terminal-window> | ||
| ==> Created environment in /g/data/.../envs/mom5-dev | ||
| ==> You can activate this environment with: | ||
| ==> spack env activate /g/data/.../envs/mom5-dev | ||
| $ spack env activate . | ||
| $ spack env status | ||
| ==> Using spack.yaml in current directory: /g/data/.../envs/mom5-dev | ||
| $ spack develop [email protected] | ||
| ==> Configuring spec [email protected]=2023.11.09 for development at path mom5 | ||
| ``` | ||
|
|
||
| 2. Concretize the updated environment. This will update what spack thinks is required to build the defined specs, in this case `access-om2` and `mom5`. Note that the `mom5` spec has a `dev_path` argument that points to the location of the sources it will use to build the package. | ||
| 1. Concretize the updated environment. This will update what spack thinks is required to build the defined specs, in this case `access-om2` and `mom5`. Note that the `mom5` spec has a `dev_path` argument that points to the location of the sources it will use to build the package. | ||
|
|
||
| ```bash | ||
| $ spack concretize -f | ||
|
|
@@ -351,5 +369,6 @@ $ spack develop -p ../../sources/MOM5/ [email protected] | |
| ``` | ||
|
|
||
| <custom-references> | ||
| - | ||
| - [https://spack.readthedocs.io/en/latest/](https://spack.readthedocs.io/en/latest/) | ||
| - [https://spack-tutorial.readthedocs.io/en/latest/tutorial_developer_workflows.html](https://spack-tutorial.readthedocs.io/en/latest/tutorial_developer_workflows.html) | ||
| </custom-references> | ||