From 9cbd608f4db51758edd893ef7cb0b000341f7ac0 Mon Sep 17 00:00:00 2001 From: Davide Marchegiani Date: Mon, 9 Sep 2024 15:29:16 +1000 Subject: [PATCH] Added How to build an ACCESS model page --- README.md | 2 +- docs/js/miscellaneous.js | 2 +- docs/models/configurations/access-esm.md | 2 +- docs/models/configurations/access-om.md | 6 +- docs/models/run-a-model/build_a_model.md | 578 ++++++++++++++++++++++ docs/models/run-a-model/index.md | 12 +- docs/models/run-a-model/run-access-esm.md | 4 + docs/models/run-a-model/run-access-om.md | 4 + mkdocs.yml | 3 +- 9 files changed, 604 insertions(+), 9 deletions(-) create mode 100644 docs/models/run-a-model/build_a_model.md diff --git a/README.md b/README.md index 050e89a9b..9375cd578 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ The following custom tags are available: |**Code block**|\```
 code block
```|\

 \code block\
\
|
code block
| |**[Admonition](https://squidfunk.github.io/mkdocs-material/reference/admonitions/#usage) (not collapsible)**|!!! warning
 this is a warning admonition|\

 this is a warning
 admonition
\
|![warning admonition](docs/assets/assets_for_readme/warning_admonition.png)| |**Collapsible [Admonition](https://squidfunk.github.io/mkdocs-material/reference/admonitions/#usage)**|??? warning title
 this is a collapsible warning admonition|\

 \
  \


   this is a collapsible
   warning admonition
  \


 \
\
|![collapsible warning admonition](docs/assets/assets_for_readme/collapsible_warning_admonition.gif)| -|**Tabs**|N/A|\

 \
 \
\

\

  Content for tab1
 \

 \

  Content for tab2
 \
|![tabs](docs/assets/assets_for_readme/tabs.gif)| +|**Tabs**|N/A|**Labels:**
\

 \
 \
\


**Contents:**
\

  Content for tab1
\

\

  Content for tab2
\


A content for a specific tab `id` can be included or not independently from the other contents.|![tabs](docs/assets/assets_for_readme/tabs.gif)| # License diff --git a/docs/js/miscellaneous.js b/docs/js/miscellaneous.js index eeb3cdaa3..f7d5174ca 100644 --- a/docs/js/miscellaneous.js +++ b/docs/js/miscellaneous.js @@ -86,7 +86,7 @@ function tabFunctionality() { button.addEventListener('click', (e) => openTab(e.currentTarget)); }) - // Add click event for links to tab IDs on the same page + // Add click event for links to tab IDs document.querySelectorAll('[href^="#"]:not([class^="md"])').forEach(el => { let href = el.getAttribute('href'); let tabEl = document.getElementById(href.slice(1,)) diff --git a/docs/models/configurations/access-esm.md b/docs/models/configurations/access-esm.md index 714c1c85d..630b2c629 100644 --- a/docs/models/configurations/access-esm.md +++ b/docs/models/configurations/access-esm.md @@ -9,7 +9,7 @@ This means it can simulate both the physical climate and global biogeochemical c [ACCESS-ESM1.5](https://www.publish.csiro.au/es/ES19035) [@Ziehn2020] is a fully-coupled climate model with land and ocean carbon cycle components. ACCESS-ESM1.5 was developed primarily to enable Australia to participate in the [Coupled Model Intercomparison Project Phase 6 (CMIP6)](https://wcrp-cmip.org/cmip6/) with an Earth System Model (ESM) version. -ACCESS-NRI has released ACCESS-ESM1.5 configurations as an adaptation of those originally developed by [CSIRO](https://www.csiro.au/en/research/environmental-impacts/climate-change/climate-science-centre) and [CLEX CMS](https://github.com/coecms/access-esm). +ACCESS-NRI has released [ACCESS-ESM1.5 configurations](https://github.com/ACCESS-NRI/access-esm1.5-configs) as an adaptation of those originally developed by [CSIRO](https://www.csiro.au/en/research/environmental-impacts/climate-change/climate-science-centre) and [CLEX CMS](https://github.com/coecms/access-esm). There are currently two supported configurations: diff --git a/docs/models/configurations/access-om.md b/docs/models/configurations/access-om.md index 685354032..4a8215ba7 100644 --- a/docs/models/configurations/access-om.md +++ b/docs/models/configurations/access-om.md @@ -29,9 +29,9 @@ Each configuration also has a biogeochemical (BGC) configuration that uses the [ ### Model Components
- - - + + +
- **Ocean**: [MOM5](/models/model_components/ocean#mom5).
diff --git a/docs/models/run-a-model/build_a_model.md b/docs/models/run-a-model/build_a_model.md new file mode 100644 index 000000000..be2afe65a --- /dev/null +++ b/docs/models/run-a-model/build_a_model.md @@ -0,0 +1,578 @@ +!!! danger + This page is tailored to experienced users and collaborators developing ACCESS models. + +{% set esm1_5_build_config = "https://github.com/ACCESS-NRI/ACCESS-ESM1.5" %} +{% set spack_setup = "/getting_started/spack" %} +[gadi]: https://opus.nci.org.au/display/Help/0.+Welcome+to+Gadi#id-0.WelcometoGadi-Overview + +# Build an ACCESS model + +## About + +The instructions below outline how to build an ACCESS model using the build-from-source package manager [Spack](https://spack.readthedocs.io).
+This build workflow is specifically designed to run on the [National Computating Infrastructure (NCI)](https://nci.org.au/about-us/who-we-are) supercomputer [_Gadi_][gadi]. + + +As an example, in the following instrutions we will show how to build a modified [mom5 component](/models/model_components/ocean/#mom5) for [ACCESS-ESM1.5](/models/configurations/access-esm/#access-esm15). All other components and packages (i.e., dependencies) will remain the same as the official [ACCESS-ESM1.5 release]({{esm1_5_build_config}}). + +!!! tip + The instructions below remain valid (with simple tweaks) for any model or package. + + +If you are looking for information on how to run a model, refer to the [Run a Model](/models/run-a-model) section. + +## Prerequisites + +- **NCI account**
+ Before building a model, you need to [Set Up your NCI Account](/getting_started/set_up_nci_account). + +- **_Spack_**
+ To set up _Spack_ on _Gadi_, refer to [Set up Spack for building ACCESS models]({{spack_setup}}). + +## Navigate into your Spack directory +Navigate into the directory where you cloned the _Spack_ repositories during [Spack's setup]({{spack_setup}}#create-a-directory-for-spack).
+The suggested directory is `/g/data/$PROJECT/$USER/spack/0.22`: +``` +cd /g/data/$PROJECT/$USER/spack/0.22 +``` + +## Enable spack + +!!! warning + This step needs to be carried out for any new login or new shell + +To add the `spack` command to your shell, as well as other settings, run: +``` +. spack-config/spack-enable.bash +``` + +!!! warning + There is a space between the `.` and the path to the file, as we are [sourcing](https://tldp.org/HOWTO/Bash-Prompt-HOWTO/x237.html#:~:text=When%20a%20file%20is%20sourced,the%20file%20they%20are%20in.) the file. + +## Create a Spack development environment + +_Spack_ has [environments](https://spack.readthedocs.io/en/latest/environments.html) that, in some ways, are similar to _Conda_ environments or _Python_ virtual environments: they create an isolated operating environment where _Spack_ can only see and access specific packages. This allows to build and deploy new packages in a coherent fashion. + +_Spack_ environments are located inside `/g/data/$PROJECT/$USER/spack/0.22/environments`, each in a separate directory. Each environment configuration is defined within its directory using the [`spack.yaml`](https://spack.readthedocs.io/en/latest/config_yaml.html#config-yaml) file. +{: id="spack-environment-folder"} + +!!! warning + _Spack_ environments' location can be changed within _Spack_ configuration files and the directory specified above represents the default location for a _Spack_ instance that has been set up following [Spack's setup instructions]({{spack_setup}}). + +### Create the environment + +To ensure we keep all dependecies the same as the official release, we will create our [_Spack_ development environment](https://spack.readthedocs.io/en/latest/environments.html#creating-a-managed-environment) as a copy of the released [ACCESS-ESM1.5 Spack environment]({{esm1_5_build_config}}/blob/main/spack.yaml).
+To create a _Spack_ development environment called `mom5_dev`, run: + +``` +git clone {{esm1_5_build_config}}.git +spack env create mom5_dev ACCESS-ESM1.5/spack.yaml +``` + + + git clone {{esm1_5_build_config}}.git + Cloning into 'ACCESS-ESM1.5'... + remote: Enumerating objects: 29, done. + remote: Counting objects: 100% (19/19), done. + remote: Compressing objects: 100% (17/17), done. + remote: Total 29 (delta 6), reused 3 (delta 1), pack-reused 10 (from 1) + Receiving objects: 100% (29/29), 16.40 KiB | 2.38 MiB/s, done. + Resolving deltas: 100% (6/6), done. + spack env create mom5_dev + \==> Created environment mom5_dev in: /g/data/$PROJECT/$USER/spack/0.22/environments/mom5_dev + \==> Activate with: spack env activate mom5_dev + + +The newly-created `mom5_dev` _Spack_ environment folder is `/g/data/$PROJECT/$USER/spack/0.22/environments/mom5_dev`. + +### Activate the environment +To activate the `mom5_dev` _Spack_ environment run: +``` +spack env activate -p mom5_dev +``` + + spack env activate -p mom5_dev + + + +## Compile Spack environment packages (optional) + +It is always good to first compile all the packages in the newly created _Spack_ environment as is (without changing). + +Compiling all the packages present in a _Spack_ environment is referred to as [installing the environment](https://spack.readthedocs.io/en/latest/environments.html#installing-an-environment). + +### Concretize the Spack environment +[Concretizing](https://spack.readthedocs.io/en/latest/environments.html#concretizing) the _Spack_ environment is necessary to force _Spack_ to update its knowledge of all the packages (their specs) within the environment. + +To concretize the `mom5_dev` environment run: +``` +spack concretize -f --fresh +``` + +!!! warning + This command might take a few minutes to complete + + + + spack concretize -f --fresh + + + \==> Concretized access-esm1p5@git.2024.05.1=2024.05.1 + + + - nkvasig access-esm1p5@git.2024.05.1=2024.05.1%intel@19.0.3.199 build_system=bundle arch=linux-rocky8-x86_64_v4 + + + - hhtnigw ^cice4@git.2024.05.21=access-esm1.5%intel@19.0.3.199 build_system=makefile arch=linux-rocky8-x86_64_v4 + + + - j6yscmm ^gmake@4.4.1%intel@19.0.3.199~guile build_system=generic arch=linux-rocky8-x86_64_v4 + + + - 5xcyy2h ^netcdf-fortran@4.5.2%intel@19.0.3.199~doc+pic+shared build_system=autotools patches=b050dbd arch=linux-rocky8-x86_64_v4 + + + - py3awb7 ^oasis3-mct@git.access-esm1.5_2024.05.24=access-esm1.5%intel@19.0.3.199~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64_v4 + + + - yfo7fum ^hdf5@1.10.11%intel@19.0.3.199~cxx~fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64_v4 + + + [e] vc4y4c6 ^cmake@3.24.2%intel@19.0.3.199~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64_v4 + + + - ugenh6g ^pkgconf@2.2.0%intel@19.0.3.199 build_system=autotools arch=linux-rocky8-x86_64_v4 + + + - h45fvyw ^zlib-ng@2.1.6%intel@19.0.3.199+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64_v4 + + + [e] ikhujrk ^openmpi@4.0.2%intel@19.0.3.199~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none patches=073477a,60ce20b romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64_v4 + + + [e] mqjolvb ^glibc@2.28%intel@19.0.3.199 build_system=autotools arch=linux-rocky8-x86_64_v4 + + + - oopqoqg ^mom5@git.access-esm1.5_2024.08.23=access-esm1.5%intel@19.0.3.199~deterministic~optimisation_report+restart_repro build_system=makefile type=ACCESS-CM arch=linux-rocky8-x86_64_v4 + + + - 725rz7c ^netcdf-c@4.7.4%intel@19.0.3.199~blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared~szip~zstd build_system=autotools arch=linux-rocky8-x86_64_v4 + + + - udr7pbn ^um7@git.2024.07.03=access-esm1.5%intel@19.0.3.199 build_system=generic optim=high arch=linux-rocky8-x86_64_v4 + + + - qy5w2d7 ^dummygrib@1.0%intel@19.0.3.199 build_system=makefile arch=linux-rocky8-x86_64_v4 + + + - ho2ie66 ^fcm@2021.05.0%intel@19.0.3.199 build_system=generic site=none arch=linux-rocky8-x86_64_v4 + + + - xalavwv ^gcom4@git.2024.05.28=access-esm1.5%intel@19.0.3.199+mpi build_system=generic arch=linux-rocky8-x86_64_v4 + + + +### Install the Spack environment + +To compile the packages in the `mom5_dev` environment run: +``` +spack install +``` + +!!! tip + It takes 30-40 mins to compile all the packages for the first time.
+ However, subsequent installations will re-use the built dependencies, therefore the total compilation time will be reduced. + + + spack install + + \==> Installing glibc-2.28-mqjolvbeskcnhz5chvtdshk4x4sfnycs [1/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/glibc-2.28-mqjolvbeskcnhz5chvtdshk4x4sfnycs + + + \==> Installing cmake-3.24.2-vc4y4c64s55j5u6kp37ciw2hcghuxhhc [2/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/cmake-3.24.2-vc4y4c64s55j5u6kp37ciw2hcghuxhhc + + + \==> Installing openmpi-4.0.2-ikhujrkyukytbkxxyk3mub44v63vuzfz [3/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/openmpi-4.0.2-ikhujrkyukytbkxxyk3mub44v63vuzfz + + + \==> Installing gmake-4.4.1-j6yscmmcn3qws7n35klote7rivw7foa6 [4/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/gmake-4.4.1-j6yscmmcn3qws7n35klote7rivw7foa6 + + + \==> Installing fcm-2021.05.0-ho2ie66tizhxpjjiilnrjnlnbi6safwq [5/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/fcm-2021.05.0-ho2ie66tizhxpjjiilnrjnlnbi6safwq + + + \==> Installing zlib-ng-2.1.6-h45fvywj47wc4uwa37mfzkdsqrgcqxux [6/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/zlib-ng-2.1.6-h45fvywj47wc4uwa37mfzkdsqrgcqxux + + + \==> Installing dummygrib-1.0-qy5w2d7tmsbmvnqng2xlopdkd4m2grvb [7/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/dummygrib-1.0-qy5w2d7tmsbmvnqng2xlopdkd4m2grvb + + + \==> Installing pkgconf-2.2.0-ugenh6g4dnhti4p6ktbkfku6pzlq5fkr [8/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/pkgconf-2.2.0-ugenh6g4dnhti4p6ktbkfku6pzlq5fkr + + + \==> Installing gcom4-git.2024.05.28=access-esm1.5-xalavwvyp3jv6emsnj7yecrqprwp3kag [9/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/gcom4-git.2024.05.28_access-esm1.5-xalavwvyp3jv6emsnj7yecrqprwp3kag + + + \==> Installing hdf5-1.10.11-yfo7fumh2agj6itfzqa6l2dpccrypp2l [10/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/hdf5-1.10.11-yfo7fumh2agj6itfzqa6l2dpccrypp2l + + + \==> Installing netcdf-c-4.7.4-725rz7cn7qupsi4egyeaix2crssvtoxp [11/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/netcdf-c-4.7.4-725rz7cn7qupsi4egyeaix2crssvtoxp + + + \==> Installing netcdf-fortran-4.5.2-5xcyy2h34vaq77ouwsgd6lfes5zycoii [12/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/netcdf-fortran-4.5.2-5xcyy2h34vaq77ouwsgd6lfes5zycoii + + + \==> Installing oasis3-mct-git.access-esm1.5_2024.05.24=access-esm1.5-py3awb76nw3lwjw5ea3uktmh2nm254gi [13/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/oasis3-mct-git.access-esm1.5_2024.05.24_access-esm1.5-py3awb76nw3lwjw5ea3uktmh2nm254gi + + + \==> Installing mom5-git.access-esm1.5_2024.08.23=access-esm1.5-oopqoqgqu65cybqht23l6m6coxbrpzqh [14/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/mom5-git.access-esm1.5_2024.08.23_access-esm1.5-oopqoqgqu65cybqht23l6m6coxbrpzqh + + + \==> Installing um7-git.2024.07.03=access-esm1.5-udr7pbnflpwzuawejuuc4xpmfuwtpc4x [15/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/um7-git.2024.07.03_access-esm1.5-udr7pbnflpwzuawejuuc4xpmfuwtpc4x + + + \==> Installing cice4-git.2024.05.21=access-esm1.5-hhtnigwxdyz7ta4dv3gvhwulze6hxqra [16/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/cice4-git.2024.05.21_access-esm1.5-hhtnigwxdyz7ta4dv3gvhwulze6hxqra + + + \==> Installing access-esm1p5-git.2024.05.1=2024.05.1-nkvasig2zrq2ocz6evva6bmurdq7nh3h [17/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/access-esm1p5-git.2024.05.1_2024.05.1-nkvasig2zrq2ocz6evva6bmurdq7nh3h + + + \==> Updating view at /home/565/$USER/myspack/environments/mom5_dev/.spack-env/view + + + +!!! info + The full output has been truncated for brevity. + +## Create development package + +When you develop a package within a _Spack_ environment, _Spack_ needs to know that the desired package is marked as "in development", and be able to access its source code.
+This is done through the [`spack develop`](https://spack.readthedocs.io/en/latest/command_index.html#spack-develop) command. + +### Mark package as a development package + +There are in general two cases that influence the command to run to mark a package as "in development", depending on the state of the package's source code: + +1. [The new source code already exists](#case1) +2. [The new source code does not exist yet](#case2) + +#### Case 1. The new source code already exists {: id='case1'} +If the new source code for the development package already exists in the filesystem (this also includes cases when the new source code is in a `git` repo that can be cloned), to mark a package as "in development" we will need two things: + +- A name for the package new tag +- The path of the new source code + +For example, to mark the `mom5` package as a development package using the `development_version` tag and with the new source code in the `/path/to/mom5/new/source/code` folder, we can run: +``` +spack develop mom5@development_version --path /path/to/mom5/new/source/code +``` +!!! tip + This command should not display any output + +This adds the following lines at the end of the `spack.yaml` file inside the [environment's folder](#spack-environment-folder): +``` +develop: + mom5: + spec: mom5@=development_version + path: /path/to/mom5/new/source/code +``` + +!!! warning + Care needs to be taken when multiple _Spack_ development environments point to the same source code location. If these environments require different independent changes of the source code, the user needs to make sure to sync the source code version (e.g., using different `git` branches for the different versions of the source code) with the desired one when switching between development environments.
+ This would still prevent building both environments simultaneously. + + +#### Case 2. The new source does not exist {: id='case2'} +If the source code does not yet exist, _Spack_ can create it as a copy of the current source code version, that can be later modified. + +In this case we need to tell _Spack_ which package tag (version) to copy as a starting base for the new source code. + +For example, to retrieve the tag for the original `mom5` package used in the `mom5_dev` _Spack_ environment, we can run: +``` +spack find mom5 +``` + + spack find mom5 + mom5@git.access-esm1.5_2024.08.23=access-esm1.5 + + +To mark the `mom5` as a development package and copy the `mom5@git.access-esm1.5_2024.08.23=access-esm1.5` version as a base copy, we can run: + +``` +spack develop mom5@git.access-esm1.5_2024.08.23=access-esm1.5 +``` +!!! tip + This command should not display any output + +This adds the following lines at the end of the `spack.yaml` file inside the [environment's folder](#spack-environment-folder): +``` +develop: + mom5: + spec: mom5@git.access-esm1.5_2024.08.23=access-esm1.5 +``` +The source code is automatically copied inside the [environment's folder](#spack-environment-folder). + +!!! warning + As you might notice, differently from [Case 1](#case1), there is no `path` specification inside the `develop` portion of the `spack.yaml` environment configuration file.
+ This means that _Spack_ automatically expects to find the source code inside the [environment's folder](#spack-environment-folder). + +## Compile modified Spack environment packages + +After the source code has been modified, the new package can be built following the same steps listed in [Compile Spack environment packages](#compile-spack-environment-packages-optional): + +``` +spack concretize -f --fresh +spack install +``` +!!! warning + Even though the `spack install` command this time will only build the development package, it might still take a long time to complete, depending on the specific package. + + + + spack concretize -f --fresh + + + \==> Concretized access-esm1p5@git.2024.05.1=2024.05.1 + + + - aysea5r access-esm1p5@git.2024.05.1=2024.05.1%intel@19.0.3.199 build_system=bundle arch=linux-rocky8-x86_64_v4 + + + [+] hhtnigw ^cice4@git.2024.05.21=access-esm1.5%intel@19.0.3.199 build_system=makefile arch=linux-rocky8-x86_64_v4 + + + [+] j6yscmm ^gmake@4.4.1%intel@19.0.3.199~guile build_system=generic arch=linux-rocky8-x86_64_v4 + + + [+] 5xcyy2h ^netcdf-fortran@4.5.2%intel@19.0.3.199~doc+pic+shared build_system=autotools patches=b050dbd arch=linux-rocky8-x86_64_v4 + + + [+] py3awb7 ^oasis3-mct@git.access-esm1.5_2024.05.24=access-esm1.5%intel@19.0.3.199~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64_v4 + + + [+] yfo7fum ^hdf5@1.10.11%intel@19.0.3.199~cxx~fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64_v4 + + + [e] vc4y4c6 ^cmake@3.24.2%intel@19.0.3.199~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64_v4 + + + [+] ugenh6g ^pkgconf@2.2.0%intel@19.0.3.199 build_system=autotools arch=linux-rocky8-x86_64_v4 + + + [+] h45fvyw ^zlib-ng@2.1.6%intel@19.0.3.199+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64_v4 + + + [e] ikhujrk ^openmpi@4.0.2%intel@19.0.3.199~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none patches=073477a,60ce20b romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64_v4 + + + [e] mqjolvb ^glibc@2.28%intel@19.0.3.199 build_system=autotools arch=linux-rocky8-x86_64_v4 + + + - oopqoqg ^mom5@git.access-esm1.5_2024.08.23=access-esm1.5%intel@19.0.3.199~deterministic~optimisation_report+restart_repro build_system=makefile dev_path=/path/to/source/code/for/mom5 type=ACCESS-CM arch=linux-rocky8-x86_64_v4 + + + [+] 725rz7c ^netcdf-c@4.7.4%intel@19.0.3.199~blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared~szip~zstd build_system=autotools arch=linux-rocky8-x86_64_v4 + + + [+] udr7pbn ^um7@git.2024.07.03=access-esm1.5%intel@19.0.3.199 build_system=generic optim=high arch=linux-rocky8-x86_64_v4 + + + [+] qy5w2d7 ^dummygrib@1.0%intel@19.0.3.199 build_system=makefile arch=linux-rocky8-x86_64_v4 + + + [+] ho2ie66 ^fcm@2021.05.0%intel@19.0.3.199 build_system=generic site=none arch=linux-rocky8-x86_64_v4 + + + [+] xalavwv ^gcom4@git.2024.05.28=access-esm1.5%intel@19.0.3.199+mpi build_system=generic arch=linux-rocky8-x86_64_v4 + + spack install + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/glibc-2.28-mqjolvbeskcnhz5chvtdshk4x4sfnycs + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/cmake-3.24.2-vc4y4c64s55j5u6kp37ciw2hcghuxhhc + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/openmpi-4.0.2-ikhujrkyukytbkxxyk3mub44v63vuzfz + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/gmake-4.4.1-j6yscmmcn3qws7n35klote7rivw7foa6 + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/fcm-2021.05.0-ho2ie66tizhxpjjiilnrjnlnbi6safwq + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/zlib-ng-2.1.6-h45fvywj47wc4uwa37mfzkdsqrgcqxux + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/dummygrib-1.0-qy5w2d7tmsbmvnqng2xlopdkd4m2grvb + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/pkgconf-2.2.0-ugenh6g4dnhti4p6ktbkfku6pzlq5fkr + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/gcom4-git.2024.05.28_access-esm1.5-xalavwvyp3jv6emsnj7yecrqprwp3kag + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/hdf5-1.10.11-yfo7fumh2agj6itfzqa6l2dpccrypp2l + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/netcdf-c-4.7.4-725rz7cn7qupsi4egyeaix2crssvtoxp + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/netcdf-fortran-4.5.2-5xcyy2h34vaq77ouwsgd6lfes5zycoii + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/oasis3-mct-git.access-esm1.5_2024.05.24_access-esm1.5-py3awb76nw3lwjw5ea3uktmh2nm254gi + + + \==> Installing mom5-git.access-esm1.5_2024.08.23=access-esm1.5-l34w7is54xzer7s4ztvb5ymgjbtduknh [14/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/mom5-git.access-esm1.5_2024.08.23_access-esm1.5-l34w7is54xzer7s4ztvb5ymgjbtduknh + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/um7-git.2024.07.03_access-esm1.5-udr7pbnflpwzuawejuuc4xpmfuwtpc4x + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/cice4-git.2024.05.21_access-esm1.5-hhtnigwxdyz7ta4dv3gvhwulze6hxqra + + + \==> Installing access-esm1p5-git.2024.05.1_2024.05.1-nkvasig2zrq2ocz6evva6bmurdq7nh3h [17/17] + + + [+] /home/565/$USER/myspack/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/access-esm1p5-git.2024.05.1_2024.05.1-nkvasig2zrq2ocz6evva6bmurdq7nh3h + + + \==> Updating view at /home/565/$USER/myspack/environments/mom5_dev/.spack-env/view + + + +!!! info + The full output has been truncated for brevity. + +## Output directory for compiled packages +All compiled packages will be placed in directories having the following format: `///`. + +`` depends on the [`install_tree`](https://spack.readthedocs.io/en/latest/config_yaml.html#install-tree-root) configuration field. _Spack_ reads this configuration field from files in several directories, following [Spack's configuration scopes](https://spack.readthedocs.io/en/latest/configuration.html#configuration-scopes). + +!!! warning + For instances of _Spack_ on _Gadi_ you should ignore the **system** scope. + +!!! tip + For the _Spack_ instance obtained through the [Spack's setup instructions]({{spack_setup}}), `$(prefix)` corresponds to the `/g/data/$PROJECT/$USER/spack/0.22/spack` directory. + +For the example above, `mom5_dev` _Spack_ environment's configuration file (`spack.yaml`) contains the following lines that fall in the **environment** scope: +```yaml +config: + install_tree: + root: $spack/../restricted/ukmo/release +``` +!!! tip + `$spack` refers to the `$(prefix)` directory above. + +This means the packages that we just built in this example can be found under `/g/data/$PROJECT/$USER/spack/0.22/spack/../restricted/ukmo/release///`. + +## Troubleshooting build errors + +Sometimes you might get errors while compiling the packages.
+_Spack_ prints out the error message and generates a full build log that can be viewed by the user. The location of the build log is shown at the end of the error message. + +For example, if we try to install the `mom5_dev` environment with an error in the new `mom5` source code (in this example a `use` statement in the `/src/accessom_coupler/ocean_solo.F90` file has been purposely commented out to force an error in compilation), we might get an output error similar to the following: + +
...
+ >> 415    /home/565/dm5220/myspack/environments/mom5_dev/mom5/src/access_coupler/ocean_solo.F90:224: undefined reference to `constant s_init_'
+ >> 416    make: *** [Makefile:931: fms_ACCESS-CM.x] Error 1
+...
+See build log for details:
+  /scratch/$PROJECT/$USER/tmp/path/to/the/spack-stage-mom5-git.access-esm1.5_2024.08.23_access-esm1.5-l34w7is54xzer7s4ztvb5ymgjbtduknh/spack-build-out.txt
+...
+==> Error: access-esm1p5-git.2024.05.1=2024.05.1-aysea5r7rbwy22lluvl64baperlokktv: Package was not installed
+==> Error: Installation request failed.  Refer to reported errors for failing package(s).
+ +In this case the error might be obvious already from the error message but, for more information, the build log can be examined. + + +- [https://spack.readthedocs.io/en/latest/](https://spack.readthedocs.io/en/latest/) +- [https://spack-tutorial.readthedocs.io/en/latest/tutorial_developer_workflows.html](https://spack-tutorial.readthedocs.io/en/latest/tutorial_developer_workflows.html) + + + + diff --git a/docs/models/run-a-model/index.md b/docs/models/run-a-model/index.md index 320060dd7..4e11d522d 100644 --- a/docs/models/run-a-model/index.md +++ b/docs/models/run-a-model/index.md @@ -16,7 +16,7 @@ If you are unsure which ACCESS model is the right choice for your experiment, ta
ACCESS-ESM
-
+
Run ACCESS-ESM
@@ -24,8 +24,16 @@ If you are unsure which ACCESS model is the right choice for your experiment, ta
ACCESS-OM
-
+
Run ACCESS-OM
+ +
+ Build a model +
+
+ Build an ACCESS model +
+
\ No newline at end of file diff --git a/docs/models/run-a-model/run-access-esm.md b/docs/models/run-a-model/run-access-esm.md index 3f6332d5e..44b315533 100644 --- a/docs/models/run-a-model/run-access-esm.md +++ b/docs/models/run-a-model/run-access-esm.md @@ -640,6 +640,10 @@ Each of [{{ model }} components][model components] contains additional configura These configuration options are specified in files located inside a subfolder of the _control_ directory, named according to the submodel's `name` specified in the `config.yaml` `submodels` section (e.g., configuration options for the _ocean_ component are in the `~/access-esm/preindustrial+concentrations/ocean` directory).
To modify these options please refer to the User Guide of the respective model component. +### Create a custom {{ model }} build +All the executables needed to run {{ model }} are pre-built into independent configurations using _Spack_.
+To customise {{ model }}'s build (for example to run {{ model }} with changes in the source code of one of its component), refer to [Build {{ model }}](/models/run-a-model/build_a_model#{{model|lower}}). + ### Controlling model output Selecting the variables to save from a simulation can be a balance between enabling future analysis and minimising storage requirements. The choice and frequency of variables saved by each model can be configured from within each submodel's _control_ directory. diff --git a/docs/models/run-a-model/run-access-om.md b/docs/models/run-a-model/run-access-om.md index d9e66cce5..b76fd5cf7 100644 --- a/docs/models/run-a-model/run-access-om.md +++ b/docs/models/run-a-model/run-access-om.md @@ -563,6 +563,10 @@ These options are typically useful to modify the physics used in the model, the These configuration options are specified in files located inside a subfolder of the _control_ directory, named according to the submodel's `name` specified in the `config.yaml` `submodels` section (e.g., configuration options for the _ocean_ component are in the `~/access-om2/1deg_jra55_ryf/ocean` directory).
To modify these options please refer to the User Guide of the respective model component. +### Create a custom {{ model }} build +All the executables needed to run {{ model }} are pre-built into independent configurations using _Spack_.
+To customise {{ model }}'s build (for example to run {{ model }} with changes in the source code of one of its component), refer to [Build {{ model }}](/models/run-a-model/build_a_model#{{model|lower}}). + ## Get Help If you have questions or need help regarding {{ model }}, consider creating a topic in the [COSIMA category of the ACCESS-Hive Forum](https://forum.access-hive.org.au/c/cosima/29).
diff --git a/mkdocs.yml b/mkdocs.yml index 052f04c27..541dd505a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -123,6 +123,7 @@ nav: - Run ACCESS-CM: models/run-a-model/run-access-cm.md - Run ACCESS-ESM: models/run-a-model/run-access-esm.md - Run ACCESS-OM: models/run-a-model/run-access-om.md + - Build an ACCESS Model: models/run-a-model/build_a_model.md - Data and Model Evaluation: - model_evaluation/index.md @@ -212,6 +213,6 @@ extra_css: extra_javascript: - https://unpkg.com/tablesort@5.3.0/dist/tablesort.min.js # For tablesort functionality - - https://cdn.jsdelivr.net/gh/atteggiani/animated-terminal@2.1/animated-terminal.min.js # Terminal animations + - https://cdn.jsdelivr.net/gh/atteggiani/animated-terminal@3.0/animated-terminal.min.js # Terminal animations - js/custom-tags.js - js/miscellaneous.js