.
diff --git a/COPYING.LESSER b/COPYING.LESSER
deleted file mode 100644
index 65c5ca88a6..0000000000
--- a/COPYING.LESSER
+++ /dev/null
@@ -1,165 +0,0 @@
- GNU LESSER GENERAL PUBLIC LICENSE
- Version 3, 29 June 2007
-
- Copyright (C) 2007 Free Software Foundation, Inc.
- Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
-
-
- This version of the GNU Lesser General Public License incorporates
-the terms and conditions of version 3 of the GNU General Public
-License, supplemented by the additional permissions listed below.
-
- 0. Additional Definitions.
-
- As used herein, "this License" refers to version 3 of the GNU Lesser
-General Public License, and the "GNU GPL" refers to version 3 of the GNU
-General Public License.
-
- "The Library" refers to a covered work governed by this License,
-other than an Application or a Combined Work as defined below.
-
- An "Application" is any work that makes use of an interface provided
-by the Library, but which is not otherwise based on the Library.
-Defining a subclass of a class defined by the Library is deemed a mode
-of using an interface provided by the Library.
-
- A "Combined Work" is a work produced by combining or linking an
-Application with the Library. The particular version of the Library
-with which the Combined Work was made is also called the "Linked
-Version".
-
- The "Minimal Corresponding Source" for a Combined Work means the
-Corresponding Source for the Combined Work, excluding any source code
-for portions of the Combined Work that, considered in isolation, are
-based on the Application, and not on the Linked Version.
-
- The "Corresponding Application Code" for a Combined Work means the
-object code and/or source code for the Application, including any data
-and utility programs needed for reproducing the Combined Work from the
-Application, but excluding the System Libraries of the Combined Work.
-
- 1. Exception to Section 3 of the GNU GPL.
-
- You may convey a covered work under sections 3 and 4 of this License
-without being bound by section 3 of the GNU GPL.
-
- 2. Conveying Modified Versions.
-
- If you modify a copy of the Library, and, in your modifications, a
-facility refers to a function or data to be supplied by an Application
-that uses the facility (other than as an argument passed when the
-facility is invoked), then you may convey a copy of the modified
-version:
-
- a) under this License, provided that you make a good faith effort to
- ensure that, in the event an Application does not supply the
- function or data, the facility still operates, and performs
- whatever part of its purpose remains meaningful, or
-
- b) under the GNU GPL, with none of the additional permissions of
- this License applicable to that copy.
-
- 3. Object Code Incorporating Material from Library Header Files.
-
- The object code form of an Application may incorporate material from
-a header file that is part of the Library. You may convey such object
-code under terms of your choice, provided that, if the incorporated
-material is not limited to numerical parameters, data structure
-layouts and accessors, or small macros, inline functions and templates
-(ten or fewer lines in length), you do both of the following:
-
- a) Give prominent notice with each copy of the object code that the
- Library is used in it and that the Library and its use are
- covered by this License.
-
- b) Accompany the object code with a copy of the GNU GPL and this license
- document.
-
- 4. Combined Works.
-
- You may convey a Combined Work under terms of your choice that,
-taken together, effectively do not restrict modification of the
-portions of the Library contained in the Combined Work and reverse
-engineering for debugging such modifications, if you also do each of
-the following:
-
- a) Give prominent notice with each copy of the Combined Work that
- the Library is used in it and that the Library and its use are
- covered by this License.
-
- b) Accompany the Combined Work with a copy of the GNU GPL and this license
- document.
-
- c) For a Combined Work that displays copyright notices during
- execution, include the copyright notice for the Library among
- these notices, as well as a reference directing the user to the
- copies of the GNU GPL and this license document.
-
- d) Do one of the following:
-
- 0) Convey the Minimal Corresponding Source under the terms of this
- License, and the Corresponding Application Code in a form
- suitable for, and under terms that permit, the user to
- recombine or relink the Application with a modified version of
- the Linked Version to produce a modified Combined Work, in the
- manner specified by section 6 of the GNU GPL for conveying
- Corresponding Source.
-
- 1) Use a suitable shared library mechanism for linking with the
- Library. A suitable mechanism is one that (a) uses at run time
- a copy of the Library already present on the user's computer
- system, and (b) will operate properly with a modified version
- of the Library that is interface-compatible with the Linked
- Version.
-
- e) Provide Installation Information, but only if you would otherwise
- be required to provide such information under section 6 of the
- GNU GPL, and only to the extent that such information is
- necessary to install and execute a modified version of the
- Combined Work produced by recombining or relinking the
- Application with a modified version of the Linked Version. (If
- you use option 4d0, the Installation Information must accompany
- the Minimal Corresponding Source and Corresponding Application
- Code. If you use option 4d1, you must provide the Installation
- Information in the manner specified by section 6 of the GNU GPL
- for conveying Corresponding Source.)
-
- 5. Combined Libraries.
-
- You may place library facilities that are a work based on the
-Library side by side in a single library together with other library
-facilities that are not Applications and are not covered by this
-License, and convey such a combined library under terms of your
-choice, if you do both of the following:
-
- a) Accompany the combined library with a copy of the same work based
- on the Library, uncombined with any other library facilities,
- conveyed under the terms of this License.
-
- b) Give prominent notice with the combined library that part of it
- is a work based on the Library, and explaining where to find the
- accompanying uncombined form of the same work.
-
- 6. Revised Versions of the GNU Lesser General Public License.
-
- The Free Software Foundation may publish revised and/or new versions
-of the GNU Lesser General Public License from time to time. Such new
-versions will be similar in spirit to the present version, but may
-differ in detail to address new problems or concerns.
-
- Each version is given a distinguishing version number. If the
-Library as you received it specifies that a certain numbered version
-of the GNU Lesser General Public License "or any later version"
-applies to it, you have the option of following the terms and
-conditions either of that published version or of any later version
-published by the Free Software Foundation. If the Library as you
-received it does not specify a version number of the GNU Lesser
-General Public License, you may choose any version of the GNU Lesser
-General Public License ever published by the Free Software Foundation.
-
- If the Library as you received it specifies that a proxy can decide
-whether future versions of the GNU Lesser General Public License shall
-apply, that proxy's public statement of acceptance of any version is
-permanent authorization for you to choose that version for the
-Library.
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000000..2d1d23e16c
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2010, Met Office.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+ list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+ contributors may be used to endorse or promote products derived from
+ this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/MANIFEST.in b/MANIFEST.in
index 329cf79c5d..354b92d735 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -17,8 +17,7 @@ exclude .readthedocs.yml
exclude CHANGES
exclude CODE_OF_CONDUCT.md
exclude codecov.yml
-include COPYING
-include COPYING.LESSER
+include LICENSE
exclude Makefile
exclude noxfile.py
diff --git a/README.md b/README.md
index 53d24b0162..f857608718 100644
--- a/README.md
+++ b/README.md
@@ -15,7 +15,7 @@
| 💬 Community | [](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) [](https://github.com/SciTools/iris/discussions) [](https://twitter.com/scitools_iris) |
| 📖 Documentation | [](https://scitools-iris.readthedocs.io/en/latest/?badge=latest) |
| 📈 Health | [](https://codecov.io/gh/SciTools/iris) |
-| ✨ Meta | [](https://github.com/psf/black) [](https://numpy.org/neps/nep-0029-deprecation_policy.html) [](https://github.com/SciTools/iris/blob/main/COPYING.LESSER) [](https://anaconda.org/conda-forge/iris) |
+| ✨ Meta | [](https://github.com/psf/black) [](https://numpy.org/neps/nep-0029-deprecation_policy.html) [](https://github.com/SciTools/iris/blob/main/LICENSE) [](https://anaconda.org/conda-forge/iris) |
| 📦 Package | [](https://doi.org/10.5281/zenodo.595182) [](https://anaconda.org/conda-forge/iris) [](https://pypi.org/project/scitools-iris/) [](https://pypi.org/project/scitools-iris/) |
| 🧰 Repo | [](https://github.com/SciTools/iris/commits/main) [](https://github.com/SciTools/iris/graphs/contributors) [](https://github.com/SciTools/iris/releases) |
| |
diff --git a/benchmarks/asv.conf.json b/benchmarks/asv.conf.json
index faa7f6daee..fab5bcb44e 100644
--- a/benchmarks/asv.conf.json
+++ b/benchmarks/asv.conf.json
@@ -4,7 +4,6 @@
"project_url": "https://github.com/SciTools/iris",
"repo": "..",
"environment_type": "conda-delegated",
- "conda_channels": ["conda-forge", "defaults"],
"show_commit_url": "http://github.com/scitools/iris/commit/",
"branches": ["upstream/main"],
diff --git a/benchmarks/asv_delegated_conda.py b/benchmarks/asv_delegated_conda.py
index 250a4e032d..a60cb7f2b7 100644
--- a/benchmarks/asv_delegated_conda.py
+++ b/benchmarks/asv_delegated_conda.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
ASV plug-in providing an alternative :class:`asv.plugins.conda.Conda`
subclass that manages the Conda environment via custom user scripts.
@@ -66,6 +65,8 @@ def __init__(
ignored.append("`requirements`")
if tagged_env_vars:
ignored.append("`tagged_env_vars`")
+ if conf.conda_channels:
+ ignored.append("conda_channels")
if conf.conda_environment_file:
ignored.append("`conda_environment_file`")
message = (
@@ -75,6 +76,8 @@ def __init__(
log.warning(message)
requirements = {}
tagged_env_vars = {}
+ # All that is required to create ASV's bare-bones environment.
+ conf.conda_channels = ["defaults"]
conf.conda_environment_file = None
super().__init__(conf, python, requirements, tagged_env_vars)
diff --git a/benchmarks/benchmarks/__init__.py b/benchmarks/benchmarks/__init__.py
index c86682ca4a..61983a969f 100644
--- a/benchmarks/benchmarks/__init__.py
+++ b/benchmarks/benchmarks/__init__.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""Common code for benchmarks."""
from os import environ
import resource
diff --git a/benchmarks/benchmarks/aux_factory.py b/benchmarks/benchmarks/aux_factory.py
index 4cc4f6c70a..7d1e266c78 100644
--- a/benchmarks/benchmarks/aux_factory.py
+++ b/benchmarks/benchmarks/aux_factory.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
AuxFactory benchmark tests.
diff --git a/benchmarks/benchmarks/coords.py b/benchmarks/benchmarks/coords.py
index 3107dcf077..b6f56b997f 100644
--- a/benchmarks/benchmarks/coords.py
+++ b/benchmarks/benchmarks/coords.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Coord benchmark tests.
diff --git a/benchmarks/benchmarks/cperf/__init__.py b/benchmarks/benchmarks/cperf/__init__.py
index fb311c44dc..7adba01b60 100644
--- a/benchmarks/benchmarks/cperf/__init__.py
+++ b/benchmarks/benchmarks/cperf/__init__.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Benchmarks for the CPerf scheme of the UK Met Office's NG-VAT project.
@@ -53,7 +52,7 @@ def setup(self, file_type, three_d, three_times):
if three_d:
create_kwargs["n_levels"] = 71
- # Will re-use a file if already present.
+ # Will reuse a file if already present.
file_path = make_cubesphere_testfile(**create_kwargs)
else:
diff --git a/benchmarks/benchmarks/cperf/equality.py b/benchmarks/benchmarks/cperf/equality.py
index 47eb255513..f27558a5ed 100644
--- a/benchmarks/benchmarks/cperf/equality.py
+++ b/benchmarks/benchmarks/cperf/equality.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Equality benchmarks for the CPerf scheme of the UK Met Office's NG-VAT project.
"""
diff --git a/benchmarks/benchmarks/cperf/load.py b/benchmarks/benchmarks/cperf/load.py
index 04bb7e1a61..efbd497e2e 100644
--- a/benchmarks/benchmarks/cperf/load.py
+++ b/benchmarks/benchmarks/cperf/load.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
File loading benchmarks for the CPerf scheme of the UK Met Office's NG-VAT project.
"""
diff --git a/benchmarks/benchmarks/cperf/save.py b/benchmarks/benchmarks/cperf/save.py
index 2eb60e2ab5..957b28e3fd 100644
--- a/benchmarks/benchmarks/cperf/save.py
+++ b/benchmarks/benchmarks/cperf/save.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
File saving benchmarks for the CPerf scheme of the UK Met Office's NG-VAT project.
"""
diff --git a/benchmarks/benchmarks/cube.py b/benchmarks/benchmarks/cube.py
index 5889ce872b..ceacb4e86c 100644
--- a/benchmarks/benchmarks/cube.py
+++ b/benchmarks/benchmarks/cube.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Cube benchmark tests.
diff --git a/benchmarks/benchmarks/experimental/__init__.py b/benchmarks/benchmarks/experimental/__init__.py
index f16e400bce..81fb222916 100644
--- a/benchmarks/benchmarks/experimental/__init__.py
+++ b/benchmarks/benchmarks/experimental/__init__.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Benchmark tests for the experimental module.
diff --git a/benchmarks/benchmarks/experimental/ugrid/__init__.py b/benchmarks/benchmarks/experimental/ugrid/__init__.py
index 2e40c525a6..1fa8b82d67 100644
--- a/benchmarks/benchmarks/experimental/ugrid/__init__.py
+++ b/benchmarks/benchmarks/experimental/ugrid/__init__.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Benchmark tests for the experimental.ugrid module.
diff --git a/benchmarks/benchmarks/experimental/ugrid/regions_combine.py b/benchmarks/benchmarks/experimental/ugrid/regions_combine.py
index c5f8fb564e..6d62cf9cd5 100644
--- a/benchmarks/benchmarks/experimental/ugrid/regions_combine.py
+++ b/benchmarks/benchmarks/experimental/ugrid/regions_combine.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Benchmarks stages of operation of the function
:func:`iris.experimental.ugrid.utils.recombine_submeshes`.
@@ -23,14 +22,14 @@
from iris.experimental.ugrid import PARSE_UGRID_ON_LOAD
from iris.experimental.ugrid.utils import recombine_submeshes
-from ... import TrackAddedMemoryAllocation
+from ... import TrackAddedMemoryAllocation, on_demand_benchmark
from ...generate_data.ugrid import make_cube_like_2d_cubesphere
class MixinCombineRegions:
# Characterise time taken + memory-allocated, for various stages of combine
# operations on cubesphere-like test data.
- params = [4, 500]
+ params = [50, 500]
param_names = ["cubesphere-N"]
def _parametrised_cache_filename(self, n_cubesphere, content_name):
@@ -200,6 +199,8 @@ class CombineRegionsComputeRealData(MixinCombineRegions):
def time_compute_data(self, n_cubesphere):
_ = self.recombined_cube.data
+ # Vulnerable to noise, so disabled by default.
+ @on_demand_benchmark
@TrackAddedMemoryAllocation.decorator
def track_addedmem_compute_data(self, n_cubesphere):
_ = self.recombined_cube.data
@@ -217,6 +218,8 @@ def time_save(self, n_cubesphere):
# Save to disk, which must compute data + stream it to file.
save(self.recombined_cube, "tmp.nc")
+ # Vulnerable to noise, so disabled by default.
+ @on_demand_benchmark
@TrackAddedMemoryAllocation.decorator
def track_addedmem_save(self, n_cubesphere):
save(self.recombined_cube, "tmp.nc")
@@ -245,6 +248,8 @@ def time_stream_file2file(self, n_cubesphere):
# Save to disk, which must compute data + stream it to file.
save(self.recombined_cube, "tmp.nc")
+ # Vulnerable to noise, so disabled by default.
+ @on_demand_benchmark
@TrackAddedMemoryAllocation.decorator
def track_addedmem_stream_file2file(self, n_cubesphere):
save(self.recombined_cube, "tmp.nc")
diff --git a/benchmarks/benchmarks/generate_data/__init__.py b/benchmarks/benchmarks/generate_data/__init__.py
index 52a5aceca8..8837e7cca9 100644
--- a/benchmarks/benchmarks/generate_data/__init__.py
+++ b/benchmarks/benchmarks/generate_data/__init__.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Scripts for generating supporting data for benchmarking.
diff --git a/benchmarks/benchmarks/generate_data/stock.py b/benchmarks/benchmarks/generate_data/stock.py
index eaf46bb405..b6702ad883 100644
--- a/benchmarks/benchmarks/generate_data/stock.py
+++ b/benchmarks/benchmarks/generate_data/stock.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Wrappers for using :mod:`iris.tests.stock` methods for benchmarking.
@@ -39,7 +38,7 @@ def _external(func_name_, temp_file_dir, **kwargs_):
)
if not REUSE_DATA or not save_path.is_file():
# The xios functions take control of save location so need to move to
- # a more specific name that allows re-use.
+ # a more specific name that allows reuse.
actual_path = run_function_elsewhere(
_external,
func_name_=func_name,
diff --git a/benchmarks/benchmarks/generate_data/ugrid.py b/benchmarks/benchmarks/generate_data/ugrid.py
index 527b49a6bb..3be5c20a48 100644
--- a/benchmarks/benchmarks/generate_data/ugrid.py
+++ b/benchmarks/benchmarks/generate_data/ugrid.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Scripts for generating supporting data for UGRID-related benchmarking.
"""
diff --git a/benchmarks/benchmarks/generate_data/um_files.py b/benchmarks/benchmarks/generate_data/um_files.py
index 39773bbb4b..23d3770aa1 100644
--- a/benchmarks/benchmarks/generate_data/um_files.py
+++ b/benchmarks/benchmarks/generate_data/um_files.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Generate FF, PP and NetCDF files based on a minimal synthetic FF file.
diff --git a/benchmarks/benchmarks/import_iris.py b/benchmarks/benchmarks/import_iris.py
index fc32ac289b..5f902fd2e0 100644
--- a/benchmarks/benchmarks/import_iris.py
+++ b/benchmarks/benchmarks/import_iris.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
from importlib import import_module, reload
################
diff --git a/benchmarks/benchmarks/iterate.py b/benchmarks/benchmarks/iterate.py
index 0a5415ac2b..6cc935498c 100644
--- a/benchmarks/benchmarks/iterate.py
+++ b/benchmarks/benchmarks/iterate.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Iterate benchmark tests.
diff --git a/benchmarks/benchmarks/load/__init__.py b/benchmarks/benchmarks/load/__init__.py
index 1b0ea696f6..a926e6b7e2 100644
--- a/benchmarks/benchmarks/load/__init__.py
+++ b/benchmarks/benchmarks/load/__init__.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
File loading benchmark tests.
@@ -27,7 +26,7 @@ class LoadAndRealise:
# For data generation
timeout = 600.0
params = [
- [(2, 2, 2), (1280, 960, 5), (2, 2, 1000)],
+ [(50, 50, 2), (1280, 960, 5), (2, 2, 1000)],
[False, True],
["FF", "PP", "NetCDF"],
]
@@ -69,7 +68,7 @@ def time_realise(self, _, __, ___, ____) -> None:
class STASHConstraint:
- # xyz sizes mimic LoadAndRealise to maximise file re-use.
+ # xyz sizes mimic LoadAndRealise to maximise file reuse.
params = [[(2, 2, 2), (1280, 960, 5), (2, 2, 1000)], ["FF", "PP"]]
param_names = ["xyz", "file_format"]
diff --git a/benchmarks/benchmarks/load/ugrid.py b/benchmarks/benchmarks/load/ugrid.py
index 350a78e128..ef01ae03be 100644
--- a/benchmarks/benchmarks/load/ugrid.py
+++ b/benchmarks/benchmarks/load/ugrid.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Mesh data loading benchmark tests.
@@ -77,7 +76,7 @@ class DataRealisation:
warmup_time = 0.0
timeout = 300.0
- params = [1, int(2e5)]
+ params = [int(1e4), int(2e5)]
param_names = ["number of faces"]
def setup_common(self, **kwargs):
diff --git a/benchmarks/benchmarks/metadata_manager_factory.py b/benchmarks/benchmarks/metadata_manager_factory.py
index 7e7fc98008..531af58b66 100644
--- a/benchmarks/benchmarks/metadata_manager_factory.py
+++ b/benchmarks/benchmarks/metadata_manager_factory.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
metadata_manager_factory benchmark tests.
diff --git a/benchmarks/benchmarks/mixin.py b/benchmarks/benchmarks/mixin.py
index bec5518eee..335bee1a0f 100644
--- a/benchmarks/benchmarks/mixin.py
+++ b/benchmarks/benchmarks/mixin.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Mixin benchmark tests.
diff --git a/benchmarks/benchmarks/plot.py b/benchmarks/benchmarks/plot.py
index 75195c86e9..73a2a51990 100644
--- a/benchmarks/benchmarks/plot.py
+++ b/benchmarks/benchmarks/plot.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Plot benchmark tests.
diff --git a/benchmarks/benchmarks/regridding.py b/benchmarks/benchmarks/regridding.py
index 44bd1b6c95..9cd77527af 100644
--- a/benchmarks/benchmarks/regridding.py
+++ b/benchmarks/benchmarks/regridding.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Regridding benchmark test
diff --git a/benchmarks/benchmarks/save.py b/benchmarks/benchmarks/save.py
index 3551c72528..6feb446c70 100644
--- a/benchmarks/benchmarks/save.py
+++ b/benchmarks/benchmarks/save.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
File saving benchmarks.
@@ -16,12 +15,12 @@
from iris import save
from iris.experimental.ugrid import save_mesh
-from . import TrackAddedMemoryAllocation
+from . import TrackAddedMemoryAllocation, on_demand_benchmark
from .generate_data.ugrid import make_cube_like_2d_cubesphere
class NetcdfSave:
- params = [[1, 600], [False, True]]
+ params = [[50, 600], [False, True]]
param_names = ["cubesphere-N", "is_unstructured"]
def setup(self, n_cubesphere, is_unstructured):
@@ -47,6 +46,8 @@ def time_netcdf_save_mesh(self, n_cubesphere, is_unstructured):
if is_unstructured:
self._save_mesh(self.cube)
+ # Vulnerable to noise, so disabled by default.
+ @on_demand_benchmark
@TrackAddedMemoryAllocation.decorator
def track_addedmem_netcdf_save(self, n_cubesphere, is_unstructured):
# Don't need to copy the cube here since track_ benchmarks don't
diff --git a/benchmarks/benchmarks/sperf/__init__.py b/benchmarks/benchmarks/sperf/__init__.py
index eccad56f6f..111cd4b841 100644
--- a/benchmarks/benchmarks/sperf/__init__.py
+++ b/benchmarks/benchmarks/sperf/__init__.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Benchmarks for the SPerf scheme of the UK Met Office's NG-VAT project.
diff --git a/benchmarks/benchmarks/sperf/combine_regions.py b/benchmarks/benchmarks/sperf/combine_regions.py
index e27b3b1996..da0cffde50 100644
--- a/benchmarks/benchmarks/sperf/combine_regions.py
+++ b/benchmarks/benchmarks/sperf/combine_regions.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Region combine benchmarks for the SPerf scheme of the UK Met Office's NG-VAT project.
"""
diff --git a/benchmarks/benchmarks/sperf/equality.py b/benchmarks/benchmarks/sperf/equality.py
index 85c73ab92b..bb3b707a75 100644
--- a/benchmarks/benchmarks/sperf/equality.py
+++ b/benchmarks/benchmarks/sperf/equality.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Equality benchmarks for the SPerf scheme of the UK Met Office's NG-VAT project.
"""
diff --git a/benchmarks/benchmarks/sperf/load.py b/benchmarks/benchmarks/sperf/load.py
index 6a60355976..ab1273e288 100644
--- a/benchmarks/benchmarks/sperf/load.py
+++ b/benchmarks/benchmarks/sperf/load.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
File loading benchmarks for the SPerf scheme of the UK Met Office's NG-VAT project.
"""
diff --git a/benchmarks/benchmarks/sperf/save.py b/benchmarks/benchmarks/sperf/save.py
index dd33924c6c..9892f0d239 100644
--- a/benchmarks/benchmarks/sperf/save.py
+++ b/benchmarks/benchmarks/sperf/save.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
File saving benchmarks for the SPerf scheme of the UK Met Office's NG-VAT project.
"""
diff --git a/benchmarks/benchmarks/trajectory.py b/benchmarks/benchmarks/trajectory.py
index 5c1d10d218..e4c3297614 100644
--- a/benchmarks/benchmarks/trajectory.py
+++ b/benchmarks/benchmarks/trajectory.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Trajectory benchmark test
diff --git a/benchmarks/bm_runner.py b/benchmarks/bm_runner.py
index f3efb0ea31..1efe8d3acb 100644
--- a/benchmarks/bm_runner.py
+++ b/benchmarks/bm_runner.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""
Argparse conveniences for executing common types of benchmark runs.
"""
@@ -15,8 +14,10 @@
from os import environ
from pathlib import Path
import re
+import shlex
import subprocess
from tempfile import NamedTemporaryFile
+from textwrap import dedent
from typing import Literal
# The threshold beyond which shifts are 'notable'. See `asv compare`` docs
@@ -24,25 +25,37 @@
COMPARE_FACTOR = 1.2
BENCHMARKS_DIR = Path(__file__).parent
+ROOT_DIR = BENCHMARKS_DIR.parent
+# Storage location for reports used in GitHub actions.
+GH_REPORT_DIR = ROOT_DIR.joinpath(".github", "workflows", "benchmark_reports")
# Common ASV arguments for all run_types except `custom`.
ASV_HARNESS = (
- "run {posargs} --attribute rounds=4 --interleave-rounds --strict "
- "--show-stderr"
+ "run {posargs} --attribute rounds=4 --interleave-rounds --show-stderr"
)
-def _subprocess_run_print(args, **kwargs):
+def echo(echo_string: str):
# Use subprocess for printing to reduce chance of printing out of sequence
# with the subsequent calls.
- subprocess.run(["echo", f"BM_RUNNER DEBUG: {' '.join(args)}"])
+ subprocess.run(["echo", f"BM_RUNNER DEBUG: {echo_string}"])
+
+
+def _subprocess_runner(args, asv=False, **kwargs):
+ # Avoid permanent modifications if the same arguments are used more than once.
+ args = args.copy()
+ kwargs = kwargs.copy()
+ if asv:
+ args.insert(0, "asv")
+ kwargs["cwd"] = BENCHMARKS_DIR
+ echo(" ".join(args))
+ kwargs.setdefault("check", True)
return subprocess.run(args, **kwargs)
-def _subprocess_run_asv(args, **kwargs):
- args.insert(0, "asv")
- kwargs["cwd"] = BENCHMARKS_DIR
- return _subprocess_run_print(args, **kwargs)
+def _subprocess_runner_capture(args, **kwargs) -> str:
+ result = _subprocess_runner(args, capture_output=True, **kwargs)
+ return result.stdout.decode().rstrip()
def _check_requirements(package: str) -> None:
@@ -61,19 +74,18 @@ def _prep_data_gen_env() -> None:
Create/access a separate, unchanging environment for generating test data.
"""
- root_dir = BENCHMARKS_DIR.parent
python_version = "3.11"
data_gen_var = "DATA_GEN_PYTHON"
if data_gen_var in environ:
- print("Using existing data generation environment.")
+ echo("Using existing data generation environment.")
else:
- print("Setting up the data generation environment ...")
+ echo("Setting up the data generation environment ...")
# Get Nox to build an environment for the `tests` session, but don't
- # run the session. Will re-use a cached environment if appropriate.
- _subprocess_run_print(
+ # run the session. Will reuse a cached environment if appropriate.
+ _subprocess_runner(
[
"nox",
- f"--noxfile={root_dir / 'noxfile.py'}",
+ f"--noxfile={ROOT_DIR / 'noxfile.py'}",
"--session=tests",
"--install-only",
f"--python={python_version}",
@@ -82,14 +94,14 @@ def _prep_data_gen_env() -> None:
# Find the environment built above, set it to be the data generation
# environment.
data_gen_python = next(
- (root_dir / ".nox").rglob(f"tests*/bin/python{python_version}")
+ (ROOT_DIR / ".nox").rglob(f"tests*/bin/python{python_version}")
).resolve()
environ[data_gen_var] = str(data_gen_python)
- print("Installing Mule into data generation environment ...")
+ echo("Installing Mule into data generation environment ...")
mule_dir = data_gen_python.parents[1] / "resources" / "mule"
if not mule_dir.is_dir():
- _subprocess_run_print(
+ _subprocess_runner(
[
"git",
"clone",
@@ -97,7 +109,7 @@ def _prep_data_gen_env() -> None:
str(mule_dir),
]
)
- _subprocess_run_print(
+ _subprocess_runner(
[
str(data_gen_python),
"-m",
@@ -107,7 +119,7 @@ def _prep_data_gen_env() -> None:
]
)
- print("Data generation environment ready.")
+ echo("Data generation environment ready.")
def _setup_common() -> None:
@@ -116,41 +128,192 @@ def _setup_common() -> None:
_prep_data_gen_env()
- print("Setting up ASV ...")
- _subprocess_run_asv(["machine", "--yes"])
+ echo("Setting up ASV ...")
+ _subprocess_runner(["machine", "--yes"], asv=True)
- print("Setup complete.")
+ echo("Setup complete.")
def _asv_compare(*commits: str, overnight_mode: bool = False) -> None:
- """Run through a list of commits comparing each one to the next."""
+ """
+ Run through a list of commits comparing each one to the next.
+ """
commits = [commit[:8] for commit in commits]
- shifts_dir = BENCHMARKS_DIR / ".asv" / "performance-shifts"
for i in range(len(commits) - 1):
before = commits[i]
after = commits[i + 1]
- asv_command = (
+ asv_command = shlex.split(
f"compare {before} {after} --factor={COMPARE_FACTOR} --split"
)
- _subprocess_run_asv(asv_command.split(" "))
-
- if overnight_mode:
- # Record performance shifts.
- # Run the command again but limited to only showing performance
- # shifts.
- shifts = _subprocess_run_asv(
- [*asv_command.split(" "), "--only-changed"],
- capture_output=True,
- text=True,
- ).stdout
- if shifts:
- # Write the shifts report to a file.
- # Dir is used by .github/workflows/benchmarks.yml,
- # but not cached - intended to be discarded after run.
- shifts_dir.mkdir(exist_ok=True, parents=True)
- shifts_path = (shifts_dir / after).with_suffix(".txt")
- with shifts_path.open("w") as shifts_file:
- shifts_file.write(shifts)
+
+ comparison = _subprocess_runner_capture(asv_command, asv=True)
+ echo(comparison)
+ shifts = _subprocess_runner_capture(
+ [*asv_command, "--only-changed"], asv=True
+ )
+
+ if shifts or (not overnight_mode):
+ # For the overnight run: only post if there are shifts.
+ _gh_create_reports(after, comparison, shifts)
+
+
+def _gh_create_reports(
+ commit_sha: str, results_full: str, results_shifts: str
+) -> None:
+ """
+ If running under GitHub Actions: record the results in report(s).
+
+ Posting the reports is done by :func:`_gh_post_reports`, which must be run
+ within a separate action to comply with GHA's security limitations.
+ """
+ if "GITHUB_ACTIONS" not in environ:
+ # Only run when within GHA.
+ return
+
+ pr_number = environ.get("PR_NUMBER", None)
+ on_pull_request = pr_number is not None
+ run_id = environ["GITHUB_RUN_ID"]
+ repo = environ["GITHUB_REPOSITORY"]
+ gha_run_link = (
+ f"[`{run_id}`](https://github.com/{repo}/actions/runs/{run_id})"
+ )
+
+ GH_REPORT_DIR.mkdir(exist_ok=True)
+ commit_dir = GH_REPORT_DIR / commit_sha
+ commit_dir.mkdir()
+ command_path = commit_dir / "command.txt"
+ body_path = commit_dir / "body.txt"
+
+ performance_report = dedent(
+ (
+ """
+ ### Performance Benchmark Report: {commit_sha}
+
+
+ Performance shifts
+
+ ```
+ {results_shifts}
+ ```
+
+
+
+
+ Full benchmark results
+
+ ```
+ {results_full}
+ ```
+
+
+
+ Generated by GHA run {gha_run_link}
+ """
+ )
+ )
+ performance_report = performance_report.format(
+ commit_sha=commit_sha,
+ results_shifts=results_shifts,
+ results_full=results_full,
+ gha_run_link=gha_run_link,
+ )
+
+ if on_pull_request:
+ # Command to post the report as a comment on the active PR.
+ body_path.write_text(performance_report)
+ command = (
+ f"gh pr comment {pr_number} "
+ f"--body-file {body_path.absolute()} "
+ f"--repo {repo}"
+ )
+ command_path.write_text(command)
+
+ else:
+ # Command to post the report as new issue.
+ commit_msg = _subprocess_runner_capture(
+ f"git log {commit_sha}^! --oneline".split(" ")
+ )
+ # Intended for benchmarking commits on trunk - should include a PR
+ # number due to our squash policy.
+ pr_tag_match = re.search("#[0-9]*", commit_msg)
+
+ assignee = ""
+ pr_tag = "pull request number unavailable"
+ if pr_tag_match is not None:
+ pr_tag = pr_tag_match.group(0)
+
+ for login_type in ("author", "mergedBy"):
+ gh_query = f'.["{login_type}"]["login"]'
+ command = shlex.split(
+ f"gh pr view {pr_tag[1:]} "
+ f"--json {login_type} -q '{gh_query}' "
+ f"--repo {repo}"
+ )
+ login = _subprocess_runner_capture(command)
+
+ command = [
+ "curl",
+ "-s",
+ f"https://api.github.com/users/{login}",
+ ]
+ login_info = _subprocess_runner_capture(command)
+ is_user = '"type": "User"' in login_info
+ if is_user:
+ assignee = login
+ break
+
+ title = f"Performance Shift(s): `{commit_sha}`"
+ body = dedent(
+ (
+ f"""
+ Benchmark comparison has identified performance shifts at:
+
+ * commit {commit_sha} ({pr_tag}).
+
+
+ Please review the report below and
+ take corrective/congratulatory action as appropriate
+ :slightly_smiling_face:
+
+ """
+ )
+ )
+ body += performance_report
+ body_path.write_text(body)
+
+ command = (
+ "gh issue create "
+ f'--title "{title}" '
+ f"--body-file {body_path.absolute()} "
+ '--label "Bot" '
+ '--label "Type: Performance" '
+ f"--repo {repo}"
+ )
+ if assignee:
+ command += f" --assignee {assignee}"
+ command_path.write_text(command)
+
+
+def _gh_post_reports() -> None:
+ """
+ If running under GitHub Actions: post pre-prepared benchmark reports.
+
+ Reports are prepared by :func:`_gh_create_reports`, which must be run
+ within a separate action to comply with GHA's security limitations.
+ """
+ if "GITHUB_ACTIONS" not in environ:
+ # Only run when within GHA.
+ return
+
+ commit_dirs = [x for x in GH_REPORT_DIR.iterdir() if x.is_dir()]
+ for commit_dir in commit_dirs:
+ command_path = commit_dir / "command.txt"
+ command = command_path.read_text()
+
+ # Security: only accept certain commands to run.
+ assert command.startswith(("gh issue create", "gh pr comment"))
+
+ _subprocess_runner(shlex.split(command))
class _SubParserGenerator(ABC):
@@ -168,18 +331,21 @@ def __init__(self, subparsers: ArgumentParser.add_subparsers) -> None:
formatter_class=argparse.RawTextHelpFormatter,
)
self.add_arguments()
- self.subparser.add_argument(
- "asv_args",
- nargs=argparse.REMAINDER,
- help="Any number of arguments to pass down to ASV.",
- )
+ self.add_asv_arguments()
self.subparser.set_defaults(func=self.func)
@abstractmethod
def add_arguments(self) -> None:
- """All self.subparser.add_argument() calls."""
+ """All custom self.subparser.add_argument() calls."""
_ = NotImplemented
+ def add_asv_arguments(self) -> None:
+ self.subparser.add_argument(
+ "asv_args",
+ nargs=argparse.REMAINDER,
+ help="Any number of arguments to pass down to the ASV benchmark command.",
+ )
+
@staticmethod
@abstractmethod
def func(args: argparse.Namespace):
@@ -197,11 +363,11 @@ class Overnight(_SubParserGenerator):
name = "overnight"
description = (
"Benchmarks all commits between the input **first_commit** to ``HEAD``, "
- "comparing each to its parent for performance shifts. If a commit causes "
- "shifts, the output is saved to a file:\n"
- "``.asv/performance-shifts/``\n\n"
+ "comparing each to its parent for performance shifts. If running on "
+ "GitHub Actions: performance shift(s) will be reported in a new issue.\n"
"Designed for checking the previous 24 hours' commits, typically in a "
- "scheduled script."
+ "scheduled script.\n"
+ "Uses `asv run`."
)
epilog = (
"e.g. python bm_runner.py overnight a1b23d4\n"
@@ -220,16 +386,20 @@ def func(args: argparse.Namespace) -> None:
_setup_common()
commit_range = f"{args.first_commit}^^.."
- asv_command = ASV_HARNESS.format(posargs=commit_range)
- _subprocess_run_asv([*asv_command.split(" "), *args.asv_args])
-
# git rev-list --first-parent is the command ASV uses.
- git_command = f"git rev-list --first-parent {commit_range}"
- commit_string = _subprocess_run_print(
- git_command.split(" "), capture_output=True, text=True
- ).stdout
- commit_list = commit_string.rstrip().split("\n")
- _asv_compare(*reversed(commit_list), overnight_mode=True)
+ git_command = shlex.split(
+ f"git rev-list --first-parent {commit_range}"
+ )
+ commit_string = _subprocess_runner_capture(git_command)
+ commit_list = commit_string.split("\n")
+
+ asv_command = shlex.split(ASV_HARNESS.format(posargs=commit_range))
+ try:
+ _subprocess_runner([*asv_command, *args.asv_args], asv=True)
+ finally:
+ # Designed for long running - want to compare/post any valid
+ # results even if some are broken.
+ _asv_compare(*reversed(commit_list), overnight_mode=True)
class Branch(_SubParserGenerator):
@@ -237,11 +407,15 @@ class Branch(_SubParserGenerator):
description = (
"Performs the same operations as ``overnight``, but always on two commits "
"only - ``HEAD``, and ``HEAD``'s merge-base with the input "
- "**base_branch**. Output from this run is never saved to a file. Designed "
+ "**base_branch**. If running on GitHub Actions: HEAD will be GitHub's "
+ "merge commit and merge-base will be the merge target. Performance "
+ "comparisons will be posted in a comment on the relevant pull request.\n"
+ "Designed "
"for testing if the active branch's changes cause performance shifts - "
"anticipating what would be caught by ``overnight`` once merged.\n\n"
"**For maximum accuracy, avoid using the machine that is running this "
- "session. Run time could be >1 hour for the full benchmark suite.**"
+ "session. Run time could be >1 hour for the full benchmark suite.**\n"
+ "Uses `asv run`."
)
epilog = (
"e.g. python bm_runner.py branch upstream/main\n"
@@ -259,19 +433,22 @@ def add_arguments(self) -> None:
def func(args: argparse.Namespace) -> None:
_setup_common()
- git_command = f"git merge-base HEAD {args.base_branch}"
- merge_base = _subprocess_run_print(
- git_command.split(" "), capture_output=True, text=True
- ).stdout[:8]
+ git_command = shlex.split("git rev-parse HEAD")
+ head_sha = _subprocess_runner_capture(git_command)[:8]
+
+ git_command = shlex.split(
+ f"git merge-base {head_sha} {args.base_branch}"
+ )
+ merge_base = _subprocess_runner_capture(git_command)[:8]
with NamedTemporaryFile("w") as hashfile:
- hashfile.writelines([merge_base, "\n", "HEAD"])
+ hashfile.writelines([merge_base, "\n", head_sha])
hashfile.flush()
commit_range = f"HASHFILE:{hashfile.name}"
- asv_command = ASV_HARNESS.format(posargs=commit_range)
- _subprocess_run_asv([*asv_command.split(" "), *args.asv_args])
+ asv_command = shlex.split(ASV_HARNESS.format(posargs=commit_range))
+ _subprocess_runner([*asv_command, *args.asv_args], asv=True)
- _asv_compare(merge_base, "HEAD")
+ _asv_compare(merge_base, head_sha)
class _CSPerf(_SubParserGenerator, ABC):
@@ -281,7 +458,8 @@ class _CSPerf(_SubParserGenerator, ABC):
"Run the on-demand {} suite of benchmarks (part of the UK Met "
"Office NG-VAT project) for the ``HEAD`` of ``upstream/main`` only, "
"and publish the results to the input **publish_dir**, within a "
- "unique subdirectory for this run."
+ "unique subdirectory for this run.\n"
+ "Uses `asv run`."
)
epilog = (
"e.g. python bm_runner.py {0} my_publish_dir\n"
@@ -321,19 +499,28 @@ def csperf(
asv_command = (
ASV_HARNESS.format(posargs=commit_range) + f" --bench={run_type}"
)
- # C/SPerf benchmarks are much bigger than the CI ones:
- # Don't fail the whole run if memory blows on 1 benchmark.
- asv_command = asv_command.replace(" --strict", "")
- # Only do a single round.
- asv_command = re.sub(r"rounds=\d", "rounds=1", asv_command)
- _subprocess_run_asv([*asv_command.split(" "), *args.asv_args])
- asv_command = f"publish {commit_range} --html-dir={publish_subdir}"
- _subprocess_run_asv(asv_command.split(" "))
+ # Only do a single round.
+ asv_command = shlex.split(
+ re.sub(r"rounds=\d", "rounds=1", asv_command)
+ )
+ try:
+ _subprocess_runner([*asv_command, *args.asv_args], asv=True)
+ except subprocess.CalledProcessError as err:
+ # C/SPerf benchmarks are much bigger than the CI ones:
+ # Don't fail the whole run if memory blows on 1 benchmark.
+ # ASV produces return code of 2 if the run includes crashes.
+ if err.returncode != 2:
+ raise
+
+ asv_command = shlex.split(
+ f"publish {commit_range} --html-dir={publish_subdir}"
+ )
+ _subprocess_runner(asv_command, asv=True)
# Print completion message.
location = BENCHMARKS_DIR / ".asv"
- print(
+ echo(
f'New ASV results for "{run_type}".\n'
f'See "{publish_subdir}",'
f'\n or JSON files under "{location / "results"}".'
@@ -380,7 +567,29 @@ def add_arguments(self) -> None:
@staticmethod
def func(args: argparse.Namespace) -> None:
_setup_common()
- _subprocess_run_asv([args.asv_sub_command, *args.asv_args])
+ _subprocess_runner([args.asv_sub_command, *args.asv_args], asv=True)
+
+
+class GhPost(_SubParserGenerator):
+ name = "_gh_post"
+ description = (
+ "Used by GitHub Actions to post benchmark reports that were prepared "
+ "during previous actions. Separated to comply with GitHub's security "
+ "requirements."
+ )
+ epilog = "Sole acceptable syntax: python bm_runner.py _gh_post"
+
+ @staticmethod
+ def func(args: argparse.Namespace) -> None:
+ _gh_post_reports()
+
+ # No arguments permitted for this subclass:
+
+ def add_arguments(self) -> None:
+ pass
+
+ def add_asv_arguments(self) -> None:
+ pass
def main():
@@ -390,7 +599,7 @@ def main():
)
subparsers = parser.add_subparsers(required=True)
- for gen in (Overnight, Branch, CPerf, SPerf, Custom):
+ for gen in (Overnight, Branch, CPerf, SPerf, Custom, GhPost):
_ = gen(subparsers).subparser
parsed = parser.parse_args()
diff --git a/docs/gallery_code/meteorology/plot_COP_1d.py b/docs/gallery_code/meteorology/plot_COP_1d.py
index bebbad4224..2181b89b8c 100644
--- a/docs/gallery_code/meteorology/plot_COP_1d.py
+++ b/docs/gallery_code/meteorology/plot_COP_1d.py
@@ -54,7 +54,7 @@ def main():
)
# Generate area-weights array. As e1 and a1b are on the same grid we can
- # do this just once and re-use. This method requires bounds on lat/lon
+ # do this just once and reuse. This method requires bounds on lat/lon
# coords, so let's add some in sensible locations using the "guess_bounds"
# method.
e1.coord("latitude").guess_bounds()
diff --git a/docs/gallery_code/meteorology/plot_COP_maps.py b/docs/gallery_code/meteorology/plot_COP_maps.py
index 5e158346a9..529018ec8c 100644
--- a/docs/gallery_code/meteorology/plot_COP_maps.py
+++ b/docs/gallery_code/meteorology/plot_COP_maps.py
@@ -171,23 +171,13 @@ def main():
)
plt.gca().coastlines()
- # Now add a colourbar who's leftmost point is the same as the leftmost
- # point of the left hand plot and rightmost point is the rightmost
- # point of the right hand plot.
-
- # Get the positions of the 2nd plot and the left position of the 1st plot.
- left, bottom, width, height = ax_array[1].get_position().bounds
- first_plot_left = ax_array[0].get_position().bounds[0]
-
- # The width of the colorbar should now be simple.
- width = left - first_plot_left + width
-
- # Add axes to the figure, to place the colour bar.
- colorbar_axes = fig.add_axes([first_plot_left, 0.18, width, 0.03])
-
- # Add the colour bar.
+ # Now add a colour bar which spans the two plots. Here we pass Figure.axes
+ # which is a list of all (two) axes currently on the figure. Note that
+ # these are different to the contents of ax_array, because those were
+ # standard Matplotlib Axes that Iris automatically replaced with Cartopy
+ # GeoAxes.
cbar = plt.colorbar(
- contour_result, colorbar_axes, orientation="horizontal"
+ contour_result, ax=fig.axes, aspect=60, orientation="horizontal"
)
# Label the colour bar and add ticks.
diff --git a/docs/gallery_tests/__init__.py b/docs/gallery_tests/__init__.py
index ac5753e58b..091e997248 100644
--- a/docs/gallery_tests/__init__.py
+++ b/docs/gallery_tests/__init__.py
@@ -1,5 +1,4 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
diff --git a/docs/gallery_tests/conftest.py b/docs/gallery_tests/conftest.py
index a218b305a2..d3ca8309f8 100644
--- a/docs/gallery_tests/conftest.py
+++ b/docs/gallery_tests/conftest.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
"""Pytest fixtures for the gallery tests."""
diff --git a/docs/gallery_tests/test_gallery_examples.py b/docs/gallery_tests/test_gallery_examples.py
index 0d0793a7da..93f361a62a 100644
--- a/docs/gallery_tests/test_gallery_examples.py
+++ b/docs/gallery_tests/test_gallery_examples.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
import importlib
diff --git a/docs/src/_templates/layout.html b/docs/src/_templates/layout.html
deleted file mode 100644
index 974bd12753..0000000000
--- a/docs/src/_templates/layout.html
+++ /dev/null
@@ -1,20 +0,0 @@
-{% extends "pydata_sphinx_theme/layout.html" %}
-
-{# This uses blocks. See:
- https://www.sphinx-doc.org/en/master/templating.html
-#}
-
-
- {%- block docs_body %}
-
- {% if on_rtd and rtd_version == 'latest' %}
-
- You are viewing the latest unreleased documentation
- v{{ version }}. You can switch to a stable version
- via the flyout menu in the bottom corner of the screen.
-
-
- {%- endif %}
-
- {{ super() }}
-{%- endblock %}
diff --git a/docs/src/community/index.rst b/docs/src/community/index.rst
index 114cb96fe9..ee227513b3 100644
--- a/docs/src/community/index.rst
+++ b/docs/src/community/index.rst
@@ -40,12 +40,14 @@ smoother interoperability:
* The :mod:`iris.pandas` module
* :doc:`iris_xarray`
+* :doc:`phrasebook`
.. toctree::
:maxdepth: 1
:hidden:
iris_xarray
+ phrasebook
Plugins
-------
diff --git a/docs/src/community/iris_xarray.rst b/docs/src/community/iris_xarray.rst
index 2250e3c0a3..9d795fcd9e 100644
--- a/docs/src/community/iris_xarray.rst
+++ b/docs/src/community/iris_xarray.rst
@@ -7,6 +7,7 @@ Iris ❤️ :term:`Xarray`
There is a lot of overlap between Iris and :term:`Xarray`, but some important
differences too. Below is a summary of the most important differences, so that
you can be prepared, and to help you choose the best package for your use case.
+See :doc:`phrasebook` for a broad comparison of terminology.
Overall Experience
------------------
diff --git a/docs/src/community/phrasebook.rst b/docs/src/community/phrasebook.rst
new file mode 100644
index 0000000000..bcd91cca83
--- /dev/null
+++ b/docs/src/community/phrasebook.rst
@@ -0,0 +1,66 @@
+.. include:: ../common_links.inc
+
+.. _phrasebook:
+
+Package Phrasebook
+===================
+
+There are a number of similar packages to Iris, and a lot of these have their own
+terminology for similar things. Whether you're coming or going, we hope this might
+be a helpful guide to these differences!
+Definitions for each can be found in :ref:`glossary`. See also
+`Xarray terminology `_.
+
+.. list-table:: Phrasebook
+ :widths: 25 25 25 50
+ :header-rows: 1
+
+ * - Iris
+ - Xarray
+ - Example
+ - Notes
+ * - Non-Lazy
+ - Eager
+ -
+ - Used to relate to functions, rather than the data.
+ * - Cube
+ - DataArray
+ -
+ -
+ * - CubeList
+ - Dataset
+ -
+ - Though similar, a CubeList is a simpler object, and is
+ not a perfect comparison to a Dataset
+ * - Merge/ Concatenate
+ - Concatenate
+ - `Xarray concatenate `_
+ - Xarray's concatenate has the capability to largely do what both
+ Iris merge and Iris concatenate do. However, this is not a perfect comparison,
+ please see the link for more information.
+ * -
+ - Merge
+ - `Xarray merge `_
+ - Xarray's Merge function doesn't map neatly map to any Iris feature.
+ Please see the link for more information.
+ * - Scalar Coordinate
+ -
+ -
+ - Iris makes a distinction between scalar coordinates and non-scalar coordinates,
+ whereas xarray documentation makes a distinction between scalar and non-scalar *data*.
+ It is possible to make coordinates with scalar data in both Iris and xarray
+ but only Iris will label such coordinates.
+ * - AuxCoord
+ - Non-Dimensional Coordinate
+ -
+ - Coordinates in Iris and xarray are categorised using different rules,
+ and so are not a one-to-one match.
+ * - DimCoord
+ - Dimension Coordinate
+ -
+ - Coordinates in Iris and xarray are categorised using different rules,
+ and so are not a one-to-one match.
+
+----
+
+`To top `_
\ No newline at end of file
diff --git a/docs/src/conf.py b/docs/src/conf.py
index b7f87d4ebc..e349000862 100644
--- a/docs/src/conf.py
+++ b/docs/src/conf.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
# -*- coding: utf-8 -*-
#
@@ -295,6 +294,10 @@ def _dotv(version):
"navigation_depth": 3,
"show_prev_next": True,
"navbar_align": "content",
+ # removes the search box from the top bar
+ "navbar_persistent": [],
+ # TODO: review if 6 links is too crowded.
+ "header_links_before_dropdown": 6,
"github_url": "https://github.com/SciTools/iris",
"twitter_url": "https://twitter.com/scitools_iris",
# icons available: https://fontawesome.com/v5.15/icons?d=gallery&m=free
@@ -327,6 +330,16 @@ def _dotv(version):
},
}
+# if we are building via Read The Docs and it is the latest (not stable)
+if on_rtd and rtd_version == "latest":
+ html_theme_options[
+ "announcement"
+ ] = f"""
+ You are viewing the latest unreleased documentation
+ {version}. You can switch to a
+ stable
+ version."""
+
rev_parse = run(["git", "rev-parse", "--short", "HEAD"], capture_output=True)
commit_sha = rev_parse.stdout.decode().strip()
@@ -376,13 +389,14 @@ def _dotv(version):
"https://docs.github.com",
"https://github.com",
"http://www.personal.psu.edu/cab38/ColorBrewer/ColorBrewer_updates.html",
- "http://schacon.github.com/git",
"http://scitools.github.com/cartopy",
"http://www.wmo.int/pages/prog/www/DPFS/documents/485_Vol_I_en_colour.pdf",
"https://software.ac.uk/how-cite-software",
"http://www.esrl.noaa.gov/psd/data/gridded/conventions/cdc_netcdf_standard.shtml",
"http://www.nationalarchives.gov.uk/doc/open-government-licence",
"https://www.metoffice.gov.uk/",
+ "https://biggus.readthedocs.io/",
+ "https://stickler-ci.com/",
]
# list of sources to exclude from the build.
diff --git a/docs/src/copyright.rst b/docs/src/copyright.rst
index 2829374f47..d5996fd999 100644
--- a/docs/src/copyright.rst
+++ b/docs/src/copyright.rst
@@ -6,18 +6,15 @@ Iris Code
---------
All Iris source code, unless explicitly stated, is ``Copyright Iris
-contributors`` and is licensed under the **GNU Lesser General Public
-License** as published by the Free Software Foundation, either version 3 of
-the License, or (at your option) any later version.
+contributors`` and is licensed under the **BSD-3 License**.
You should find all source files with the following header:
.. admonition:: Code License
Copyright Iris contributors
- This file is part of Iris and is released under the LGPL license.
- See COPYING and COPYING.LESSER in the root of the repository for full
- licensing details.
+ This file is part of Iris and is released under the BSD license.
+ See LICENSE in the root of the repository for full licensing details.
Iris Documentation and Examples
diff --git a/docs/src/developers_guide/contributing_documentation_easy.rst b/docs/src/developers_guide/contributing_documentation_easy.rst
index f54de628bf..51554f9e19 100755
--- a/docs/src/developers_guide/contributing_documentation_easy.rst
+++ b/docs/src/developers_guide/contributing_documentation_easy.rst
@@ -81,9 +81,9 @@ Describing what you've changed and why will help the person who reviews your cha
.. tip::
If you're not sure that you're making your pull request right, or have a
- question, then make it anyway! You can then comment on it tagging
- ``@SciTools/iris-devs`` to ask your question (then edit your pull request if
- you need to).
+ question, then make it anyway! You can then comment on it to ask your
+ question, then someone from the dev team will be happy to help you out (then
+ edit your pull request if you need to).
What Happens Next?
^^^^^^^^^^^^^^^^^^
diff --git a/docs/src/developers_guide/gitwash/development_workflow.rst b/docs/src/developers_guide/gitwash/development_workflow.rst
index b086922d5b..8545a04308 100644
--- a/docs/src/developers_guide/gitwash/development_workflow.rst
+++ b/docs/src/developers_guide/gitwash/development_workflow.rst
@@ -14,8 +14,7 @@ Workflow Summary
In what follows we'll refer to the upstream iris ``main`` branch, as
"trunk".
-* Don't use your ``main`` (that is on your fork) branch for anything.
- Consider deleting it.
+* Don't use your ``main`` (that is on your fork) branch for development.
* When you are starting a new set of changes, fetch any changes from trunk,
and start a new *feature branch* from that.
* Make a new branch for each separable set of changes |emdash| "one task, one
@@ -34,13 +33,6 @@ what you've done, and why you did it.
See `linux git workflow`_ for some explanation.
-Consider Deleting Your Main Branch
-==================================
-
-It may sound strange, but deleting your own ``main`` branch can help reduce
-confusion about which branch you are on. See `deleting master on github`_ for
-details.
-
.. _update-mirror-trunk:
Update the Mirror of Trunk
@@ -82,7 +74,7 @@ what the changes in the branch are for. For example ``add-ability-to-fly``, or
git checkout my-new-feature
Generally, you will want to keep your feature branches on your public github_
-fork of `iris`_. To do this, you `git push`_ this new branch up to your
+fork of `iris`_. To do this, you ``git push`` this new branch up to your
github repo. Generally (if you followed the instructions in these pages, and by
default), git will have a link to your github repo, called ``origin``. You push
up to your own repo on github with::
@@ -116,7 +108,7 @@ In More Detail
--------------
#. Make some changes
-#. See which files have changed with ``git status`` (see `git status`_).
+#. See which files have changed with ``git status``.
You'll see a listing like this one::
# On branch ny-new-feature
@@ -132,16 +124,15 @@ In More Detail
# INSTALL
no changes added to commit (use "git add" and/or "git commit -a")
-#. Check what the actual changes are with ``git diff`` (`git diff`_).
-#. Add any new files to version control ``git add new_file_name`` (see
- `git add`_).
+#. Check what the actual changes are with ``git diff``.
+#. Add any new files to version control ``git add new_file_name``.
#. To commit all modified files into the local copy of your repo, do
``git commit -am 'A commit message'``. Note the ``-am`` options to
``commit``. The ``m`` flag just signals that you're going to type a
message on the command line. The ``a`` flag will automatically stage
all files that have been modified and deleted.
#. To push the changes up to your forked repo on github, do a ``git
- push`` (see `git push`_).
+ push``.
Testing Your Changes
diff --git a/docs/src/developers_guide/gitwash/git_links.inc b/docs/src/developers_guide/gitwash/git_links.inc
index 11d037ccf4..bf20d13e5f 100644
--- a/docs/src/developers_guide/gitwash/git_links.inc
+++ b/docs/src/developers_guide/gitwash/git_links.inc
@@ -13,21 +13,6 @@
.. _github help: https://help.github.com
.. _git documentation: https://git-scm.com/docs
-.. _git clone: http://schacon.github.com/git/git-clone.html
-.. _git checkout: http://schacon.github.com/git/git-checkout.html
-.. _git commit: http://schacon.github.com/git/git-commit.html
-.. _git push: http://schacon.github.com/git/git-push.html
-.. _git pull: http://schacon.github.com/git/git-pull.html
-.. _git add: http://schacon.github.com/git/git-add.html
-.. _git status: http://schacon.github.com/git/git-status.html
-.. _git diff: http://schacon.github.com/git/git-diff.html
-.. _git log: http://schacon.github.com/git/git-log.html
-.. _git branch: http://schacon.github.com/git/git-branch.html
-.. _git remote: http://schacon.github.com/git/git-remote.html
-.. _git rebase: http://schacon.github.com/git/git-rebase.html
-.. _git config: http://schacon.github.com/git/git-config.html
-
.. _linux git workflow: http://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html
-.. _deleting master on github: https://matthew-brett.github.io/pydagogue/gh_delete_master.html
.. |emdash| unicode:: U+02014
diff --git a/docs/src/further_topics/dask_best_practices/dask_bags_and_greed.rst b/docs/src/further_topics/dask_best_practices/dask_bags_and_greed.rst
new file mode 100644
index 0000000000..007a58d400
--- /dev/null
+++ b/docs/src/further_topics/dask_best_practices/dask_bags_and_greed.rst
@@ -0,0 +1,235 @@
+.. _examples_bags_greed:
+
+3. Dask Bags and Greedy Parallelism
+-----------------------------------
+
+Here is a journey that demonstrates:
+
+* How to apply dask.bags to an existing script
+* The equal importance of optimisation of non-parallel parts of a script
+* Protection against multiple softwares trying to manage parallelism
+ simultaneously
+
+
+3.1 The Problem - Slow Loading
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+We have ~7000 GRIB files spread between 256 dated directories::
+
+ .
+ |-- 20180401
+ | |-- gfs.t00z.icing.0p25.grb2f006
+ | |-- gfs.t00z.icing.0p25.grb2f006.1
+ | |-- gfs.t00z.icing.0p25.grb2f012
+ | |-- gfs.t00z.icing.0p25.grb2f018
+ | |-- gfs.t00z.icing.0p25.grb2f024
+ | |-- gfs.t00z.icing.0p25.grb2f030
+ | `-- gfs.t00z.icing.0p25.grb2f036
+ |-- 20180402
+ | `-- gfs.t00z.icing.0p25.grb2f006
+ |-- 20180403
+ | |-- gfs.t12z.icing.0p25.grb2f006
+ | |-- gfs.t12z.icing.0p25.grb2f012
+
+With this script, a sample of 11 GRIB files takes ~600secs to load::
+
+ import iris
+ import glob
+
+ fpaths=glob.glob('20190416/*t18z*f???')
+ cubes = iris.load(fpaths, callback=callback)
+
+ def callback(cube, field, fname):
+ if field.sections[5]['bitsPerValue'] == 0:
+ raise iris.exceptions.IgnoreCubeException
+ if field.sections[4]['parameterNumber'] == 20:
+ raise iris.exceptions.IgnoreCubeException
+ elif field.sections[4]['parameterNumber'] == 234:
+ cube.long_name = 'Icing Severity'
+
+3.2 Parallelisation
+^^^^^^^^^^^^^^^^^^^
+We'll try using `dask.bag `_ to
+parallelise the function calls. It's important that Dask is given the freedom
+to break the task down in an efficient manner - the function that is mapped
+across the bag should only load a single file, and the bag itself can
+iterate through the list of files. Here's the restructured script::
+
+ import glob
+ import multiprocessing
+ import os
+
+ import dask
+ import dask.bag as db
+ import iris
+
+ def callback(cube, field, fname):
+ if field.sections[5]['bitsPerValue'] == 0:
+ raise iris.exceptions.IgnoreCubeException
+ if field.sections[4]['parameterNumber'] == 20:
+ raise iris.exceptions.IgnoreCubeException
+ elif field.sections[4]['parameterNumber'] == 234:
+ cube.long_name = 'Icing Severity'
+
+ def func(fname):
+ return iris.load_cube(fname, callback=callback)
+
+ fpaths = list(glob.glob('20190416/*t18z*f???'))
+
+ # Determine the number of processors visible ..
+ cpu_count = multiprocessing.cpu_count()
+
+ # .. or as given by slurm allocation.
+ # Only relevant when using Slurm for job scheduling
+ if 'SLURM_NTASKS' in os.environ:
+ cpu_count = os.environ['SLURM_NTASKS']
+
+ # Do not exceed the number of CPUs available, leaving 1 for the system.
+ num_workers = cpu_count - 1
+ print('Using {} workers from {} CPUs...'.format(num_workers, cpu_count))
+
+ # Now do the parallel load.
+ with dask.config.set(num_workers=num_workers):
+ bag = db.from_sequence(fpaths).map(func)
+ cubes = iris.cube.CubeList(bag.compute()).merge()
+
+This achieves approximately a 10-fold improvement if enough CPUs are
+available to have one per file. See this benchmarking:
+
++---------------+-----------------------+---------------+---------------+
+| Machine | CPUs Available | CPUs Used | Time Taken |
++===============+=======================+===============+===============+
+| A | 4 | 3 | 4m 05s |
+| | +---------------+---------------+
+| | | 4 | 3m 22s |
++---------------+-----------------------+---------------+---------------+
+| B | 8 | 1 | 9m 10s |
+| | +---------------+---------------+
+| | | 7 | 2m 35s |
+| | +---------------+---------------+
+| | | 8 | 2m 20s |
++---------------+-----------------------+---------------+---------------+
+
+
+.. _examples_bags_greed_profile:
+
+3.3 Profiling
+^^^^^^^^^^^^^
+1m 10s is still a surprisingly long time. When faced with a mystery like
+this it is helpful to profile the script to see if there are any steps that
+are taking more time than we would expect. For this we use a tool called
+`kapture `_ to produce a
+flame chart visualising the time spent performing each call:
+
+.. image:: images/grib-bottleneck.png
+ :width: 1000
+ :align: center
+
+From this we see that 96% of the runtime is taken by this call::
+
+ res = gribapi.grib_get_array(self._message_id, key)
+
+This is the call being used during the ``callback`` function when it uses
+GRIB messages to filter out cubes with certain unwanted properties.
+
+3.4 Improving GRIB Key Handling
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Even with parallelisation, we are still limited by the time it takes to run
+a single instance of a function. This is going to become much more important
+when running 7000 files instead of 11, since there will be nowhere near
+enough CPUs even on a large multi-processing system, meaning each CPU will be running many instances
+of the function. **Parallelisation can only go so far to solving speed issues** --
+it's effectively the 'brute force' method.
+
+:ref:`examples_bags_greed_profile` showed us where the major bottleneck is. To improve efficiency
+we can re-write the script to filter on GRIB messages *before* converting
+the GRIB file to a cube::
+
+ import dask
+ import dask.bag as db
+ import glob
+ import iris
+ import multiprocessing
+ import os
+
+ def func(fname):
+ import iris
+ from iris_grib import load_pairs_from_fields
+ from iris_grib.message import GribMessage # perform GRIB message level filtering...
+ filtered_messages = []
+ for message in GribMessage.messages_from_filename(fname):
+ if (message.sections[5]['bitsPerValue'] != 0 and
+ message.sections[4]['parameterNumber'] == 234):
+ filtered_messages.append(message) # now convert the messages to cubes...
+ cubes = [cube for cube, message in load_pairs_from_fields(filtered_messages)]
+ return iris.cube.CubeList(cubes).merge_cube()
+
+ fpaths = list(glob.glob('/scratch/frcz/ICING/GFS_DATA/20190416/*t18z*f???'))
+ cpu_count = multiprocessing.cpu_count()
+
+ # Only relevant when using Slurm for job scheduling
+ if 'SLURM_NTASKS' in os.environ:
+ cpu_count = os.environ['SLURM_NTASKS']
+
+ num_workers = cpu_count - 1
+
+ print('Using {} workers from {} CPUs...'.format(num_workers, cpu_count))
+ with dask.config.set(num_workers=num_workers):
+ bag = db.from_sequence(fpaths).map(func)
+ cubes = iris.cube.CubeList(bag.compute())
+
+This achieves a significant performance improvement - more than twice as
+fast as the previous benchmarks:
+
++---------------+-----------------------+---------------+---------------+-----------+
+| Machine | CPUs Available | CPUs Used | Previous Time | New Time |
++===============+=======================+===============+===============+===========+
+| Example | 8 | 7 | 2m 35s | 1m 05s |
+| | +---------------+---------------+-----------+
+| | | 8 | 2m 20s | 1m 03s |
++---------------+-----------------------+---------------+---------------+-----------+
+
+3.5 Managing External Factors
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The speed will still need to be further improved before we can process 7000
+files. The main gains we can achieve are by making sure it is **only Dask**
+that manages multi-processing - if multi-processing is coming from more
+than one place there are predictable clashes.
+
+First, NumPy must be prevented from performing it's own multi-processing by
+adding the following **before** ``import numpy`` is called. You can read more
+about this in :ref:`numpy_threads`.
+
+::
+
+ import os
+
+ os.environ["OMP_NUM_THREADS"] = "1"
+ os.environ["OPENBLAS_NUM_THREADS"] = "1"
+ os.environ["MKL_NUM_THREADS"] = "1"
+ os.environ["VECLIB_MAXIMUM_THREADS"] = "1"
+ os.environ["NUMEXPR_NUM_THREADS"] = "1"
+
+Lastly, if you are using SLURM on the computing cluster then SLURM must be configured to prevent it
+optimising the number of cores necessary for the job. See the SLURM commands
+below, to be added before running the python script. It's important that
+``ntasks`` matches the number of CPUs specified in the python script. You
+can read more about these points in :ref:`multi-pro_slurm`.
+
+::
+
+ #SBATCH --ntasks=12
+ #SBATCH --ntasks-per-core=1
+
+This has all been based on a real example. Once all the above had been set
+up correctly, the completion time had dropped from an estimated **55 days**
+to **less than 1 day**.
+
+3.6 Lessons
+^^^^^^^^^^^
+* Dask isn't a magic switch - it's important to write your script so that
+ there is a way to create small sub-tasks. In this case by providing
+ dask.bag with the file list and the function separated
+* Parallelism is not the only performance improvement to try - the script
+ will still be slow if the individual function is slow
+* All multi-processing needs to be managed by Dask. Several other factors
+ may introduce multi-processing and these need to be configured not to
diff --git a/docs/src/further_topics/dask_best_practices/dask_parallel_loop.rst b/docs/src/further_topics/dask_best_practices/dask_parallel_loop.rst
new file mode 100644
index 0000000000..836503314c
--- /dev/null
+++ b/docs/src/further_topics/dask_best_practices/dask_parallel_loop.rst
@@ -0,0 +1,169 @@
+.. _examples_parallel_loop:
+
+2. Parallelising a Loop of Multiple Calls to a Third Party Library
+------------------------------------------------------------------
+
+Whilst Iris does provide extensive functionality for performing statistical and
+mathematical operations on your data, it is sometimes necessary to use a third
+party library.
+
+The following example describes a real world use case of how to parallelise
+multiple calls to a third party library using dask bags.
+
+2.1 The Problem - Parallelising a Loop
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+In this particular example, the user is calculating a sounding parcel for each
+column in their dataset. The cubes that are used are of shape::
+
+ (model_level_number: 20; grid_latitude: 1536; grid_longitude: 1536)
+
+As a sounding is calculated for each column, this means there are 1536x1536
+individual calculations.
+
+In Python, it is common practice to vectorize the calculation of for loops.
+Vectorising is done by using NumPy to operate on the whole array at once rather
+than a single element at a time. Unfortunately, not all operations are
+vectorisable, including the calculation in this example, and so we look to
+other methods to improve the performance.
+
+2.2 Original Code with Loop
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+We start out by loading cubes of pressure, temperature, dewpoint temperature and height::
+
+ import iris
+ import numpy as np
+ from skewt import SkewT as sk
+
+ pressure = iris.load_cube('a.press.19981109.pp')
+ temperature = iris.load_cube('a.temp.19981109.pp')
+ dewpoint = iris.load_cube('a.dewp.19981109.pp')
+ height = iris.load_cube('a.height.19981109.pp')
+
+We set up the NumPy arrays we will be filling with the output data::
+
+ output_arrays = [np.zeros(pressure.shape[0]) for _ in range(6)]
+ cape, cin, lcl, lfc, el, tpw = output_data
+
+Now we loop over the columns in the data to calculate the soundings::
+
+ for y in range(nlim):
+ for x in range(nlim):
+ mydata = {'pres': pressure[:, y, x],
+ 'temp': temperature[:, y, x],
+ 'dwpt': dewpoint[:, y, x],
+ 'hght': height[:, y, x]}
+
+ # Calculate the sounding with the selected column of data.
+ S = sk.Sounding(soundingdata=mydata)
+ try:
+ startp, startt, startdp, type_ = S.get_parcel(parcel_def)
+ P_lcl, P_lfc, P_el, CAPE, CIN = S.get_cape(
+ startp, startt, startdp, totalcape='tot')
+ TPW = S.precipitable_water()
+ except:
+ P_lcl, P_lfc, P_el, CAPE, CIN, TPW = [
+ np.ma.masked for _ in range(6)]
+
+ # Fill the output arrays with the results
+ cape[y,x] = CAPE
+ cin[y,x] = CIN
+ lcl[y,x] = P_lcl
+ lfc[y,x] = P_lfc
+ el[y,x] = P_el
+ tpw[y,x] = TPW
+
+2.3 Profiling the Code with Kapture
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Kapture is a useful statistical profiler. For more information see `the
+Kapture repo `_.
+
+Results below:
+
+.. image:: images/loop_third_party_kapture_results.png
+ :width: 1000
+ :align: center
+
+As we can see above, (looking at the highlighted section of the red bar) it spends most of the time in the call to ::
+
+ S.get_parcel(parcel_def)
+
+As there are over two million columns in the data, we would greatly benefit
+from parallelising this work.
+
+2.4 Parallelising with Dask Bags
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Dask bags are collections of Python objects that you can map a computation over
+in a parallel manner.
+
+For more information about dask bags, see the `Dask Bag Documentation
+`_.
+
+Dask bags work best with lightweight objects, so we will create a collection of
+indices into our data arrays.
+
+First, we put the loop into a function that takes a slice object to index the
+appropriate section of the array.::
+
+ def calculate_sounding(y_slice):
+ for y in range(y_slice.stop-y_slice.start):
+ for x in range(nlim):
+ mydata = {'pres': pressure[:, y_slice][:, y, x],
+ 'temp': temperature[:, y_slice][:, y, x],
+ 'dwpt': dewpoint[:, y_slice][:, y, x],
+ 'hght': height[:, y_slice][:, y, x]}
+
+ # Calculate the sounding with the selected column of data.
+ S = sk.Sounding(soundingdata=mydata)
+ try:
+ startp, startt, startdp, type_ = S.get_parcel(parcel_def)
+ P_lcl, P_lfc, P_el, CAPE, CIN = S.get_cape(
+ startp, startt, startdp, totalcape=total_cape)
+ TPW = S.precipitable_water()
+ except:
+ P_lcl, P_lfc, P_el, CAPE, CIN, TPW = [
+ np.ma.masked for _ in range(6)]
+
+ # Fill the output arrays with the results
+ cape[:, y_slice][y,x] = CAPE
+ cin[:, y_slice][y,x] = CIN
+ lcl[:, y_slice][y,x] = P_lcl
+ lfc[:, y_slice][y,x] = P_lfc
+ el[:, y_slice][y,x] = P_el
+ tpw[:, y_slice][y,x] = TPW
+
+Then we create a dask bag of slice objects that will create multiple partitions
+along the y axis.::
+
+ num_of_workers = 4
+ len_of_y_axis = pressure.shape[1]
+
+ part_loc = [int(loc) for loc in np.floor(np.linspace(0, len_of_y_axis,
+ num_of_workers + 1))]
+
+ dask_bag = db.from_sequence(
+ [slice(part_loc[i], part_loc[i+1]) for i in range(num_of_workers)])
+
+ with dask.config.set(scheduler='processes'):
+ dask_bag.map(calculate_sounding).compute()
+
+When this was run on a machine with 4 workers, a speedup of ~4x was achieved,
+as expected.
+
+Note that if using the processes scheduler this is some extra time spent
+serialising the data to pass it between workers. For more information on the
+different schedulers available in Dask, see `Dask Scheduler Overview
+`_.
+
+For more speed up, it is possible to run the same code on a multi-processing
+system where you will have access to more CPUs.
+
+In this particular example, we are handling multiple numpy arrays and so we use
+dask bags. If working with a single numpy array, it may be more appropriate to
+use Dask Arrays (see `Dask Arrays
+`_ for more information).
+
+
+2.5 Lessons
+^^^^^^^^^^^
+* If possible, dask bags should contain lightweight objects
+* Minimise the number of tasks that are created
diff --git a/docs/src/further_topics/dask_best_practices/dask_pp_to_netcdf.rst b/docs/src/further_topics/dask_best_practices/dask_pp_to_netcdf.rst
new file mode 100644
index 0000000000..28784154b4
--- /dev/null
+++ b/docs/src/further_topics/dask_best_practices/dask_pp_to_netcdf.rst
@@ -0,0 +1,92 @@
+.. _examples_pp_to_ff:
+
+1. Speed up Converting PP Files to NetCDF
+-----------------------------------------
+
+Here is an example of how dask objects can be tuned for better performance.
+
+1.1 The Problem - Slow Saving
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+We have ~300 PP files which we load as follows:
+
+.. code-block:: python
+
+ import iris
+ import glob
+
+ files = glob.glob("pp_files/*.pp")
+ cube = iris.load_cube(files, "mass_fraction_of_ozone_in_air")
+
+Note that loading here may also be parallelised in a similar manner as
+described in :ref:`examples_bags_greed`. Either way, the resulting cube looks
+as follows:
+
+.. code-block:: text
+
+ mass_fraction_of_ozone_in_air / (kg kg-1) (time: 276; model_level_number: 85; latitude: 144; longitude: 192)
+ Dimension coordinates:
+ time x - - -
+ model_level_number - x - -
+ latitude - - x -
+ longitude - - - x
+ Auxiliary coordinates:
+ forecast_period x - - -
+ level_height - x - -
+ sigma - x - -
+ Scalar coordinates:
+ forecast_reference_time: 1850-01-01 00:00:00
+ Attributes:
+ STASH: m01s34i001
+ source: Data from Met Office Unified Model
+ um_version: 10.9
+ Cell methods:
+ mean: time (1 hour)
+
+The cube is then immediately saved as a netCDF file.
+
+.. code-block:: python
+
+ nc_chunks = [chunk[0] for chunk in cube.lazy_data().chunks]
+ iris.save(cube, "outfile.nc", nc_chunks)
+
+This operation was taking longer than expected and we would like to improve
+performance. Note that when this cube is being saved, the data is still lazy,
+data is both read and written at the saving step and is done so in chunks.
+The way this data is divided into chunks can affect performance. By tweaking
+the way these chunks are structured it may be possible to improve performance
+when saving.
+
+
+.. _dask_rechunking:
+
+1.2 Rechunking
+^^^^^^^^^^^^^^
+We may inspect the cube's lazy data before saving:
+
+.. code-block:: python
+
+ # We can access the cubes Dask array
+ lazy_data = cube.lazy_data()
+ # We can find the shape of the chunks
+ # Note that the chunksize of a Dask array is the shape of the chunk
+ # as a tuple.
+ print(lazy_data.chunksize)
+
+Doing so, we find that the chunks currently have the shape::
+
+(1, 1, 144, 192)
+
+This is significantly smaller than the `size which Dask recommends
+`_. Bear in mind that the
+ideal chunk size depends on the platform you are running on (for this example,
+the code is being run on a desktop with 8 CPUs). In this case, we have 23460
+small chunks. We can reduce the number of chunks by rechunking before saving:
+
+.. code-block:: python
+
+ lazy_data = cube.lazy_data()
+ lazy_data = lazy_data.rechunk(1, 85, 144, 192)
+ cube.data = lazy_data
+
+We now have 276 moderately sized chunks. When we try saving again, we find
+that it is approximately 4 times faster, saving in 2m13s rather than 10m33s.
diff --git a/docs/src/further_topics/dask_best_practices/images/grib-bottleneck.png b/docs/src/further_topics/dask_best_practices/images/grib-bottleneck.png
new file mode 100644
index 0000000000..c029d57e5e
Binary files /dev/null and b/docs/src/further_topics/dask_best_practices/images/grib-bottleneck.png differ
diff --git a/docs/src/further_topics/dask_best_practices/images/loop_third_party_kapture_results.png b/docs/src/further_topics/dask_best_practices/images/loop_third_party_kapture_results.png
new file mode 100644
index 0000000000..8f388bb89c
Binary files /dev/null and b/docs/src/further_topics/dask_best_practices/images/loop_third_party_kapture_results.png differ
diff --git a/docs/src/further_topics/dask_best_practices/index.rst b/docs/src/further_topics/dask_best_practices/index.rst
new file mode 100644
index 0000000000..f126427d3f
--- /dev/null
+++ b/docs/src/further_topics/dask_best_practices/index.rst
@@ -0,0 +1,221 @@
+.. _dask best practices:
+
+Dask Best Practices
+*******************
+
+This section outlines some of the best practices when using Dask with Iris. These
+practices involve improving performance through rechunking, making the best use of
+computing clusters and avoiding parallelisation conflicts between Dask and NumPy.
+
+
+.. note::
+
+ Here, we have collated advice and a handful of examples, from the topics most
+ relevant when using Dask with Iris, that we hope will assist users to make
+ the best start when using Dask. It is *not* a fully comprehensive guide
+ encompassing all best practices. You can find more general dask information in the
+ `official Dask Documentation `_.
+
+
+Introduction
+============
+
+`Dask `_ is a powerful tool for speeding up data handling
+via lazy loading and parallel processing. To get the full benefit of using
+Dask, it is important to configure it correctly and supply it with
+appropriately structured data. For example, we may need to "chunk" data arrays
+into smaller pieces to process, read and write it; getting the "chunking" right
+can make a significant different to performance!
+
+
+.. _numpy_threads:
+
+NumPy Threads
+=============
+
+In certain scenarios NumPy will attempt to perform threading using an
+external library - typically OMP, MKL or openBLAS - making use of **every**
+CPU available. This interacts badly with Dask:
+
+* Dask may create multiple instances of NumPy, each generating enough
+ threads to use **all** the available CPUs. The resulting sharing of CPUs
+ between threads greatly reduces performance. The more cores there are, the
+ more pronounced this problem is.
+* NumPy will generate enough threads to use all available CPUs even
+ if Dask is deliberately configured to only use a subset of CPUs. The
+ resulting sharing of CPUs between threads greatly reduces performance.
+* `Dask is already designed to parallelise with NumPy arrays `_, so adding NumPy's 'competing' layer of
+ parallelisation could cause unpredictable performance.
+
+Therefore it is best to prevent NumPy performing its own parallelisation, `a
+suggestion made in Dask's own documentation `_.
+The following commands will ensure this in all scenarios:
+
+in Python...
+
+::
+
+ # Must be run before importing NumPy.
+ import os
+ os.environ["OMP_NUM_THREADS"] = "1"
+ os.environ["OPENBLAS_NUM_THREADS"] = "1"
+ os.environ["MKL_NUM_THREADS"] = "1"
+ os.environ["VECLIB_MAXIMUM_THREADS"] = "1"
+ os.environ["NUMEXPR_NUM_THREADS"] = "1"
+
+or in Linux command line...
+
+::
+
+ export OMP_NUM_THREADS=1
+ export OPENBLAS_NUM_THREADS=1
+ export MKL_NUM_THREADS=1
+ export VECLIB_MAXIMUM_THREADS=1
+ export NUMEXPR_NUM_THREADS=1
+
+
+.. _multi-pro_systems:
+
+Dask on Computing Clusters
+==========================
+
+Dask is well suited for use on computing clusters, but there are some important factors you must be
+aware of. In particular, you will always need to explicitly control parallel
+operation, both in Dask and likewise in NumPy.
+
+
+.. _multi-pro_slurm:
+
+CPU Allocation
+--------------
+
+When running on a computing cluster, unless configured otherwise, Dask will attempt to create
+one parallel 'worker' task for each CPU. However, when using a job scheduler such as Slurm, only *some* of
+these CPUs are actually accessible -- often, and by default, only one. This leads to a serious
+over-commitment unless it is controlled.
+
+So, **whenever Iris is used on a computing cluster, you must always control the number
+of dask workers to a sensible value**, matching the slurm allocation. You do
+this with::
+
+ dask.config.set(num_workers=N)
+
+For an example, see :doc:`dask_bags_and_greed`.
+
+Alternatively, when there is only one CPU allocated, it may actually be more
+efficient to use a "synchronous" scheduler instead, with::
+
+ dask.config.set(scheduler='synchronous')
+
+See the Dask documentation on `Single thread synchronous scheduler
+`_.
+
+
+.. _multi-pro_numpy:
+
+NumPy Threading
+---------------
+
+NumPy also interrogates the visible number of CPUs to multi-thread its operations.
+The large number of CPUs available in a computing cluster will thus cause confusion if NumPy
+attempts its own parallelisation, so this must be prevented. Refer back to
+:ref:`numpy_threads` for more detail.
+
+
+Distributed
+-----------
+
+Even though allocations on a computing cluster are generally restricted to a single node, there
+are still good reasons for using 'dask.distributed' in many cases. See `Single Machine: dask.distributed
+`_ in the Dask documentation.
+
+
+Chunking
+========
+
+Dask breaks down large data arrays into chunks, allowing efficient
+parallelisation by processing several smaller chunks simultaneously. For more
+information, see the documentation on
+`Dask Array `_.
+
+Iris provides a basic chunking shape to Dask, attempting to set the shape for
+best performance. The chunking that is used can depend on the file format that
+is being loaded. See below for how chunking is performed for:
+
+* :ref:`chunking_netcdf`
+* :ref:`chunking_pp_ff`
+
+It can in some cases be beneficial to re-chunk the arrays in Iris cubes.
+For information on how to do this, see :ref:`dask_rechunking`.
+
+
+.. _chunking_netcdf:
+
+NetCDF Files
+------------
+
+NetCDF files can include their own chunking specification. This is either
+specified when creating the file, or is automatically assigned if one or
+more of the dimensions is `unlimited `_.
+Importantly, netCDF chunk shapes are **not optimised for Dask
+performance**.
+
+Chunking can be set independently for any variable in a netCDF file.
+When a netCDF variable uses an unlimited dimension, it is automatically
+chunked: the chunking is the shape of the whole variable, but with '1' instead
+of the length in any unlimited dimensions.
+
+When chunking is specified for netCDF data, Iris will set the dask chunking
+to an integer multiple or fraction of that shape, such that the data size is
+near to but not exceeding the dask array chunk size.
+
+
+.. _chunking_pp_ff:
+
+PP and Fieldsfiles
+------------------
+
+PP and Fieldsfiles contain multiple 2D fields of data. When loading PP or
+Fieldsfiles into Iris cubes, the chunking will automatically be set to a chunk
+per field.
+
+For example, if a PP file contains 2D lat-lon fields for each of the
+85 model level numbers, it will load in a cube that looks as follows::
+
+ (model_level_number: 85; latitude: 144; longitude: 192)
+
+The data in this cube will be partitioned with chunks of shape
+:code:`(1, 144, 192)`.
+
+If the file(s) being loaded contain multiple fields, this can lead to an
+excessive amount of chunks which will result in poor performance.
+
+When the default chunking is not appropriate, it is possible to rechunk.
+:doc:`dask_pp_to_netcdf` provides a detailed demonstration of how Dask can optimise
+that process.
+
+
+Examples
+========
+
+We have written some examples of use cases for using Dask, that come with advice and
+explanations for why and how the tasks are performed the way they are.
+
+If you feel you have an example of a Dask best practice that you think may be helpful to others,
+please share them with us by raising a new `discussion on the Iris repository `_.
+
+* :doc:`dask_pp_to_netcdf`
+* :doc:`dask_parallel_loop`
+* :doc:`dask_bags_and_greed`
+
+.. toctree::
+ :hidden:
+ :maxdepth: 1
+
+ dask_pp_to_netcdf
+ dask_parallel_loop
+ dask_bags_and_greed
diff --git a/docs/src/further_topics/filtering_warnings.rst b/docs/src/further_topics/filtering_warnings.rst
new file mode 100644
index 0000000000..a69247008a
--- /dev/null
+++ b/docs/src/further_topics/filtering_warnings.rst
@@ -0,0 +1,271 @@
+.. _filtering-warnings:
+
+==================
+Filtering Warnings
+==================
+
+Since Iris cannot predict your specific needs, it by default raises Warnings
+for anything that might be a problem for **any** user, and is designed to work with
+you to ``ignore`` Warnings which you do not find helpful.
+
+.. testsetup:: filtering_warnings
+
+ from pathlib import Path
+ import sys
+ import warnings
+
+ import iris
+ import iris.coord_systems
+ import iris.exceptions
+
+ # Hack to ensure doctests actually see Warnings that are raised, and that
+ # they have a relative path (so a test pass is not machine-dependent).
+ warnings.filterwarnings("default")
+ IRIS_FILE = Path(iris.__file__)
+ def custom_warn(message, category, filename, lineno, file=None, line=None):
+ filepath = Path(filename)
+ filename = str(filepath.relative_to(IRIS_FILE.parents[1]))
+ sys.stdout.write(warnings.formatwarning(message, category, filename, lineno))
+ warnings.showwarning = custom_warn
+
+ geog_cs_globe = iris.coord_systems.GeogCS(6400000)
+ orthographic_coord_system = iris.coord_systems.Orthographic(
+ longitude_of_projection_origin=0,
+ latitude_of_projection_origin=0,
+ ellipsoid=geog_cs_globe,
+ )
+
+
+ def my_operation():
+ geog_cs_globe.inverse_flattening = 0.1
+ _ = orthographic_coord_system.as_cartopy_crs()
+
+Here is a hypothetical operation - ``my_operation()`` - which raises two
+Warnings:
+
+.. doctest:: filtering_warnings
+
+ >>> my_operation()
+ ...
+ iris/coord_systems.py:455: IrisUserWarning: Setting inverse_flattening does not affect other properties of the GeogCS object. To change other properties set them explicitly or create a new GeogCS instance.
+ warnings.warn(wmsg, category=iris.exceptions.IrisUserWarning)
+ iris/coord_systems.py:822: IrisDefaultingWarning: Discarding false_easting and false_northing that are not used by Cartopy.
+ warnings.warn(
+
+Warnings can be suppressed using the Python warnings filter with the ``ignore``
+action. Detailed information is available in the Python documentation:
+:external+python:mod:`warnings`.
+
+The key points are:
+
+- :ref:`When`: a warnings filter can be applied
+ either from the command line or from within Python.
+- :ref:`What`: a warnings filter accepts
+ various arguments to specify which Warnings are being filtered. Both broad
+ and narrow filters are possible.
+
+.. _warning-filter-application:
+
+**When** a Warnings Filter can be Applied
+-----------------------------------------
+
+- **Command line:** setting the :external+python:envvar:`PYTHONWARNINGS`
+ environment variable.
+- **Command line:** the `python -W `_
+ command line argument.
+- **Within Python:** use :func:`warnings.filterwarnings` .
+
+The :ref:`warning-filter-specificity` section demonstrates using
+:func:`warnings.filterwarnings`, and shows the equivalent **command line**
+approaches.
+
+
+.. _warning-filter-specificity:
+
+**What** Warnings will be Filtered
+----------------------------------
+
+.. note::
+
+ For all of these examples we are using the
+ :class:`~warnings.catch_warnings` context manager to ensure any changes to
+ settings are temporary.
+
+ This should always work fine for the ``ignore``
+ warning filter action, but note that some of the other actions
+ may not behave correctly with all Iris operations, as
+ :class:`~warnings.catch_warnings` is not thread-safe (e.g. using the
+ ``once`` action may cause 1 warning per chunk of lazy data).
+
+Specific Warnings
+~~~~~~~~~~~~~~~~~
+
+**When you do not want a specific warning, but still want all others.**
+
+You can target specific Warning messages, e.g.
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings("ignore", message="Discarding false_easting")
+ ... my_operation()
+ ...
+ iris/coord_systems.py:455: IrisUserWarning: Setting inverse_flattening does not affect other properties of the GeogCS object. To change other properties set them explicitly or create a new GeogCS instance.
+ warnings.warn(wmsg, category=iris.exceptions.IrisUserWarning)
+
+::
+
+ python -W ignore:"Discarding false_easting"
+ export PYTHONWARNINGS=ignore:"Discarding false_easting"
+
+----
+
+Or you can target Warnings raised by specific lines of specific modules, e.g.
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings("ignore", module="iris.coord_systems", lineno=455)
+ ... my_operation()
+ ...
+ iris/coord_systems.py:822: IrisDefaultingWarning: Discarding false_easting and false_northing that are not used by Cartopy.
+ warnings.warn(
+
+::
+
+ python -W ignore:::iris.coord_systems:453
+ export PYTHONWARNINGS=ignore:::iris.coord_systems:453
+
+Warnings from a Common Source
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**When you do not want ANY warnings raised by a module, or collection of
+modules.**
+
+E.g. filtering the ``coord_systems`` module:
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings("ignore", module="iris.coord_systems")
+ ... my_operation()
+
+::
+
+ python -W ignore:::iris.coord_systems
+ export PYTHONWARNINGS=ignore:::iris.coord_systems
+
+----
+
+If using :func:`warnings.filterwarnings` , you can also use partial
+definitions. The below example will ``ignore`` all Warnings from ``iris`` as a
+whole.
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings("ignore", module="iris")
+ ... my_operation()
+
+The above 'partial' filter is not available with the command line approaches.
+
+Warnings of a Common Type
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**When you do not want any Warnings of the same nature, from anywhere in the
+code you are calling.**
+
+The below example will ``ignore`` any
+:class:`~iris.exceptions.IrisDefaultingWarning` that gets raised by *any*
+module during execution:
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings(
+ ... "ignore",
+ ... category=iris.exceptions.IrisDefaultingWarning
+ ... )
+ ... my_operation()
+ ...
+ iris/coord_systems.py:455: IrisUserWarning: Setting inverse_flattening does not affect other properties of the GeogCS object. To change other properties set them explicitly or create a new GeogCS instance.
+ warnings.warn(wmsg, category=iris.exceptions.IrisUserWarning)
+
+----
+
+Using :class:`~iris.exceptions.IrisUserWarning` in the filter will ``ignore``
+both Warnings, since :class:`~iris.exceptions.IrisDefaultingWarning` subclasses
+:class:`~iris.exceptions.IrisUserWarning` :
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings(
+ ... "ignore",
+ ... category=iris.exceptions.IrisUserWarning
+ ... )
+ ... my_operation()
+
+----
+
+The command line approaches can only handle the built-in Warning
+categories (`cpython#66733`_)::
+
+ python -W ignore::UserWarning
+ export PYTHONWARNINGS=ignore::UserWarning
+
+----
+
+There are several built-in Python warning categories that can be used here
+(:class:`DeprecationWarning` being a popular example, see
+:external+python:mod:`warnings` for more). Since Iris has
+so many different warnings that might be raised, Iris subclasses
+:class:`UserWarning` to :class:`~iris.exceptions.IrisUserWarning`, which itself
+has **many** specialised subclasses. These subclasses exist to give you more
+granularity in your warning filtering; you can see the full list by
+searching the :mod:`iris.exceptions` page for ``warning`` .
+
+.. attention::
+
+ If you have ideas for adding/altering Iris' warning categories, please
+ :ref:`get in touch`! The categories exist to
+ make your life easier, and it is simple to make modifications.
+
+
+More Detail
+-----------
+
+Different people use Iris for very different purposes, from quick file
+visualisation to extract-transform-load to statistical analysis. These
+contrasting priorities mean disagreement on which Iris problems can be ignored
+and which are critically important.
+
+For problems that prevent Iris functioning: **Concrete Exceptions** are raised, which
+stop code from running any further - no debate here. For less catastrophic
+problems: **Warnings** are raised,
+which notify you (in ``stderr``) but allow code to continue running. The Warnings are
+there because Iris may **OR may not** function in the way you expect,
+depending on what you need - e.g. a problem might prevent data being saved to
+NetCDF, but statistical analysis will still work fine.
+
+Examples of Iris Warnings
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- If you attempt to plot un-bounded point data as a ``pcolormesh``: Iris will
+ guess appropriate bounds around each point so that quadrilaterals can be
+ plotted. This permanently modifies the relevant coordinates, so the you are
+ warned in case downstream operations assume un-bounded coordinates.
+- If you load a NetCDF file where a CF variable references another variable -
+ e.g. ``my_var:coordinates = "depth_var" ;`` - but the referenced variable
+ (``depth_var``) is not in the file: Iris will still construct
+ its data model, but without this reference relationship. You are warned since
+ the file includes an error and the loaded result might therefore not be as
+ expected.
+
+
+.. testcleanup:: filtering_warnings
+
+ warnings.filterwarnings("ignore")
+
+
+.. _cpython#66733: https://github.com/python/cpython/issues/66733
diff --git a/docs/src/further_topics/ugrid/data_model.rst b/docs/src/further_topics/ugrid/data_model.rst
index cc3cc7b793..208254ada6 100644
--- a/docs/src/further_topics/ugrid/data_model.rst
+++ b/docs/src/further_topics/ugrid/data_model.rst
@@ -484,20 +484,20 @@ How UGRID information is stored
| Described in detail in `MeshCoords`_.
| Stores the following information:
- * | :attr:`~iris.experimental.ugrid.MeshCoord.mesh`
- | The :class:`~iris.experimental.ugrid.Mesh` associated with this
- :class:`~iris.experimental.ugrid.MeshCoord`. This determines the
- :attr:`~iris.cube.Cube.mesh` attribute of any :class:`~iris.cube.Cube`
- this :class:`~iris.experimental.ugrid.MeshCoord` is attached to (see
- `The Basics`_)
-
- * | :attr:`~iris.experimental.ugrid.MeshCoord.location`
- | ``node``/``edge``/``face`` - the element detailed by this
- :class:`~iris.experimental.ugrid.MeshCoord`. This determines the
- :attr:`~iris.cube.Cube.location` attribute of any
- :class:`~iris.cube.Cube` this
- :class:`~iris.experimental.ugrid.MeshCoord` is attached to (see
- `The Basics`_).
+ * | :attr:`~iris.experimental.ugrid.MeshCoord.mesh`
+ | The :class:`~iris.experimental.ugrid.Mesh` associated with this
+ :class:`~iris.experimental.ugrid.MeshCoord`. This determines the
+ :attr:`~iris.cube.Cube.mesh` attribute of any :class:`~iris.cube.Cube`
+ this :class:`~iris.experimental.ugrid.MeshCoord` is attached to (see
+ `The Basics`_)
+
+ * | :attr:`~iris.experimental.ugrid.MeshCoord.location`
+ | ``node``/``edge``/``face`` - the element detailed by this
+ :class:`~iris.experimental.ugrid.MeshCoord`. This determines the
+ :attr:`~iris.cube.Cube.location` attribute of any
+ :class:`~iris.cube.Cube` this
+ :class:`~iris.experimental.ugrid.MeshCoord` is attached to (see
+ `The Basics`_).
.. _ugrid MeshCoords:
diff --git a/docs/src/further_topics/ugrid/operations.rst b/docs/src/further_topics/ugrid/operations.rst
index f0638800fa..a088b588e1 100644
--- a/docs/src/further_topics/ugrid/operations.rst
+++ b/docs/src/further_topics/ugrid/operations.rst
@@ -267,6 +267,7 @@ saves the file in a UGRID-conformant format:
edge_data:units = "K" ;
edge_data:mesh = "my_mesh" ;
edge_data:location = "edge" ;
+ edge_data:coordinates = "latitude_0 longitude_0" ;
int64 height(height) ;
height:standard_name = "height" ;
int64 face_data(Mesh2d_face, height) ;
@@ -274,6 +275,7 @@ saves the file in a UGRID-conformant format:
face_data:units = "K" ;
face_data:mesh = "my_mesh" ;
face_data:location = "face" ;
+ face_data:coordinates = "latitude_1 longitude_1" ;
// global attributes:
:Conventions = "CF-1.7" ;
diff --git a/docs/src/index.rst b/docs/src/index.rst
index 21971c2322..b353406f58 100644
--- a/docs/src/index.rst
+++ b/docs/src/index.rst
@@ -187,19 +187,20 @@ The legacy support resources:
.. toctree::
- :caption: Iris API
+ :caption: What's New in Iris
:maxdepth: 1
+ :name: whats_new_index
:hidden:
- Iris API
+ whatsnew/index
.. toctree::
- :caption: What's New in Iris
+ :caption: Iris API
:maxdepth: 1
- :name: whats_new_index
:hidden:
- whatsnew/index
+ Iris API
+
-.. todolist::
\ No newline at end of file
+.. todolist::
diff --git a/docs/src/installing.rst b/docs/src/installing.rst
index cff9a27952..a8207c37a5 100644
--- a/docs/src/installing.rst
+++ b/docs/src/installing.rst
@@ -3,30 +3,26 @@
Installing
==========
-Iris is available using conda for the following platforms:
-
-* Linux 64-bit,
-* Mac OSX 64-bit, and
-* Windows 64-bit.
-
-Windows 10 now has support for Linux distributions via WSL_ (Windows
-Subsystem for Linux). This is a great option to get started with Iris
-for users and developers. Be aware that we do not currently test against
-any WSL_ distributions.
-
-.. _WSL: https://learn.microsoft.com/en-us/windows/wsl/install
+Iris can be installed using conda or pip.
.. note:: Iris is currently supported and tested against |python_support|
running on Linux. We do not currently actively test on other
platforms such as Windows or macOS.
+ Windows 10 now has support for Linux distributions via WSL_ (Windows
+ Subsystem for Linux). This is a great option to get started with
+ Iris for users and contributors. Be aware that we do not currently
+ test against any WSL_ distributions.
+
+.. _WSL: https://learn.microsoft.com/en-us/windows/wsl/install
+
.. note:: This documentation was built using Python |python_version|.
.. _installing_using_conda:
-Installing Using Conda (Users)
-------------------------------
+Installing a Released Version Using Conda
+-----------------------------------------
To install Iris using conda, you must first download and install conda,
for example from https://docs.conda.io/en/latest/miniconda.html.
@@ -44,33 +40,24 @@ need the Iris sample data. This can also be installed using conda::
Further documentation on using conda and the features it provides can be found
at https://docs.conda.io/projects/conda/en/latest/index.html.
-.. _installing_from_source_without_conda:
-
-Installing from Source Without Conda on Debian-Based Linux Distros (Developers)
--------------------------------------------------------------------------------
+.. _installing_using_pip:
-Iris can also be installed without a conda environment. The instructions in
-this section are valid for Debian-based Linux distributions (Debian, Ubuntu,
-Kubuntu, etc.).
+Installing a Released Version Using Pip
+---------------------------------------
-Iris and its dependencies need some shared libraries in order to work properly.
-These can be installed with apt::
+Iris is also available from https://pypi.org/ so can be installed with ``pip``::
- sudo apt-get install python3-pip python3-tk libudunits2-dev libproj-dev proj-bin libgeos-dev libcunit1-dev
+ pip install scitools-iris
-The rest can be done with pip::
-
- pip3 install scitools-iris
+If you wish to run any of the code in the gallery you will also
+need the Iris sample data. This can also be installed using pip::
-This procedure was tested on a Ubuntu 20.04 system on the
-26th of July, 2021.
-Be aware that through updates of the involved Debian packages,
-dependency conflicts might arise or the procedure might have to be modified.
+ pip install iris-sample-data
.. _installing_from_source:
-Installing from Source with Conda (Developers)
-----------------------------------------------
+Installing a Development Version from a Git Checkout
+----------------------------------------------------
The latest Iris source release is available from
https://github.com/SciTools/iris.
@@ -78,9 +65,9 @@ https://github.com/SciTools/iris.
For instructions on how to obtain the Iris project source from GitHub see
:ref:`forking` and :ref:`set-up-fork` for instructions.
-Once conda is installed, you can install Iris using conda and then activate
-it. The example commands below assume you are in the root directory of your
-local copy of Iris::
+Once conda is installed, you can create a development environment for Iris
+using conda and then activate it. The example commands below assume you are in
+the root directory of your local copy of Iris::
conda env create --force --file=requirements/iris.yml
conda activate iris-dev
diff --git a/docs/src/sphinxext/api_rst_formatting.py b/docs/src/sphinxext/api_rst_formatting.py
index 8f1aa3c5f3..6dd82de91e 100644
--- a/docs/src/sphinxext/api_rst_formatting.py
+++ b/docs/src/sphinxext/api_rst_formatting.py
@@ -1,8 +1,7 @@
# Copyright Iris contributors
#
-# This file is part of Iris and is released under the LGPL license.
-# See COPYING and COPYING.LESSER in the root of the repository for full
-# licensing details.
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
# This script will process all .rst files that have been created by
# sphinxcontrib.apidoc extension and perform minor changes, specifically:
diff --git a/docs/src/techpapers/um_files_loading.rst b/docs/src/techpapers/um_files_loading.rst
index f8c94cab08..f94898b3aa 100644
--- a/docs/src/techpapers/um_files_loading.rst
+++ b/docs/src/techpapers/um_files_loading.rst
@@ -125,21 +125,21 @@ with latitude and longitude axes are also supported).
For an ordinary latitude-longitude grid, the cubes have coordinates called
'longitude' and 'latitude':
- * These are mapped to the appropriate data dimensions.
- * They have units of 'degrees'.
- * They have a coordinate system of type :class:`iris.coord_systems.GeogCS`.
- * The coordinate points are normally set to the regular sequence
- ``ZDX/Y + BDX/Y * (1 .. LBNPT/LBROW)`` (*except*, if BDX/BDY is zero, the
- values are taken from the extra data vector X/Y, if present).
- * If X/Y_LOWER_BOUNDS extra data is available, this appears as bounds values
- of the horizontal coordinates.
+* These are mapped to the appropriate data dimensions.
+* They have units of 'degrees'.
+* They have a coordinate system of type :class:`iris.coord_systems.GeogCS`.
+* The coordinate points are normally set to the regular sequence
+ ``ZDX/Y + BDX/Y * (1 .. LBNPT/LBROW)`` (*except*, if BDX/BDY is zero, the
+ values are taken from the extra data vector X/Y, if present).
+* If X/Y_LOWER_BOUNDS extra data is available, this appears as bounds values
+ of the horizontal coordinates.
For **rotated** latitude-longitude coordinates (as for LBCODE=101), the
horizontal coordinates differ only slightly --
- * The names are 'grid_latitude' and 'grid_longitude'.
- * The coord_system is a :class:`iris.coord_systems.RotatedGeogCS`, created
- with a pole defined by BPLAT, BPLON.
+* The names are 'grid_latitude' and 'grid_longitude'.
+* The coord_system is a :class:`iris.coord_systems.RotatedGeogCS`, created
+ with a pole defined by BPLAT, BPLON.
For example:
>>> # Load a PP field.
@@ -304,10 +304,9 @@ For hybrid height levels (LBVC=65):
multidimensional or non-monotonic.
See an example printout of a hybrid height cube,
-:ref:`here `:
-
- Notice that this contains all of the above coordinates --
- 'model_level_number', 'sigma', 'level_height' and the derived 'altitude'.
+:ref:`here `. Notice that this contains all of the
+above coordinates -- ``model_level_number``, ``sigma``, ``level_height`` and
+the derived ``altitude``.
.. note::
@@ -364,7 +363,7 @@ Data at a single measurement timepoint (LBTIM.IB=0):
defined according to LBTIM.IC.
Values forecast from T2, valid at T1 (LBTIM.IB=1):
- Coordinates ``time` and ``forecast_reference_time`` are created from the T1
+ Coordinates ``time`` and ``forecast_reference_time`` are created from the T1
and T2 values, respectively. These have no bounds, and units of
'hours since 1970-01-01 00:00:00', with the appropriate calendar.
A ``forecast_period`` coordinate is also created, with values T1-T2, no
@@ -383,12 +382,11 @@ these may become dimensions of the resulting data cube. This will depend on
the values actually present in the source fields for each of the elements.
See an example printout of a forecast data cube,
-:ref:`here ` :
-
- Notice that this example contains all of the above coordinates -- 'time',
- 'forecast_period' and 'forecast_reference_time'. In this case the data are
- forecasts, so 'time' is a dimension, 'forecast_period' varies with time and
- 'forecast_reference_time' is a constant.
+:ref:`here `. Notice that this example
+contains all of the above coordinates -- ``time``, ``forecast_period`` and
+``forecast_reference_time``. In this case the data are forecasts, so ``time``
+is a dimension, ``forecast_period``` varies with time and
+``forecast_reference_time`` is a constant.
Statistical Measures
diff --git a/docs/src/userguide/index.rst b/docs/src/userguide/index.rst
index fdd0c4d03e..c87323da8e 100644
--- a/docs/src/userguide/index.rst
+++ b/docs/src/userguide/index.rst
@@ -42,7 +42,9 @@ they may serve as a useful reference for future exploration.
:maxdepth: 2
:caption: Further Topics
+ ../further_topics/filtering_warnings
../further_topics/metadata
../further_topics/lenient_metadata
../further_topics/lenient_maths
+ ../further_topics/dask_best_practices/index
../further_topics/ugrid/index
diff --git a/docs/src/userguide/navigating_a_cube.rst b/docs/src/userguide/navigating_a_cube.rst
index b4c16b094b..ec3cd8e0dc 100644
--- a/docs/src/userguide/navigating_a_cube.rst
+++ b/docs/src/userguide/navigating_a_cube.rst
@@ -191,10 +191,10 @@ Adding and Removing Metadata to the Cube at Load Time
Sometimes when loading a cube problems occur when the amount of metadata is more or less than expected.
This is often caused by one of the following:
- * The file does not contain enough metadata, and therefore the cube cannot know everything about the file.
- * Some of the metadata of the file is contained in the filename, but is not part of the actual file.
- * There is not enough metadata loaded from the original file as Iris has not handled the format fully. *(in which case,
- please let us know about it)*
+* The file does not contain enough metadata, and therefore the cube cannot know everything about the file.
+* Some of the metadata of the file is contained in the filename, but is not part of the actual file.
+* There is not enough metadata loaded from the original file as Iris has not handled the format fully. *(in which case,
+ please let us know about it)*
To solve this, all of :func:`iris.load`, :func:`iris.load_cube`, and :func:`iris.load_cubes` support a callback keyword.
diff --git a/docs/src/userguide/real_and_lazy_data.rst b/docs/src/userguide/real_and_lazy_data.rst
index ef4de0c429..e4c041886c 100644
--- a/docs/src/userguide/real_and_lazy_data.rst
+++ b/docs/src/userguide/real_and_lazy_data.rst
@@ -247,20 +247,21 @@ output file, to be performed by `Dask `_ lat
thus enabling parallel save operations.
This works in the following way :
- 1. an :func:`iris.save` call is made, with a NetCDF file output and the additional
- keyword ``compute=False``.
- This is currently *only* available when saving to NetCDF, so it is documented in
- the Iris NetCDF file format API. See: :func:`iris.fileformats.netcdf.save`.
- 2. the call creates the output file, but does not fill in variables' data, where
- the data is a lazy array in the Iris object. Instead, these variables are
- initially created "empty".
+1. an :func:`iris.save` call is made, with a NetCDF file output and the additional
+ keyword ``compute=False``.
+ This is currently *only* available when saving to NetCDF, so it is documented in
+ the Iris NetCDF file format API. See: :func:`iris.fileformats.netcdf.save`.
- 3. the :meth:`~iris.save` call returns a ``result`` which is a
- :class:`~dask.delayed.Delayed` object.
+2. the call creates the output file, but does not fill in variables' data, where
+ the data is a lazy array in the Iris object. Instead, these variables are
+ initially created "empty".
- 4. the save can be completed later by calling ``result.compute()``, or by passing it
- to the :func:`dask.compute` call.
+3. the :meth:`~iris.save` call returns a ``result`` which is a
+ :class:`~dask.delayed.Delayed` object.
+
+4. the save can be completed later by calling ``result.compute()``, or by passing it
+ to the :func:`dask.compute` call.
The benefit of this, is that costly data transfer operations can be performed in
parallel with writes to other data files. Also, where array contents are calculated
diff --git a/docs/src/voted_issues.rst b/docs/src/voted_issues.rst
index 0c99638bbd..33d1982a7b 100644
--- a/docs/src/voted_issues.rst
+++ b/docs/src/voted_issues.rst
@@ -53,5 +53,3 @@ the below table.
.. note:: The data in this table is updated every 30 minutes and is sourced
from `voted-issues.json`_.
For the latest data please see the `issues on GitHub`_.
- Note that the list on Github does not show the number of votes 👍
- only the total number of comments for the whole issue.
\ No newline at end of file
diff --git a/docs/src/whatsnew/1.4.rst b/docs/src/whatsnew/1.4.rst
index 989198296c..912a1e3bad 100644
--- a/docs/src/whatsnew/1.4.rst
+++ b/docs/src/whatsnew/1.4.rst
@@ -58,7 +58,7 @@ Features
* Use the latest release of Cartopy, v0.8.0.
-.. _OPeNDAP: http://www.opendap.org/about
+.. _OPeNDAP: http://www.opendap.org
.. _exp-regrid:
Experimental Regridding Enhancements
diff --git a/docs/src/whatsnew/3.6.rst b/docs/src/whatsnew/3.6.rst
index 151c63ef51..389356df46 100644
--- a/docs/src/whatsnew/3.6.rst
+++ b/docs/src/whatsnew/3.6.rst
@@ -55,6 +55,56 @@ This document explains the changes made to Iris for this release
or feature requests for improving Iris. Enjoy!
+v3.6.1 (26 June 2023)
+=====================
+
+.. dropdown:: v3.6.1 Patches
+ :color: primary
+ :icon: alert
+ :animate: fade-in
+
+ 📢 **Announcements**
+
+ Welcome and congratulations to `@sloosvel`_ who made their first contribution to
+ Iris! 🎉
+
+ The patches in this release of Iris include:
+
+ ✨ **Features**
+
+ #. `@rcomer`_ rewrote :func:`~iris.util.broadcast_to_shape` so it now handles
+ lazy data. This pull-request has been included to support :pull:`5341`.
+ (:pull:`5307`) [``pre-v3.7.0``]
+
+ 🐛 **Bugs Fixed**
+
+ #. `@stephenworsley`_ fixed :meth:`~iris.cube.Cube.convert_units` to allow unit
+ conversion of lazy data when using a `Distributed`_ scheduler.
+ (:issue:`5347`, :pull:`5349`)
+
+ #. `@schlunma`_ fixed a bug in the concatenation of cubes with aux factories
+ which could lead to a `KeyError` due to dependencies that have not been
+ properly updated.
+ (:issue:`5339`, :pull:`5340`)
+
+ #. `@schlunma`_ fixed a bug which realized all weights during weighted
+ aggregation. Now weighted aggregation is fully lazy again.
+ (:issue:`5338`, :pull:`5341`)
+
+ 🚀 **Performance Enhancements**
+
+ #. `@sloosvel`_ improved :meth:`~iris.cube.CubeList.concatenate_cube` and
+ :meth:`~iris.cube.CubeList.concatenate` to ensure that lazy auxiliary coordinate
+ points and bounds are not realized. This change now allows cubes with
+ high-resolution auxiliary coordinates to concatenate successfully whilst using a
+ minimal in-core memory footprint.
+ (:issue:`5115`, :pull:`5142`)
+
+ Note that, the above contribution labelled with ``pre-v3.7.0`` is part of the
+ forthcoming Iris ``v3.7.0`` release, but requires to be included in this patch
+ release.
+
+
📢 Announcements
================
@@ -169,6 +219,7 @@ This document explains the changes made to Iris for this release
core dev names are automatically included by the common_links.inc:
.. _@fnattino: https://github.com/fnattino
+.. _@sloosvel: https://github.com/sloosvel
.. comment
@@ -180,4 +231,5 @@ This document explains the changes made to Iris for this release
.. _PEP-0621: https://peps.python.org/pep-0621/
.. _pypa/build: https://pypa-build.readthedocs.io/en/stable/
.. _NEP29: https://numpy.org/neps/nep-0029-deprecation_policy.html
-.. _Contributor Covenant: https://www.contributor-covenant.org/version/2/1/code_of_conduct/
\ No newline at end of file
+.. _Contributor Covenant: https://www.contributor-covenant.org/version/2/1/code_of_conduct/
+.. _Distributed: https://distributed.dask.org/en/stable/
diff --git a/docs/src/whatsnew/3.7.rst b/docs/src/whatsnew/3.7.rst
new file mode 100644
index 0000000000..f1c7fb5f2c
--- /dev/null
+++ b/docs/src/whatsnew/3.7.rst
@@ -0,0 +1,158 @@
+.. include:: ../common_links.inc
+
+v3.7 (16 Aug 2023)
+******************
+
+This document explains the changes made to Iris for this release
+(:doc:`View all changes `.)
+
+
+.. dropdown:: v3.7 Release Highlights
+ :color: primary
+ :icon: info
+ :animate: fade-in
+ :open:
+
+ There are no major feature highlights for this release of Iris, but it's worth
+ noting that, in addition to some important bug fixes in specific areas, this time
+ we have made a number of improvements for user-experience and usability,
+ notably :
+
+ * improved messaging for :ref:`CubeList.concatenate() `
+ and :ref:`Cube.convert_units() `.
+
+ * avoid warnings which may occur in :ref:`pp loading `
+ and :ref:`contourf `.
+
+ * :ref:`documentation supports Dark mode `.
+
+ * :ref:`added a "Dask Best Practices" guide `
+ ( :ref:`here ` ) .
+
+ * :ref:`improved the Installation Guide `.
+
+ Please do get in touch with us on :issue:`GitHub` if you have
+ any issues or feature requests for improving Iris. Enjoy!
+
+
+📢 Announcements
+================
+
+#. N/A
+
+
+✨ Features
+===========
+
+#. `@rcomer`_ rewrote :func:`~iris.util.broadcast_to_shape` so it now handles
+ lazy data. (:pull:`5307`)
+
+ .. _concat_warnings:
+
+#. `@acchamber`_ added error and warning messages about coordinate overlaps to
+ :func:`~iris.cube.CubeList.concatenate` to improve the concatenation process.
+ (:pull:`5382`)
+
+#. `@trexfeathers`_ included mesh location coordinates
+ (e.g. :attr:`~iris.experimental.ugrid.Mesh.face_coords`) in
+ the data variable's ``coordinates`` attribute when saving to NetCDF.
+ (:issue:`5206`, :pull:`5389`)
+
+#. `@pp-mo`_ modified the install process to record the release version of the CF
+ standard-names table, when it creates the ``iris/std_names.py`` module.
+ The release number is also now available as
+ ``iris.std_names.CF_STANDARD_NAMES_TABLE_VERSION``.
+ (:pull:`5423`)
+
+
+🐛 Bugs Fixed
+=============
+
+#. `@acchamber`_ fixed a bug with :func:`~iris.util.unify_time_units` so it does not block
+ concatenation through different data types in rare instances. (:pull:`5372`)
+
+#. `@acchamber`_ removed some obsolete code that prevented extraction of time points
+ from cubes with bounded times (:pull:`5175`)
+
+ .. _cftime_warnings:
+
+#. `@rcomer`_ modified pp-loading to avoid a ``cftime`` warning for non-standard
+ calendars. (:pull:`5357`)
+
+#. `@rsdavies`_ modified the CF compliant standard name for m01s00i023 (:issue:`4566`)
+
+
+💣 Incompatible Changes
+=======================
+
+#. N/A
+
+
+🚀 Performance Enhancements
+===========================
+
+#. `@rcomer`_ made :meth:`~iris.cube.Cube.aggregated_by` faster. (:pull:`4970`)
+
+🔥 Deprecations
+===============
+
+#. N/A
+
+
+🔗 Dependencies
+===============
+
+#. N/A
+
+📚 Documentation
+================
+.. _docs_dark:
+
+#. `@tkknight`_ prepared the documentation for dark mode and enable the option
+ to use it. By default the theme will be based on the users system settings,
+ defaulting to ``light`` if no system setting is found. (:pull:`5299`)
+
+ .. _dask_guide:
+
+#. `@HGWright`_ added a :doc:`/further_topics/dask_best_practices/index`
+ section into the user guide, containing advice and use cases to help users
+ get the best out of Dask with Iris. (:pull:`5190`)
+
+ .. _convert_docs:
+
+#. `@acchamber`_ improved documentation for :meth:`~iris.cube.Cube.convert_units`
+ and :meth:`~iris.coords.Coord.convert_units` by including a link to the UDUNITS-2
+ documentation which contains lists of compatible units and aliases for them.
+ (:pull:`5388`)
+
+ .. _installdocs_update:
+
+#. `@rcomer`_ updated the :ref:`Installation Guide` to reflect
+ that some things are now simpler. (:pull:`5416`)
+
+
+💼 Internal
+===========
+
+#. `@pp-mo`_ supported loading and saving netcdf :class:`netCDF4.Dataset` compatible
+ objects in place of file-paths, as hooks for a forthcoming
+ `"Xarray bridge" `_ facility.
+ (:pull:`5214`, :pull:`5212`)
+
+ .. _contour_future:
+
+#. `@rcomer`_ updated :func:`~iris.plot.contourf` to avoid using functionality
+ that is deprecated in Matplotlib v3.8 (:pull:`5405`)
+
+
+
+.. comment
+ Whatsnew author names (@github name) in alphabetical order. Note that,
+ core dev names are automatically included by the common_links.inc:
+.. _@rsdavies: https://github.com/rsdavies
+.. _@acchamber: https://github.com/acchamber
+
+
+
+.. comment
+ Whatsnew resources in alphabetical order:
diff --git a/docs/src/whatsnew/index.rst b/docs/src/whatsnew/index.rst
index dce7458a13..c556f82761 100644
--- a/docs/src/whatsnew/index.rst
+++ b/docs/src/whatsnew/index.rst
@@ -12,6 +12,7 @@ What's New in Iris
:hidden:
latest.rst
+ 3.7.rst
3.6.rst
3.5.rst
3.4.rst
diff --git a/docs/src/whatsnew/latest.rst b/docs/src/whatsnew/latest.rst
index 8b4e9f549d..46274ba1ac 100644
--- a/docs/src/whatsnew/latest.rst
+++ b/docs/src/whatsnew/latest.rst
@@ -24,7 +24,7 @@ This document explains the changes made to Iris for this release
📢 Announcements
================
-#. N/A
+#. `@lbdreyer`_ relicensed Iris from LGPL-3 to BSD-3. (:pull: `5577`)
✨ Features
@@ -32,7 +32,23 @@ This document explains the changes made to Iris for this release
#. `@rcomer`_ rewrote :func:`~iris.util.broadcast_to_shape` so it now handles
lazy data. (:pull:`5307`)
-#. `@HGWright`_ added :func:`~iris.coords.Coord.ignore_axis` to allow manual
+
+#. `@trexfeathers`_ and `@HGWright`_ (reviewer) sub-categorised all Iris'
+ :class:`UserWarning`\s for richer filtering. The full index of
+ sub-categories can be seen here: :mod:`iris.exceptions` . (:pull:`5498`)
+
+#. `@trexfeathers`_ added the :class:`~iris.coord_systems.ObliqueMercator`
+ and :class:`~iris.coord_systems.RotatedMercator` coordinate systems,
+ complete with NetCDF loading and saving. (:pull:`5548`)
+
+#. `@trexfeathers`_ added the ``use_year_at_season_start`` parameter to
+ :func:`iris.coord_categorisation.add_season_year`. When
+ ``use_year_at_season_start==True``: seasons spanning the year boundary (e.g.
+ Winter - December to February) will be assigned to the preceding year (e.g.
+ the year of December) instead of the following year (the default behaviour).
+ (:pull:`5573`)
+
+ #. `@HGWright`_ added :func:`~iris.coords.Coord.ignore_axis` to allow manual
intervention preventing :func:`~iris.util.guess_coord_axis` from acting on a
coordinate. (:pull:`5551`)
@@ -40,7 +56,14 @@ This document explains the changes made to Iris for this release
🐛 Bugs Fixed
=============
-#. N/A
+#. `@scottrobinson02`_ fixed the output units when dividing a coordinate by a
+ cube. (:issue:`5305`, :pull:`5331`)
+
+#. `@ESadek-MO`_ has updated :mod:`iris.tests.graphics.idiff` to stop duplicated file names
+ preventing acceptance. (:issue:`5098`, :pull:`5482`)
+
+#. `@acchamber`_ and `@rcomer`_ modified 2D plots so that time axes and their
+ ticks have more sensible default labels. (:issue:`5426`, :pull:`5561`)
💣 Incompatible Changes
@@ -52,8 +75,8 @@ This document explains the changes made to Iris for this release
🚀 Performance Enhancements
===========================
-#. `@rcomer`_ made :meth:`~iris.cube.Cube.aggregated_by` faster. (:pull:`4970`)
-#. `@rsdavies`_ modified the CF compliant standard name for m01s00i023 :issue:`4566`
+#. N/A
+
🔥 Deprecations
===============
@@ -64,32 +87,71 @@ This document explains the changes made to Iris for this release
🔗 Dependencies
===============
-#. N/A
+#. `@bjlittle`_ enforced the minimum pin of ``numpy>1.21`` in accordance with the `NEP29 Drop Schedule`_.
+ (:pull:`5525`)
📚 Documentation
================
-#. `@tkknight`_ prepared the documentation for dark mode and enable the option
- to use it. By default the theme will be based on the users system settings,
- defaulting to ``light`` if no system setting is found. (:pull:`5299`)
+#. `@trexfeathers`_ documented the intended use of warnings filtering with
+ Iris. See :ref:`filtering-warnings`. (:pull:`5509`)
+
+#. `@rcomer`_ updated the
+ :ref:`sphx_glr_generated_gallery_meteorology_plot_COP_maps.py` to show how
+ a colourbar may steal space from multiple axes. (:pull:`5537`)
+
+#. `@tkknight`_ improved the top navgation bar alignment and amount of
+ links shown. Also improved how the warning banner is implemented.
+ (:pull:`5505` and :pull:`5508`)
+
+#. `@tkknight`_ removed broken git links. (:pull:`5569`)
+
+#. `@ESadek-MO`_ added a phrasebook for synonymous terms used in similar
+ packages. (:pull:`5564`)
💼 Internal
===========
-#. `@pp-mo`_ supported loading and saving netcdf :class:`netCDF4.Dataset` compatible
- objects in place of file-paths, as hooks for a forthcoming
- `"Xarray bridge" `_ facility.
- (:pull:`5214`)
+#. `@trexfeathers`_ and `@ESadek-MO`_ (reviewer) performed a suite of fixes and
+ improvements for benchmarking, primarily to get
+ :ref:`on demand pull request benchmarking `
+ working properly. (Main pull request: :pull:`5437`, more detail:
+ :pull:`5430`, :pull:`5431`, :pull:`5432`, :pull:`5434`, :pull:`5436`)
+
+#. `@trexfeathers`_ set a number of memory benchmarks to be on-demand, as they
+ were vulnerable to false positives in CI runs. (:pull:`5481`)
+
+#. `@acchamber`_ and `@ESadek-MO`_ resolved several deprecation to reduce
+ number of warnings raised during tests.
+ (:pull:`5493`, :pull:`5511`)
+
+#. `@trexfeathers`_ replaced all uses of the ``logging.WARNING`` level, in
+ favour of using Python warnings, following team agreement. (:pull:`5488`)
+
+#. `@trexfeathers`_ adapted benchmarking to work with ASV ``>=v0.6`` by no
+ longer using the ``--strict`` argument. (:pull:`5496`)
+
+#. `@fazledyn-or`_ replaced ``NotImplementedError`` with ``NotImplemented`` as
+ a proper method call. (:pull:`5544`)
+
+#. `@bjlittle`_ corrected various comment spelling mistakes detected by
+ `codespell`_. (:pull:`5546`)
.. comment
Whatsnew author names (@github name) in alphabetical order. Note that,
core dev names are automatically included by the common_links.inc:
-.. _@rsdavies: https://github.com/rsdavies
+.. _@scottrobinson02: https://github.com/scottrobinson02
+.. _@acchamber: https://github.com/acchamber
+.. _@fazledyn-or: https://github.com/fazledyn-or
.. comment
Whatsnew resources in alphabetical order:
+
+.. _NEP29 Drop Schedule: https://numpy.org/neps/nep-0029-deprecation_policy.html#drop-schedule
+.. _codespell: https://github.com/codespell-project/codespell
+
diff --git a/etc/cf-standard-name-table.xml b/etc/cf-standard-name-table.xml
index 3b145ae86e..6e3c014849 100644
--- a/etc/cf-standard-name-table.xml
+++ b/etc/cf-standard-name-table.xml
@@ -1,7 +1,7 @@
- 81
- 2023-04-25T10:43:33Z
+ 82
+ 2023-07-06T13:17:07Z
Centre for Environmental Data Analysis
support@ceda.ac.uk
@@ -269,14 +269,14 @@
m
- The altitude at top of atmosphere boundary layer is the elevation above sea level of the top of the (atmosphere) planetary boundary layer. The phrase "defined_by" provides the information of the tracer used for identifying the atmospheric boundary layer top. "Ambient_aerosol" means that the aerosol is measured or modelled at the ambient state of pressure, temperature and relative humidity that exists in its immediate environment. "Ambient aerosol particles" are aerosol particles that have taken up ambient water through hygroscopic growth. The extent of hygroscopic growth depends on the relative humidity and the composition of the particles. "By ranging instrument" means that the backscattering is obtained through ranging techniques like lidar and radar.
+ The altitude at top of atmosphere boundary layer is the elevation above sea level of the top of the (atmosphere) planetary boundary layer. The phrase "defined_by" provides the information of the tracer used for identifying the atmospheric boundary layer top. "Ambient_aerosol" means that the aerosol is measured or modelled at the ambient state of pressure, temperature and relative humidity that exists in its immediate environment. "Ambient aerosol particles" are aerosol particles that have taken up ambient water through hygroscopic growth. The extent of hygroscopic growth depends on the relative humidity and the composition of the particles. "By ranging instrument" means that the backscattering is obtained through ranging techniques like lidar and radar.
m
- The altitude at top of atmosphere mixed layer is the elevation above sea level of the top of the (atmosphere) mixed layer or convective boundary layer. The phrase "defined_by" provides the information of the tracer used for identifying the atmospheric boundary layer top. "Ambient_aerosol" means that the aerosol is measured or modelled at the ambient state of pressure, temperature and relative humidity that exists in its immediate environment. "Ambient aerosol particles" are aerosol particles that have taken up ambient water through hygroscopic growth. The extent of hygroscopic growth depends on the relative humidity and the composition of the particles. "By ranging instrument" means that the volume backscattering coefficient is obtained through ranging techniques like lidar and radar.
+ The altitude at top of atmosphere mixed layer is the elevation above sea level of the top of the (atmosphere) mixed layer or convective boundary layer. The phrase "defined_by" provides the information of the tracer used for identifying the atmospheric boundary layer top. "Ambient_aerosol" means that the aerosol is measured or modelled at the ambient state of pressure, temperature and relative humidity that exists in its immediate environment. "Ambient aerosol particles" are aerosol particles that have taken up ambient water through hygroscopic growth. The extent of hygroscopic growth depends on the relative humidity and the composition of the particles. "By ranging instrument" means that the volume backscattering coefficient is obtained through ranging techniques like lidar and radar.
@@ -469,21 +469,21 @@
- J kg -1
+ J kg-1
Convective(ly) available potential energy (often abbreviated CAPE) is a stability measure calculated by integrating the positive temperature difference between the surrounding atmosphere and a parcel of air lifted adiabatically from the surface to its equilibrium level. CAPE exists under conditions of potential instability, and measures the potential energy per unit mass that would be released by the unstable parcel if it were able to convect upwards to equilibrium.
- J kg -1
+ J kg-1
Convective inhibition is the amount of energy per unit mass required to overcome the negatively buoyant energy exerted by the environment on a parcel of air. Convective inhibition is often abbreviated as "CIN" or "CINH". It is calculated by integrating the negative temperature difference between the surrounding atmosphere and a parcel of air lifted adiabatically from a given starting height to its equilibrium level. A coordinate variable of original_air_pressure_of_lifted_parcel should be specified to indicate the starting height of the lifted parcel.
- J kg -1
+ J kg-1
Convective inhibition is the amount of energy per unit mass required to overcome the negatively buoyant energy exerted by the environment on a parcel of air. Convective inhibition is often abbreviated as "CIN" or "CINH". It is calculated by integrating the negative temperature difference between the surrounding atmosphere and a parcel of air lifted adiabatically from the surface to its equilibrium level.
@@ -3170,6 +3170,20 @@
The phrase "ratio_of_X_to_Y" means X/Y. It may be expressed as a fraction, a percentage, or any other dimensionless representation of a fraction. Also known as specific gravity, where soil represents a dry soil sample. The density of a substance is its mass per unit volume.
+
+ degree_north
+
+
+ The latitude of deployment of a station or instrument. The term can be used whenever the deployment position of a station or instrument needs to be supplied along with other types of positions. If a data variable has only one latitude coordinate variable, the standard name of latitude should generally be preferred to deployment_latitude, because latitude is recognised by generic software. If the deployment latitude is also the nominal latitude for a discrete geometry (as in Section 9.5 of the CF convention), the deployment latitude should also, or instead, be recorded in a coordinate variable with the standard name of latitude and axis="Y". Latitude is positive northward; its units of "degree_north" (or equivalent) indicate this explicitly.
+
+
+
+ degree_east
+
+
+ The longitude of deployment of a station or instrument. The term can be used whenever the deployment position of a station or instrument needs to be supplied along with other types of positions. If a data variable has only one longitude coordinate variable, the standard name of longitude should generally be preferred to deployment_longitude, because longitude is recognised by generic software. If the deployment longitude is also the nominal longitude for a discrete geometry (as in Section 9.5 of the CF convention), the deployment longitude should also, or instead, be recorded in a coordinate variable with the standard name of longitude and axis="X". Longitude is positive eastward; its units of "degree_east" (or equivalent) indicate this explicitly.
+
+
m
@@ -3720,7 +3734,7 @@
kg m-2
- The quantity with standard name drainage_amount_through_base_of_soil_model is the amount of water that drains through the bottom of a soil column extending from the surface to a specified depth. “Drainage” is the process of removal of excess water from soil by gravitational flow. "Amount" means mass per unit area. A vertical coordinate variable or scalar coordinate with standard name "depth" should be used to specify the depth to which the soil column extends.
+ The quantity with standard name drainage_amount_through_base_of_soil_model is the amount of water that drains through the bottom of a soil column extending from the surface to a specified depth. "Drainage" is the process of removal of excess water from soil by gravitational flow. "Amount" means mass per unit area. A vertical coordinate variable or scalar coordinate with standard name "depth" should be used to specify the depth to which the soil column extends.
@@ -4437,6 +4451,13 @@
Longitude is positive eastward; its units of degree_east (or equivalent) indicate this explicitly. In a latitude-longitude system defined with respect to a rotated North Pole, the standard name of grid_longitude should be used instead of longitude. Grid longitude is positive in the grid-eastward direction, but its units should be plain degree.
+
+ mol m-3 s-1
+
+
+ "Gross mole production" means the rate of creation of biomass per unit volume with no correction for respiration loss in terms of quantity of matter (moles). The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. "Prokaryotes" means all Bacteria and Archaea excluding photosynthetic cyanobacteria such as Synechococcus and Prochlorococcus or other separately named components of the prokaryotic population.
+
+
kg m-2 s-1
@@ -4458,6 +4479,13 @@
"Production of carbon" means the production of biomass expressed as the mass of carbon which it contains. Gross primary production is the rate of synthesis of biomass from inorganic precursors by autotrophs ("producers"), for example, photosynthesis in plants or phytoplankton. The producers also respire some of this biomass and the difference is "net_primary_production". "Productivity" means production per unit area. The phrase "expressed_as" is used in the construction A_expressed_as_B, where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A.
+
+ kg m-3 s-1
+
+
+ "Gross production" means the rate of creation of biomass per unit volume with no correction for respiration. The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. "Prokaryotes" means all Bacteria and Archaea excluding photosynthetic cyanobacteria such as Synechococcus and Prochlorococcus or other separately named components of the prokaryotic population.
+
+
1
@@ -7854,7 +7882,7 @@
- J Kg-1
+ J kg-1
The lightning_potential_index measures the potential for charge generation and separation that leads to lightning flashes in convective thunderstorms. It is derived from the model simulated grid-scale updraft velocity and the mass mixing-ratios of liquid water, cloud ice, snow, and graupel.
@@ -8539,6 +8567,13 @@
Mass concentration means mass per unit volume and is used in the construction mass_concentration_of_X_in_Y, where X is a material constituent of Y. A chemical species denoted by X may be described by a single term such as 'nitrogen' or a phrase such as 'nox_expressed_as_nitrogen'. The chemical formula for chlorine nitrate is ClONO2.
+
+ kg m-3
+
+
+ "Mass concentration" means mass per unit volume and is used in the construction "mass_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". Chlorophylls are the green pigments found in most plants, algae and cyanobacteria; their presence is essential for photosynthesis to take place. There are several different forms of chlorophyll that occur naturally. All contain a chlorin ring (chemical formula C20H16N4) which gives the green pigment and a side chain whose structure varies. The naturally occurring forms of chlorophyll contain between 35 and 55 carbon atoms. Chlorophyll-a is the most commonly occurring form of natural chlorophyll. The chemical formula of chlorophyll-a is C55H72O5N4Mg. "Sea ice" means all ice floating in the sea which has formed from freezing sea water, rather than by other processes such as calving of land ice to form icebergs.
+
+
kg m-3
@@ -9260,6 +9295,13 @@
Mass concentration means mass per unit volume and is used in the construction mass_concentration_of_X_in_Y, where X is a material constituent of Y. It means the ratio of the mass of X to the mass of Y (including X). A chemical species denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". Petroleum hydrocarbons are compounds containing just carbon and hydrogen originating from the fossil fuel crude oil.
+
+ kg m-3
+
+
+ "Mass concentration" means mass per unit volume and is used in the construction "mass_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". Phaeopigments are non-photosynthetic pigments that are the degradation product of algal chlorophyll pigments. It is commonly formed during and after marine phytoplankton blooms. "Sea ice" means all ice floating in the sea which has formed from freezing sea water, rather than by other processes such as calving of land ice to form icebergs.
+
+
kg m-3
@@ -9351,6 +9393,13 @@
Mass concentration means mass per unit volume and is used in the construction mass_concentration_of_X_in_Y, where X is a material constituent of Y. A chemical species denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". "Aerosol" means the system of suspended liquid or solid particles in air (except cloud droplets) and their carrier gas, the air itself. Aerosol takes up ambient water (a process known as hygroscopic growth) depending on the relative humidity and the composition of the aerosol. "Dry aerosol particles" means aerosol particles without any water uptake. "Primary particulate organic matter " means all organic matter emitted directly to the atmosphere as particles except elemental carbon. The sum of primary_particulate_organic_matter_dry_aerosol and secondary_particulate_organic_matter_dry_aerosol is particulate_organic_matter_dry_aerosol.
+
+ kg m-3
+
+
+ "Mass concentration" means mass per unit volume and is used in the construction "mass_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. "Prokaryotes" means all Bacteria and Archaea excluding photosynthetic cyanobacteria such as Synechococcus and Prochlorococcus or other separately named components of the prokaryotic population.
+
+
kg m-3
@@ -11669,7 +11718,7 @@
- mol/m3
+ mol m-3
"Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". "Dissolved organic nitrogen" describes the nitrogen held in carbon compounds in solution. These are mostly generated by plankton excretion and decay.
@@ -12123,6 +12172,13 @@
Mole concentration means number of moles per unit volume, also called "molarity", and is used in the construction mole_concentration_of_X_in_Y, where X is a material constituent of Y. A chemical species denoted by X may be described by a single term such as 'nitrogen' or a phrase such as 'nox_expressed_as_nitrogen'. The chemical formula for ozone is O3.
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes.
+
+
mol m-3
@@ -12130,6 +12186,41 @@
"Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A.
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes.
+
+
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes.
+
+
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes.
+
+
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes. Phosphorus means phosphorus in all chemical forms, commonly referred to as "total phosphorus".
+
+
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes.
+
+
mol m-3
@@ -12137,6 +12228,27 @@
Mole concentration means number of moles per unit volume, also called"molarity", and is used in the construction mole_concentration_of_X_in_Y, where X is a material constituent of Y. A chemical or biological species denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction A_expressed_as_B, where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A.
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes.
+
+
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes.
+
+
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Particulate means suspended solids of all sizes.
+
+
mol m-3
@@ -12235,6 +12347,13 @@
Mole concentration means number of moles per unit volume, also called "molarity", and is used in the construction mole_concentration_of_X_in_Y, where X is a material constituent of Y. A chemical or biological species denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction A_expressed_as_B, where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. Picophytoplankton are phytoplankton of less than 2 micrometers in size. Phytoplankton are algae that grow where there is sufficient light to support photosynthesis.
+
+ mol m-3
+
+
+ "Mole concentration" means number of moles per unit volume, also called "molarity", and is used in the construction "mole_concentration_of_X_in_Y", where X is a material constituent of Y. A chemical species or biological group denoted by X may be described by a single term such as "nitrogen" or a phrase such as "nox_expressed_as_nitrogen". The phrase "expressed_as" is used in the construction "A_expressed_as_B", where B is a chemical constituent of A. It means that the quantity indicated by the standard name is calculated solely with respect to the B contained in A, neglecting all other chemical constituents of A. "Prokaryotes" means all Bacteria and Archaea excluding photosynthetic cyanobacteria such as Synechococcus and Prochlorococcus or other separately named components of the prokaryotic population.
+
+
mol m-3
@@ -13877,7 +13996,7 @@
W
- "Northward" indicates a vector component which is positive when directed northward (negative southward). The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection.
+ "Northward" indicates a vector component which is positive when directed northward (negative southward). The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection.
@@ -14409,7 +14528,7 @@
kg s-1
- The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection.
+ The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection.
@@ -14759,7 +14878,7 @@
kg s-1
- "y" indicates a vector component along the grid y-axis, positive with increasing y. The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection.
+ "y" indicates a vector component along the grid y-axis, positive with increasing y. The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection.
@@ -14920,7 +15039,7 @@
m s-1
- "Heave rate" is the rate of displacement along the local vertical axis. Heave rate might not include changes to the “at rest” position of the platform with respect to the axis of displacement, which may change over time. The standard name platform_heave_rate should be chosen only if the sign convention of the data is unknown. For cases where the sign convention of the heave rate is known, a standard name of platform_heave_rate_down or platform_heave_rate_up should be chosen, as appropriate. A "platform" is a structure or vehicle that serves as a base for mounting sensors. Platforms include, but are not limited to, satellites, aeroplanes, ships, buoys, instruments, ground stations, and masts.
+ "Heave rate" is the rate of displacement along the local vertical axis. Heave rate might not include changes to the "at rest" position of the platform with respect to the axis of displacement, which may change over time. The standard name platform_heave_rate should be chosen only if the sign convention of the data is unknown. For cases where the sign convention of the heave rate is known, a standard name of platform_heave_rate_down or platform_heave_rate_up should be chosen, as appropriate. A "platform" is a structure or vehicle that serves as a base for mounting sensors. Platforms include, but are not limited to, satellites, aeroplanes, ships, buoys, instruments, ground stations, and masts.
@@ -18154,7 +18273,7 @@
sr
- The ratio of volume extinction coefficient to volume backwards scattering coefficient by ranging instrument in air due to ambient aerosol particles (often called "lidar ratio") is the ratio of the "volume extinction coefficient" and the "volume backwards scattering coefficient of radiative flux by ranging instrument in air due to ambient aerosol particles". The ratio is assumed to be related to the same wavelength as the incident radiation. "Ambient_aerosol" means that the aerosol is measured or modelled at the ambient state of pressure, temperature and relative humidity that exists in its immediate environment. "Ambient aerosol particles" are aerosol particles that have taken up ambient water through hygroscopic growth. The extent of hygroscopic growth depends on the relative humidity and the composition of the particles. The specification of a physical process by the phrase "due_to_" process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase.
+ The ratio of volume extinction coefficient to volume backwards scattering coefficient by ranging instrument in air due to ambient aerosol particles (often called "lidar ratio") is the ratio of the "volume extinction coefficient" and the "volume backwards scattering coefficient of radiative flux by ranging instrument in air due to ambient aerosol particles". The ratio is assumed to be related to the same wavelength of incident radiation. "Ambient_aerosol" means that the aerosol is measured or modelled at the ambient state of pressure, temperature and relative humidity that exists in its immediate environment. "Ambient aerosol particles" are aerosol particles that have taken up ambient water through hygroscopic growth. The extent of hygroscopic growth depends on the relative humidity and the composition of the particles. The specification of a physical process by the phrase "due_to_" process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase.
@@ -18388,6 +18507,13 @@
The sea_floor_depth_below_sea_surface is the vertical distance between the sea surface and the seabed as measured at a given point in space including the variance caused by tides and possibly waves.
+
+ s
+
+
+ "Sea floor sediment" is sediment deposited at the sea bed. "Sediment age" means the length of time elapsed since the sediment was deposited. The phrase "before_1950" is a transparent representation of the phrase "before_present", often used in the geological and archaeological domains to refer to time elapsed between an event and 1950 AD.
+
+
m
@@ -19704,6 +19830,13 @@
Speed is the magnitude of velocity. The specification of a physical process by the phrase "due_to_" process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. "Due to tides" means due to all astronomical gravity changes which manifest as tides. No distinction is made between different tidal components.
+
+ s-1
+
+
+ Speed is the magnitude of velocity. Sea water speed shear is the derivative of sea water speed with respect to depth.
+
+
K
80
@@ -20355,6 +20488,13 @@
"specific" means per unit mass. Dry energy is the sum of dry static energy and kinetic energy. Dry static energy is the sum of enthalpy and potential energy (itself the sum of gravitational and centripetal potential energy). Enthalpy can be written either as (1) CpT, where Cp is heat capacity at constant pressure, T is absolute temperature, or (2) U+pV, where U is internal energy, p is pressure and V is volume.
+
+ J kg-1
+
+
+ The specific_enthalpy_of_air is the enthalpy of air per unit mass, which can be computed for an air sample as the sum of the enthalpy of the dry air and the enthalpy of the water vapor in that air, divided by the mass of dry air.
+
+
J kg-1
@@ -20404,6 +20544,13 @@
"Specific" means per unit mass. "Turbulent kinetic energy" is the kinetic energy of chaotic fluctuations of the fluid flow. The dissipation of kinetic energy arises in ocean models as a result of the viscosity of sea water.
+
+ m2 s-2
+
+
+ Specific means per unit mass. "Turbulent kinetic energy" is the kinetic energy of all eddy-induced motion that is not resolved on the grid scale of the model.
+
+
m2 s-2
@@ -24594,8 +24741,7 @@
kg m-2 s-1
- Methane emitted from the surface, generated by biomass burning (fires). Positive direction upwards.
-The surface called "surface" means the lower boundary of the atmosphere. "Upward" indicates a vector component which is positive when directed upward (negative downward). In accordance with common usage in geophysical disciplines, "flux" implies per unit area, called "flux density" in physics. The chemical formula for methane is CH4. The mass is the total mass of the molecules. The specification of a physical process by the phrase "due_to_" process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. "Emission" means emission from a primary source located anywhere within the atmosphere, including at the lower boundary (i.e. the surface of the earth). "Emission" is a process entirely distinct from "re-emission" which is used in some standard names. The term "fires" means all biomass fires, whether naturally occurring or ignited by humans. The precise conditions under which fires produce and consume methane can vary between models.
+ Methane emitted from the surface, generated by biomass burning (fires). Positive direction upwards. The surface called "surface" means the lower boundary of the atmosphere. "Upward" indicates a vector component which is positive when directed upward (negative downward). In accordance with common usage in geophysical disciplines, "flux" implies per unit area, called "flux density" in physics. The chemical formula for methane is CH4. The mass is the total mass of the molecules. The specification of a physical process by the phrase "due_to_" process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. "Emission" means emission from a primary source located anywhere within the atmosphere, including at the lower boundary (i.e. the surface of the earth). "Emission" is a process entirely distinct from "re-emission" which is used in some standard names. The term "fires" means all biomass fires, whether naturally occurring or ignited by humans. The precise conditions under which fires produce and consume methane can vary between models.
@@ -30335,7 +30481,7 @@ The surface called "surface" means the lower boundary of the atmospher
W m-2
- The phrase "tendency_of_X" means derivative of X with respect to time. The phrase "expressed_as_heat_content" means that this quantity is calculated as the specific heat capacity times density of sea water multiplied by the conservative temperature of the sea water in the grid cell and integrated over depth. If used for a layer heat content, coordinate bounds should be used to define the extent of the layers. If no coordinate bounds are specified, it is assumed that the integral is calculated over the entire vertical extent of the medium, e.g, if the medium is sea water the integral is assumed to be calculated over the full depth of the ocean. Conservative Temperature is defined as part of the Thermodynamic Equation of Seawater 2010 (TEOS-10) which was adopted in 2010 by the International Oceanographic Commission (IOC). Conservative Temperature is specific potential enthalpy (which has the standard name sea_water_specific_potential_enthalpy) divided by a fixed value of the specific heat capacity of sea water, namely cp_0 = 3991.86795711963 J kg-1 K-1. Conservative Temperature is a more accurate measure of the "heat content" of sea water, by a factor of one hundred, than is potential temperature. Because of this, it can be regarded as being proportional to the heat content of sea water per unit mass. Reference: www.teos-10.org; McDougall, 2003 doi: 10.1175/1520-0485(2003)033<0945:PEACOV>2.0.CO;2. The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection. Additionally, when the parameterized advective process is represented in the model as a skew-diffusion rather than an advection, then the parameterized skew diffusion should be included in this diagnostic. The convergence of a skew-flux is identical (in the continuous formulation) to the convergence of an advective flux, making their tendencies the same.
+ The phrase "tendency_of_X" means derivative of X with respect to time. The phrase "expressed_as_heat_content" means that this quantity is calculated as the specific heat capacity times density of sea water multiplied by the conservative temperature of the sea water in the grid cell and integrated over depth. If used for a layer heat content, coordinate bounds should be used to define the extent of the layers. If no coordinate bounds are specified, it is assumed that the integral is calculated over the entire vertical extent of the medium, e.g, if the medium is sea water the integral is assumed to be calculated over the full depth of the ocean. Conservative Temperature is defined as part of the Thermodynamic Equation of Seawater 2010 (TEOS-10) which was adopted in 2010 by the International Oceanographic Commission (IOC). Conservative Temperature is specific potential enthalpy (which has the standard name sea_water_specific_potential_enthalpy) divided by a fixed value of the specific heat capacity of sea water, namely cp_0 = 3991.86795711963 J kg-1 K-1. Conservative Temperature is a more accurate measure of the "heat content" of sea water, by a factor of one hundred, than is potential temperature. Because of this, it can be regarded as being proportional to the heat content of sea water per unit mass. Reference: www.teos-10.org; McDougall, 2003 doi: 10.1175/1520-0485(2003)033<0945:PEACOV>2.0.CO;2. The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection. Additionally, when the parameterized advective process is represented in the model as a skew-diffusion rather than an advection, then the parameterized skew diffusion should be included in this diagnostic. The convergence of a skew-flux is identical (in the continuous formulation) to the convergence of an advective flux, making their tendencies the same.
@@ -30384,7 +30530,7 @@ The surface called "surface" means the lower boundary of the atmospher
W m-2
- The phrase "tendency_of_X" means derivative of X with respect to time. The phrase "expressed_as_heat_content" means that this quantity is calculated as the specific heat capacity times density of sea water multiplied by the potential temperature of the sea water in the grid cell and integrated over depth. If used for a layer heat content, coordinate bounds should be used to define the extent of the layers. If no coordinate bounds are specified, it is assumed that the integral is calculated over the entire vertical extent of the medium, e.g, if the medium is sea water the integral is assumed to be calculated over the full depth of the ocean. Potential temperature is the temperature a parcel of air or sea water would have if moved adiabatically to sea level pressure. The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection. Additionally, when the parameterized advective process is represented in the model as a skew-diffusion rather than an advection, then the parameterized skew diffusion should be included in this diagnostic. The convergence of a skew-flux is identical (in the continuous formulation) to the convergence of an advective flux, making their tendencies the same.
+ The phrase "tendency_of_X" means derivative of X with respect to time. The phrase "expressed_as_heat_content" means that this quantity is calculated as the specific heat capacity times density of sea water multiplied by the potential temperature of the sea water in the grid cell and integrated over depth. If used for a layer heat content, coordinate bounds should be used to define the extent of the layers. If no coordinate bounds are specified, it is assumed that the integral is calculated over the entire vertical extent of the medium, e.g, if the medium is sea water the integral is assumed to be calculated over the full depth of the ocean. Potential temperature is the temperature a parcel of air or sea water would have if moved adiabatically to sea level pressure. The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection. Additionally, when the parameterized advective process is represented in the model as a skew-diffusion rather than an advection, then the parameterized skew diffusion should be included in this diagnostic. The convergence of a skew-flux is identical (in the continuous formulation) to the convergence of an advective flux, making their tendencies the same.
@@ -30475,7 +30621,7 @@ The surface called "surface" means the lower boundary of the atmospher
kg m-2 s-1
- The phrase "tendency_of_X" means derivative of X with respect to time. "Content" indicates a quantity per unit area. The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection. Additionally, when the parameterized advective process is represented in the model as a skew-diffusion rather than an advection, then the parameterized skew diffusion should be included in this diagnostic. The convergence of a skew-flux is identical (in the continuous formulation) to the convergence of an advective flux, making their tendencies the same.
+ The phrase "tendency_of_X" means derivative of X with respect to time. "Content" indicates a quantity per unit area. The specification of a physical process by the phrase due_to_process means that the quantity named is a single term in a sum of terms which together compose the general quantity named by omitting the phrase. Parameterized eddy advection in an ocean model means the part due to a scheme representing parameterized eddy-induced advective effects not included in the resolved model velocity field. Parameterized submesoscale eddy advection occurs on a spatial scale of the order of 1 km horizontally. Reference: James C. McWilliams 2016, Submesoscale currents in the ocean, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, volume 472, issue 2189. DOI: 10.1098/rspa.2016.0117. There are also standard names for parameterized_mesoscale_eddy_advection which, along with parameterized_submesoscale_eddy_advection, contributes to the total parameterized eddy advection. Additionally, when the parameterized advective process is represented in the model as a skew-diffusion rather than an advection, then the parameterized skew diffusion should be included in this diagnostic. The convergence of a skew-flux is identical (in the continuous formulation) to the convergence of an advective flux, making their tendencies the same.
@@ -31388,7 +31534,7 @@ The surface called "surface" means the lower boundary of the atmospher
The "Ultraviolet Index" (UVI) is a measure of the amount of solar ultraviolet radiation that reaches the surface of the earth depending on factors such as time of day and cloud cover. It is often used to alert the public of the need to limit sun exposure and use sun creams to protect the skin. Each point on the Index scale is equivalent to 25 mW m-2 of UV radiation (reference: Australian Bureau of Meteorology, http://www.bom.gov.au/uv/about_uv_index.shtml). The UVI range is expressed as a numeric value from 0 to 20 and sometimes graphically as bands of color indicating the attendant risk of skin damage. A UVI of 0-2 is described as 'Low' (represented graphically in green); a UVI of 11 or greater is described as "Extreme" (represented graphically in purple). The higher the UVI, the greater the potential health risk to humans and the less time it takes for harm to occur. A phrase "assuming_condition" indicates that the named quantity is the value which would obtain if all aspects of the system were unaltered except for the assumption of the circumstances specified by the condition. "Overcast" means a fractional sky cover of 95% or more when at least a portion of this amount is attributable to clouds or obscuring phenomena (such as haze, dust, smoke, fog, etc.) aloft. (Reference: AMS Glossary: http://glossary.ametsoc.org/wiki/Main_Page). Standard names are also defined for the quantities ultraviolet_index and ultraviolet_index_assuming_clear_sky.
-
+
degree_C
@@ -31724,13 +31870,6 @@ The surface called "surface" means the lower boundary of the atmospher
The vertical_component_of_ocean_xy_tracer_diffusivity means the vertical component of the diffusivity of tracers in the ocean due to lateral mixing. This quantity could appear in formulations of lateral diffusivity in which "lateral" does not mean "iso-level", e.g. it would not be used for isopycnal diffusivity. "Tracer diffusivity" means the diffusivity of heat and salinity due to motion which is not resolved on the grid scale of the model.
-
- kg m-2
-
-
- “Drainage” is the process of removal of excess water from soil by gravitational flow. "Amount" means mass per unit area. The vertical drainage amount in soil is the amount of water that drains through the bottom of a soil column extending from the surface to a specified depth.
-
-
m
@@ -32298,6 +32437,20 @@ The surface called "surface" means the lower boundary of the atmospher
Speed is the magnitude of velocity. Wind is defined as a two-dimensional (horizontal) air velocity vector, with no vertical component. (Vertical motion in the atmosphere has the standard name upward_air_velocity.) The wind speed is the magnitude of the wind velocity. A gust is a sudden brief period of high wind speed. In an observed timeseries of wind speed, the gust wind speed can be indicated by a cell_methods of maximum for the time-interval. In an atmospheric model which has a parametrised calculation of gustiness, the gust wind speed may be separately diagnosed from the wind speed.
+
+ m s-1
+
+
+ Speed is the magnitude of velocity. Wind is defined as a two-dimensional (horizontal) air velocity vector, with no vertical component. (Vertical motion in the atmosphere has the standard name upward_air_velocity.) The wind speed is the magnitude of the wind velocity. A gust is a sudden brief period of high wind speed. In an observed timeseries of wind speed, the gust wind speed can be indicated by a cell_methods of maximum for the time-interval. In an atmospheric model which has a parametrised calculation of gustiness, the gust wind speed may be separately diagnosed from the wind speed. The specification of a physical process by the phrase "due_to" process means that the quantity named is a single term in a list of terms, the maximum of which composes the general quantity named by omitting the phrase.
+
+
+
+ m s-1
+
+
+ Speed is the magnitude of velocity. Wind is defined as a two-dimensional (horizontal) air velocity vector, with no vertical component. (Vertical motion in the atmosphere has the standard name upward_air_velocity.) The wind speed is the magnitude of the wind velocity. A gust is a sudden brief period of high wind speed. In an observed timeseries of wind speed, the gust wind speed can be indicated by a cell_methods of maximum for the time-interval. In an atmospheric model which has a parametrised calculation of gustiness, the gust wind speed may be separately diagnosed from the wind speed. The specification of a physical process by the phrase "due_to" process means that the quantity named is a single term in a list of terms, the maximum of which composes the general quantity named by omitting the phrase.
+
+
s-1
N136
@@ -32390,2248 +32543,2256 @@ The surface called "surface" means the lower boundary of the atmospher
-
- moles_of_particulate_inorganic_carbon_per_unit_mass_in_sea_water
+
+ isotropic_longwave_radiance_in_air
-
- temperature_in_ground
+
+ isotropic_shortwave_radiance_in_air
-
- biological_taxon_lsid
+
+ water_evapotranspiration_flux
-
- tendency_of_atmosphere_number_content_of_aerosol_particles_due_to_turbulent_deposition
+
+ drainage_amount_through_base_of_soil_model
-
- lagrangian_tendency_of_atmosphere_sigma_coordinate
+
+ mole_fraction_of_ozone_in_air
-
- lagrangian_tendency_of_atmosphere_sigma_coordinate
+
+ product_of_northward_wind_and_specific_humidity
-
- electrical_mobility_diameter_of_ambient_aerosol_particles
+
+ radiation_wavelength
-
- diameter_of_ambient_aerosol_particles
+
+ specific_gravitational_potential_energy
-
- mass_concentration_of_biomass_burning_dry_aerosol_particles_in_air
+
+ surface_drag_coefficient_for_heat_in_air
-
- effective_radius_of_stratiform_cloud_rain_particles
+
+ surface_drag_coefficient_for_momentum_in_air
-
- effective_radius_of_stratiform_cloud_ice_particles
+
+ surface_drag_coefficient_in_air
-
- effective_radius_of_stratiform_cloud_graupel_particles
+
+ water_flux_into_sea_water
-
- effective_radius_of_convective_cloud_snow_particles
+
+ wind_mixing_energy_flux_into_sea_water
-
- effective_radius_of_convective_cloud_rain_particles
+
+ mole_fraction_of_chlorine_dioxide_in_air
-
- effective_radius_of_convective_cloud_ice_particles
+
+ mole_fraction_of_chlorine_monoxide_in_air
-
- histogram_of_backscattering_ratio_in_air_over_height_above_reference_ellipsoid
+
+ mole_fraction_of_hypochlorous_acid_in_air
-
- backscattering_ratio_in_air
+
+ surface_net_downward_radiative_flux
-
- soot_content_of_surface_snow
+
+ surface_temperature
-
- liquid_water_content_of_surface_snow
+
+ surface_temperature
-
- surface_snow_thickness
+
+ surface_temperature
-
- thermal_energy_content_of_surface_snow
+
+ surface_upward_sensible_heat_flux
-
- temperature_in_surface_snow
+
+ mass_concentration_of_suspended_matter_in_sea_water
-
- integral_wrt_time_of_surface_downward_eastward_stress
+
+ universal_thermal_comfort_index
-
- integral_wrt_time_of_surface_downward_northward_stress
+
+ sea_surface_swell_wave_period
-
- sea_water_velocity_from_direction
+
+ sea_surface_wind_wave_period
-
- sea_water_velocity_to_direction
+
+ atmosphere_net_upward_convective_mass_flux
-
- sea_water_velocity_to_direction
+
+ tendency_of_mass_content_of_water_vapor_in_atmosphere_layer_due_to_deep_convection
-
- integral_wrt_depth_of_product_of_salinity_and_sea_water_density
+
+ tendency_of_mass_content_of_water_vapor_in_atmosphere_layer_due_to_shallow_convection
-
- integral_wrt_depth_of_product_of_conservative_temperature_and_sea_water_density
+
+ tendency_of_mass_content_of_water_vapor_in_atmosphere_layer_due_to_turbulence
-
- integral_wrt_depth_of_product_of_potential_temperature_and_sea_water_density
+
+ wave_frequency
-
- volume_fraction_of_condensed_water_in_soil_at_wilting_point
+
+ northward_eliassen_palm_flux_in_air
-
- volume_fraction_of_condensed_water_in_soil_at_field_capacity
+
+ northward_heat_flux_in_air_due_to_eddy_advection
-
- volume_fraction_of_condensed_water_in_soil_at_critical_point
+
+ upward_eliassen_palm_flux_in_air
-
- volume_fraction_of_condensed_water_in_soil
+
+ upward_eastward_momentum_flux_in_air_due_to_nonorographic_eastward_gravity_waves
-
- product_of_lagrangian_tendency_of_air_pressure_and_specific_humidity
+
+ upward_eastward_momentum_flux_in_air_due_to_nonorographic_westward_gravity_waves
-
- product_of_lagrangian_tendency_of_air_pressure_and_specific_humidity
+
+ upward_eastward_momentum_flux_in_air_due_to_orographic_gravity_waves
-
- product_of_lagrangian_tendency_of_air_pressure_and_geopotential_height
+
+ lwe_thickness_of_atmosphere_mass_content_of_water_vapor
-
- product_of_lagrangian_tendency_of_air_pressure_and_air_temperature
+
+ mass_content_of_cloud_condensed_water_in_atmosphere_layer
-
- product_of_lagrangian_tendency_of_air_pressure_and_air_temperature
+
+ mass_content_of_cloud_ice_in_atmosphere_layer
-
- tendency_of_sea_water_salinity_expressed_as_salt_content_due_to_parameterized_dianeutral_mixing
+
+ mass_content_of_water_in_atmosphere_layer
-
- tendency_of_sea_water_potential_temperature_expressed_as_heat_content_due_to_parameterized_dianeutral_mixing
+
+ mass_content_of_water_vapor_in_atmosphere_layer
-
- tendency_of_sea_water_conservative_temperature_expressed_as_heat_content_due_to_parameterized_dianeutral_mixing
+
+ tendency_of_atmosphere_mass_content_of_water_due_to_advection
-
- rate_of_hydroxyl_radical_destruction_due_to_reaction_with_nmvoc
+
+ tendency_of_atmosphere_mass_content_of_water_vapor
-
- net_primary_mole_productivity_of_biomass_expressed_as_carbon_by_miscellaneous_phytoplankton
+
+ tendency_of_atmosphere_mass_content_of_water_vapor_due_to_convection
-
- mole_fraction_of_inorganic_bromine_in_air
+
+ tendency_of_atmosphere_mass_content_of_water_vapor_due_to_deep_convection
-
- water_vapor_saturation_deficit_in_air
+
+ tendency_of_atmosphere_mass_content_of_water_vapor_due_to_shallow_convection
-
- tendency_of_atmosphere_mass_content_of_elemental_carbon_dry_aerosol_particles_due_to_emission_from_agricultural_waste_burning
+
+ tendency_of_atmosphere_mass_content_of_water_vapor_due_to_turbulence
-
- tendency_of_atmosphere_moles_of_carbon_tetrachloride
+
+ tendency_of_mass_content_of_water_vapor_in_atmosphere_layer
-
- tendency_of_atmosphere_moles_of_carbon_monoxide
+
+ tendency_of_mass_content_of_water_vapor_in_atmosphere_layer_due_to_convection
-
- platform_yaw
+
+ tendency_of_middle_atmosphere_moles_of_carbon_monoxide
-
- platform_pitch
+
+ tendency_of_middle_atmosphere_moles_of_methane
-
- platform_roll
+
+ tendency_of_middle_atmosphere_moles_of_methyl_bromide
-
- net_primary_mole_productivity_of_biomass_expressed_as_carbon_due_to_nitrate_utilization
+
+ tendency_of_middle_atmosphere_moles_of_methyl_chloride
-
- net_primary_mole_productivity_of_biomass_expressed_as_carbon_by_picophytoplankton
+
+ tendency_of_middle_atmosphere_moles_of_molecular_hydrogen
-
- net_primary_mole_productivity_of_biomass_expressed_as_carbon_by_phytoplankton
+
+ tendency_of_troposphere_moles_of_carbon_monoxide
-
- net_primary_mole_productivity_of_biomass_expressed_as_carbon_by_diatoms
+
+ tendency_of_troposphere_moles_of_methane
-
- net_primary_mole_productivity_of_biomass_expressed_as_carbon_by_calcareous_phytoplankton
+
+ tendency_of_troposphere_moles_of_methyl_bromide
-
- mole_concentration_of_diatoms_expressed_as_nitrogen_in_sea_water
+
+ tendency_of_troposphere_moles_of_methyl_chloride
-
- tendency_of_mole_concentration_of_dissolved_inorganic_silicon_in_sea_water_due_to_biological_processes
+
+ tendency_of_troposphere_moles_of_molecular_hydrogen
-
- tendency_of_mole_concentration_of_dissolved_inorganic_phosphorus_in_sea_water_due_to_biological_processes
+
+ mass_fraction_of_convective_cloud_condensed_water_in_air
-
- tendency_of_atmosphere_mole_concentration_of_carbon_monoxide_due_to_chemical_destruction
+
+ mass_fraction_of_ozone_in_air
-
- volume_extinction_coefficient_in_air_due_to_ambient_aerosol_particles
+
+ tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_due_to_emission
-
- mole_fraction_of_noy_expressed_as_nitrogen_in_air
+
+ tendency_of_atmosphere_mass_content_of_dust_dry_aerosol_particles_due_to_emission
-
- tendency_of_atmosphere_moles_of_methane
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_residential_and_commercial_combustion
-
- tendency_of_specific_humidity_due_to_stratiform_precipitation
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_waste_treatment_and_disposal
-
- tendency_of_air_temperature_due_to_stratiform_precipitation
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_savanna_and_grassland_fires
-
- stratiform_precipitation_flux
+
+ tendency_of_atmosphere_mass_content_of_primary_particulate_organic_matter_dry_aerosol_particles_due_to_emission
-
- stratiform_precipitation_amount
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_land_transport
-
- lwe_thickness_of_stratiform_precipitation_amount
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_agricultural_waste_burning
-
- lwe_stratiform_precipitation_rate
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_energy_production_and_distribution
-
- water_evaporation_amount_from_canopy
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_maritime_transport
-
- water_evaporation_flux_from_canopy
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_due_to_net_chemical_production_and_emission
-
- precipitation_flux_onto_canopy
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_due_to_net_chemical_production_and_emission
-
- outgoing_water_volume_transport_along_river_channel
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_forest_fires
-
- tendency_of_sea_ice_amount_due_to_conversion_of_snow_to_sea_ice
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_expressed_as_carbon_due_to_emission_from_industrial_processes_and_combustion
-
- tendency_of_atmosphere_mass_content_of_mercury_dry_aerosol_particles_due_to_emission
+
+ sea_surface_swell_wave_significant_height
-
- mass_fraction_of_mercury_dry_aerosol_particles_in_air
+
+ sea_surface_wind_wave_significant_height
-
- tendency_of_atmosphere_mass_content_of_water_vapor_due_to_sublimation_of_surface_snow_and_ice
+
+ sea_surface_wave_significant_height
-
- surface_snow_density
+
+ mass_content_of_water_in_soil_layer
-
- atmosphere_upward_relative_vorticity
+
+ mass_content_of_water_in_soil
-
- atmosphere_upward_absolute_vorticity
+
+ sea_surface_swell_wave_to_direction
-
- area_type
+
+ tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur_due_to_gravitational_settling
-
- area_type
+
+ tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur_due_to_gravitational_settling
-
- mass_fraction_of_liquid_precipitation_in_air
+
+ tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur_due_to_turbulent_deposition
-
- mass_fraction_of_liquid_precipitation_in_air
+
+ tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur_due_to_turbulent_deposition
-
- tendency_of_mole_concentration_of_particulate_organic_matter_expressed_as_carbon_in_sea_water_due_to_net_primary_production_by_diazotrophic_phytoplankton
+
+ tendency_of_atmosphere_moles_of_nitric_acid_trihydrate_ambient_aerosol_particles
-
- nitrogen_growth_limitation_of_diazotrophic_phytoplankton
+
+ eastward_water_vapor_flux_in_air
-
- net_primary_mole_productivity_of_biomass_expressed_as_carbon_by_diazotrophic_phytoplankton
+
+ kinetic_energy_dissipation_in_atmosphere_boundary_layer
-
- net_primary_mole_productivity_of_biomass_expressed_as_carbon_by_diazotrophic_phytoplankton
+
+ lwe_stratiform_snowfall_rate
-
- mole_concentration_of_diazotrophic_phytoplankton_expressed_as_carbon_in_sea_water
+
+ lwe_thickness_of_stratiform_snowfall_amount
-
- mass_concentration_of_diazotrophic_phytoplankton_expressed_as_chlorophyll_in_sea_water
+
+ northward_water_vapor_flux_in_air
-
- iron_growth_limitation_of_diazotrophic_phytoplankton
+
+ stratiform_rainfall_amount
-
- growth_limitation_of_diazotrophic_phytoplankton_due_to_solar_irradiance
+
+ stratiform_rainfall_flux
-
- air_pseudo_equivalent_potential_temperature
+
+ stratiform_rainfall_rate
-
- tendency_of_mass_fraction_of_stratiform_cloud_ice_in_air_due_to_melting_to_cloud_liquid_water
+
+ stratiform_snowfall_amount
-
- tendency_of_mass_fraction_of_stratiform_cloud_ice_in_air_due_to_heterogeneous_nucleation_from_cloud_liquid_water
+
+ stratiform_snowfall_flux
-
- tendency_of_mass_fraction_of_stratiform_cloud_ice_in_air_due_to_riming_from_cloud_liquid_water
+
+ thickness_of_stratiform_rainfall_amount
-
- tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur_due_to_wet_deposition
-
+
+ thickness_of_stratiform_snowfall_amount
+
-
- tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur_due_to_wet_deposition
+
+ atmosphere_mass_content_of_cloud_condensed_water
-
- stratiform_cloud_area_fraction
+
+ atmosphere_mass_content_of_cloud_ice
-
- surface_upwelling_radiance_per_unit_wavelength_in_air_reflected_by_sea_water
+
+ atmosphere_mass_content_of_convective_cloud_condensed_water
-
- surface_upwelling_radiance_per_unit_wavelength_in_air_emerging_from_sea_water
+
+ atmosphere_mass_content_of_water_vapor
-
- surface_upwelling_radiance_per_unit_wavelength_in_air
+
+ surface_downward_mole_flux_of_carbon_dioxide
-
- surface_upwelling_longwave_flux_in_air
+
+ surface_upward_mole_flux_of_carbon_dioxide
-
- incoming_water_volume_transport_along_river_channel
+
+ atmosphere_mass_content_of_sulfate
-
- sea_water_potential_temperature_expressed_as_heat_content
+
+ atmosphere_mass_content_of_sulfate
-
- sea_water_potential_temperature_expressed_as_heat_content
+
+ change_over_time_in_atmosphere_mass_content_of_water_due_to_advection
-
- sea_ice_temperature_expressed_as_heat_content
+
+ change_over_time_in_atmosphere_mass_content_of_water_due_to_advection
-
- sea_ice_temperature_expressed_as_heat_content
+
+ atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles
-
- surface_downwelling_spherical_irradiance_per_unit_wavelength_in_sea_water
+
+ atmosphere_mass_content_of_primary_particulate_organic_matter_dry_aerosol_particles
-
- surface_downwelling_shortwave_flux_in_air_assuming_clear_sky_and_no_aerosol
+
+ mass_fraction_of_particulate_organic_matter_dry_aerosol_particles_in_air
-
- surface_downwelling_shortwave_flux_in_air_assuming_clear_sky
+
+ mass_fraction_of_primary_particulate_organic_matter_dry_aerosol_particles_in_air
-
- surface_downwelling_shortwave_flux_in_air
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_due_to_dry_deposition
-
- surface_downwelling_radiative_flux_per_unit_wavelength_in_sea_water
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_due_to_gravitational_settling
-
- surface_downwelling_radiative_flux_per_unit_wavelength_in_air
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_due_to_turbulent_deposition
-
- surface_downwelling_radiance_per_unit_wavelength_in_sea_water
+
+ tendency_of_atmosphere_mass_content_of_particulate_organic_matter_dry_aerosol_particles_due_to_wet_deposition
-
- surface_downwelling_photon_spherical_irradiance_per_unit_wavelength_in_sea_water
+
+ tendency_of_atmosphere_mass_content_of_primary_particulate_organic_matter_dry_aerosol_particles_due_to_dry_deposition
-
- surface_downwelling_photon_radiance_per_unit_wavelength_in_sea_water
+
+ tendency_of_atmosphere_mass_content_of_primary_particulate_organic_matter_dry_aerosol_particles_due_to_wet_deposition
-
- surface_downwelling_photon_flux_per_unit_wavelength_in_sea_water
+
+ atmosphere_absorption_optical_thickness_due_to_ambient_aerosol_particles
-
- surface_downwelling_longwave_flux_in_air
+
+ angstrom_exponent_of_ambient_aerosol_in_air
-
- integral_wrt_time_of_surface_downwelling_shortwave_flux_in_air
+
+ atmosphere_absorption_optical_thickness_due_to_dust_ambient_aerosol_particles
-
- integral_wrt_time_of_surface_downwelling_longwave_flux_in_air
+
+ atmosphere_absorption_optical_thickness_due_to_particulate_organic_matter_ambient_aerosol_particles
-
- downwelling_spherical_irradiance_per_unit_wavelength_in_sea_water
+
+ atmosphere_absorption_optical_thickness_due_to_sulfate_ambient_aerosol_particles
-
- downwelling_shortwave_flux_in_air_assuming_clear_sky_and_no_aerosol
+
+ atmosphere_mass_content_of_ammonium_dry_aerosol_particles
-
- downwelling_radiative_flux_per_unit_wavelength_in_sea_water
+
+ atmosphere_mass_content_of_dust_dry_aerosol_particles
-
- downwelling_radiative_flux_per_unit_wavelength_in_air
+
+ atmosphere_mass_content_of_mercury_dry_aerosol_particles
-
- downwelling_radiance_per_unit_wavelength_in_sea_water
+
+ atmosphere_mass_content_of_nitrate_dry_aerosol_particles
-
- downwelling_radiance_per_unit_wavelength_in_air
+
+ atmosphere_mass_content_of_nitric_acid_trihydrate_ambient_aerosol_particles
-
- downwelling_photon_spherical_irradiance_per_unit_wavelength_in_sea_water
+
+ atmosphere_mass_content_of_secondary_particulate_organic_matter_dry_aerosol_particles
-
- downwelling_photon_radiance_per_unit_wavelength_in_sea_water
+
+ atmosphere_mass_content_of_sulfate_ambient_aerosol_particles
-
- downwelling_photon_flux_per_unit_wavelength_in_sea_water
+
+ atmosphere_mass_content_of_sulfate_ambient_aerosol_particles
-
- surface_upwelling_shortwave_flux_in_air_assuming_clear_sky
+
+ atmosphere_mass_content_of_sulfate_dry_aerosol_particles
-
- surface_upwelling_longwave_flux_in_air_assuming_clear_sky
+
+ atmosphere_mass_content_of_water_in_ambient_aerosol_particles
-
- upwelling_shortwave_flux_in_air_assuming_clear_sky_and_no_aerosol
+
+ atmosphere_moles_of_nitric_acid_trihydrate_ambient_aerosol_particles
-
- upwelling_radiative_flux_per_unit_wavelength_in_sea_water
+
+ atmosphere_optical_thickness_due_to_ambient_aerosol_particles
-
- upwelling_radiative_flux_per_unit_wavelength_in_air
+
+ atmosphere_optical_thickness_due_to_ambient_aerosol_particles
-
- upwelling_radiance_per_unit_wavelength_in_air
+
+ atmosphere_optical_thickness_due_to_dust_ambient_aerosol_particles
-
- surface_upwelling_shortwave_flux_in_air_assuming_clear_sky_and_no_aerosol
+
+ atmosphere_optical_thickness_due_to_dust_dry_aerosol_particles
-
- surface_upwelling_shortwave_flux_in_air
+
+ atmosphere_optical_thickness_due_to_particulate_organic_matter_ambient_aerosol_particles
-
- surface_upwelling_radiative_flux_per_unit_wavelength_in_sea_water
+
+ mass_concentration_of_dust_dry_aerosol_particles_in_air
-
- surface_upwelling_radiative_flux_per_unit_wavelength_in_air
+
+ mass_concentration_of_coarse_mode_ambient_aerosol_particles_in_air
-
- surface_upwelling_radiance_per_unit_wavelength_in_sea_water
+
+ mass_concentration_of_ammonium_dry_aerosol_particles_in_air
-
- platform_name
+
+ atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur
-
- water_vapor_partial_pressure_in_air
+
+ atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur
-
- effective_radius_of_stratiform_cloud_snow_particles
+
+ mass_concentration_of_primary_particulate_organic_matter_dry_aerosol_particles_in_air
-
- tendency_of_atmosphere_moles_of_cfc11
+
+ mass_concentration_of_particulate_organic_matter_dry_aerosol_particles_in_air
-
- moles_of_cfc11_per_unit_mass_in_sea_water
+
+ atmosphere_optical_thickness_due_to_water_in_ambient_aerosol_particles
-
- atmosphere_moles_of_cfc11
+
+ mass_concentration_of_mercury_dry_aerosol_particles_in_air
-
- tendency_of_atmosphere_moles_of_cfc113
+
+ mass_concentration_of_nitrate_dry_aerosol_particles_in_air
-
- atmosphere_moles_of_cfc113
+
+ mass_concentration_of_nitric_acid_trihydrate_ambient_aerosol_particles_in_air
-
- tendency_of_atmosphere_moles_of_cfc114
+
+ mass_concentration_of_secondary_particulate_organic_matter_dry_aerosol_particles_in_air
-
- atmosphere_moles_of_cfc114
+
+ mass_concentration_of_sulfate_ambient_aerosol_particles_in_air
-
- tendency_of_atmosphere_moles_of_cfc115
+
+ mass_concentration_of_sulfate_ambient_aerosol_particles_in_air
-
- atmosphere_moles_of_cfc115
+
+ mass_concentration_of_sulfate_dry_aerosol_particles_in_air
-
- tendency_of_atmosphere_moles_of_cfc12
+
+ mass_concentration_of_water_in_ambient_aerosol_particles_in_air
-
- tendency_of_atmosphere_moles_of_halon2402
+
+ mass_fraction_of_ammonium_dry_aerosol_particles_in_air
-
- atmosphere_moles_of_halon2402
+
+ mass_fraction_of_dust_dry_aerosol_particles_in_air
-
- tendency_of_atmosphere_moles_of_hcc140a
+
+ mass_fraction_of_nitrate_dry_aerosol_particles_in_air
-
- atmosphere_moles_of_hcc140a
+
+ mass_fraction_of_nitric_acid_trihydrate_ambient_aerosol_particles_in_air
-
- tendency_of_troposphere_moles_of_hcc140a
+
+ mass_fraction_of_secondary_particulate_organic_matter_dry_aerosol_particles_in_air
-
- tendency_of_middle_atmosphere_moles_of_hcc140a
+
+ mass_fraction_of_sulfate_dry_aerosol_particles_in_air
-
- tendency_of_troposphere_moles_of_hcfc22
+
+ mass_fraction_of_water_in_ambient_aerosol_particles_in_air
-
- tendency_of_atmosphere_moles_of_hcfc22
+
+ mole_concentration_of_nitric_acid_trihydrate_ambient_aerosol_particles_in_air
-
- atmosphere_moles_of_hcfc22
+
+ mole_fraction_of_nitric_acid_trihydrate_ambient_aerosol_particles_in_air
-
- product_of_northward_wind_and_lagrangian_tendency_of_air_pressure
+
+ number_concentration_of_ambient_aerosol_particles_in_air
-
- product_of_eastward_wind_and_lagrangian_tendency_of_air_pressure
+
+ number_concentration_of_coarse_mode_ambient_aerosol_particles_in_air
-
- carbon_mass_flux_into_litter_and_soil_due_to_anthropogenic_land_use_or_land_cover_change
+
+ number_concentration_of_nucleation_mode_ambient_aerosol_particles_in_air
-
- floating_ice_shelf_area_fraction
+
+ optical_thickness_of_atmosphere_layer_due_to_ambient_aerosol_particles
-
- atmosphere_moles_of_carbon_tetrachloride
+
+ optical_thickness_of_atmosphere_layer_due_to_ambient_aerosol_particles
-
- mole_fraction_of_methylglyoxal_in_air
+
+ tendency_of_atmosphere_mass_content_of_ammonium_dry_aerosol_particles_due_to_dry_deposition
-
- mole_fraction_of_dichlorine_peroxide_in_air
+
+ tendency_of_atmosphere_mass_content_of_ammonium_dry_aerosol_particles_due_to_wet_deposition
-
- volume_scattering_coefficient_of_radiative_flux_in_air_due_to_ambient_aerosol_particles
+
+ tendency_of_atmosphere_mass_content_of_dust_dry_aerosol_particles_due_to_dry_deposition
-
- volume_scattering_coefficient_of_radiative_flux_in_air_due_to_dried_aerosol_particles
+
+ tendency_of_atmosphere_mass_content_of_dust_dry_aerosol_particles_due_to_gravitational_settling
-
- soil_mass_content_of_carbon
+
+ tendency_of_atmosphere_mass_content_of_dust_dry_aerosol_particles_due_to_turbulent_deposition
-
- slow_soil_pool_mass_content_of_carbon
+
+ tendency_of_atmosphere_mass_content_of_dust_dry_aerosol_particles_due_to_wet_deposition
-
- root_mass_content_of_carbon
+
+ tendency_of_atmosphere_mass_content_of_mercury_dry_aerosol_particles_due_to_dry_deposition
-
- miscellaneous_living_matter_mass_content_of_carbon
+
+ tendency_of_atmosphere_mass_content_of_mercury_dry_aerosol_particles_due_to_wet_deposition
-
- fast_soil_pool_mass_content_of_carbon
+
+ tendency_of_atmosphere_mass_content_of_nitrate_dry_aerosol_particles_due_to_dry_deposition
-
- medium_soil_pool_mass_content_of_carbon
+
+ tendency_of_atmosphere_mass_content_of_secondary_particulate_organic_matter_dry_aerosol_particles_due_to_dry_deposition
-
- leaf_mass_content_of_carbon
+
+ tendency_of_atmosphere_mass_content_of_secondary_particulate_organic_matter_dry_aerosol_particles_due_to_net_chemical_production
-
- carbon_mass_content_of_forestry_and_agricultural_products
+
+ tendency_of_atmosphere_mass_content_of_secondary_particulate_organic_matter_dry_aerosol_particles_due_to_net_chemical_production
-
- carbon_mass_content_of_forestry_and_agricultural_products
+
+ tendency_of_atmosphere_mass_content_of_secondary_particulate_organic_matter_dry_aerosol_particles_due_to_wet_deposition
-
- surface_upward_mass_flux_of_carbon_dioxide_expressed_as_carbon_due_to_plant_respiration_for_biomass_maintenance
+
+ tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_due_to_dry_deposition
-
- surface_upward_mass_flux_of_carbon_dioxide_expressed_as_carbon_due_to_plant_respiration_for_biomass_growth
+
+ tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur_due_to_dry_deposition
-
- surface_upward_mass_flux_of_carbon_dioxide_expressed_as_carbon_due_to_plant_respiration
+
+ tendency_of_atmosphere_mass_content_of_sulfate_dry_aerosol_particles_expressed_as_sulfur_due_to_dry_deposition
-
- surface_upward_mass_flux_of_carbon_dioxide_expressed_as_carbon_due_to_respiration_in_soil
+
+ x_wind
-
- surface_upward_mass_flux_of_carbon_dioxide_expressed_as_carbon_due_to_heterotrophic_respiration
+
+ y_wind
-
- eastward_transformed_eulerian_mean_air_velocity
+
+ land_ice_surface_specific_mass_balance_rate
-
- surface_litter_mass_content_of_carbon
+
+ land_ice_lwe_surface_specific_mass_balance_rate
-
- litter_mass_content_of_carbon
+
+ isotropic_radiance_per_unit_wavelength_in_air
-
- tendency_of_atmosphere_mass_content_of_nitrogen_compounds_expressed_as_nitrogen_due_to_wet_deposition
+
+ isotropic_radiance_per_unit_wavelength_in_air
-
- mole_concentration_of_phytoplankton_expressed_as_nitrogen_in_sea_water
+
+ omnidirectional_spherical_irradiance_per_unit_wavelength_in_sea_water
-
- atmosphere_mass_content_of_convective_cloud_liquid_water
+
+ mass_concentration_of_chlorophyll_in_sea_water
-
- effective_radius_of_cloud_liquid_water_particles_at_liquid_water_cloud_top
+
+ mass_concentration_of_chlorophyll_in_sea_water
-
- air_equivalent_temperature
+
+ tendency_of_atmosphere_moles_of_sulfate_dry_aerosol_particles
-
- air_pseudo_equivalent_temperature
+
+ tendency_of_atmosphere_moles_of_methyl_bromide
-
- mass_content_of_cloud_liquid_water_in_atmosphere_layer
+
+ tendency_of_atmosphere_moles_of_methyl_chloride
-
- air_equivalent_potential_temperature
+
+ tendency_of_atmosphere_moles_of_molecular_hydrogen
-
- number_concentration_of_stratiform_cloud_liquid_water_particles_at_stratiform_liquid_water_cloud_top
+
+ tendency_of_atmosphere_moles_of_nitrous_oxide
-
- number_concentration_of_convective_cloud_liquid_water_particles_at_convective_liquid_water_cloud_top
+
+ tendency_of_atmosphere_mass_content_of_water_vapor_due_to_advection
-
- effective_radius_of_stratiform_cloud_liquid_water_particles_at_stratiform_liquid_water_cloud_top
+
+ atmosphere_moles_of_carbon_monoxide
-
- effective_radius_of_stratiform_cloud_liquid_water_particles
+
+