Add comparison between FM and structured models (#37)

This PR adds a comparison example between the structured and FM models called `comparison.py`. Detailed changes are as follows:

* Added `examples/comparison.py` example script.
* Added results of the comparison example to the documentation.
* Changed the way reused example simulations are stored to allow sharing between the grid convergence example and the comparison example.
* Added the ability to export & import `CaseStudy` classes to and from YAML files.
* Relaxed equality check for sequences in `CaseStudy` classes (i.e. a tuple can match a list).
* Added the ability to add labels to headings, tables and images in the `Content` class. 
* Bump version to 0.6.0

.conda/recipe/meta.yaml

@@ -1,4 +1,4 @@
-{% set version = "0.5.3" %}
+{% set version = "0.6.0" %}
 
 package:
   name: snl-delft3d-cec-verify

README.md

@@ -32,7 +32,7 @@ From a conda prompt create a named environment in which to install the
 for future updates:
 
 ```
-(base) > conda create -y -n snld3d --override-channels -c conda-forge -c dataonlygreater snl-delft3d-cec-verify=0.5.3
+(base) > conda create -y -n snld3d --override-channels -c conda-forge -c dataonlygreater snl-delft3d-cec-verify=0.6.0
 (base) > conda activate snld3d
 (snld3d) > conda config --env --add channels conda-forge --add channels dataonlygreater
 (snld3d) > conda config --env --set channel_priority strict
@@ -212,10 +212,10 @@ If successful, the report files (and images) will be placed into a
 sub-directory based on the model type. For the flexible mesh model, this is 
 `structured/grid_convergence_report`. To avoid repeating simulations in the 
 event of an unexpected failure or change to the `grid_convergence.py` file, 
-the Delft3D simulations are stored in a sub-directory based on the model type. 
-For the flexible mesh model, this is `structured/grid_convergence_runs`. If 
-Delft3D is updated, ensure to delete or move this folder, so that new 
-simulations are conducted.
+the Delft3D simulations, and a copy of their case study parameters, are stored 
+in a sub-directory based on the model type. For the structured grid model, for 
+example, this is `structured/runs`. If the Delft3D solver is updated, ensure to 
+delete or move this folder, so that new simulations are conducted.
 
 By default, the study is conducted using just one CPU thread. To reduce 
 simulation time of the `fm` model, assuming additional capacity is available, 
@@ -237,6 +237,73 @@ optional argument to reduce the number of experiments. For example:
 Pre-calculated results of the full study are available in the [online 
 documentation][110].
 
+### Model Comparison Study
+
+Required files:
++   `comparison.py`
++   `examples.bib` (for conversion to Word format)
++   `reference.docx` (for conversion to Word format)
+
+The second production example is a comparison of the flexible mesh and 
+structured grid solvers for a turbine simulation using identical settings.
+
+This example uses the [pandoc-crossref][119] package to reference sections
+and figures within the generated report. To install the package (for converting
+to Word format with pypandoc) issue the following command:
+
+```
+(snld3d) > conda install pandoc-crossref=0.3.10.0
+```
+
+For the example to run, two environment variable **must be set**. For path to 
+the flexible mesh solver, set the `D3D_FM_BIN` variable. In PowerShell, for 
+example:
+
+```
+(snld3d) > $env:D3D_FM_BIN = "\path\to\SNL-Delft3D-FM-CEC\src\bin"
+```
+
+For the path to the structured grid solver, set the `D3D_4_BIN` environment
+variable. In PowerShell again:
+
+```
+(snld3d) > $env:D3D_4_BIN = "\path\to\SNL-Delft3D-CEC\src\bin"
+```
+
+Then move to the directory containing `comparison.py` and call the script using 
+Python:
+
+```
+(snld3d) > python comparison.py
+```
+
+If successful, the report files (and images) will be placed into a 
+sub-directory called `comparison_report`. To avoid repeating simulations, the 
+Delft3D simulations, and a copy of their case study parameters, are stored in 
+a sub-directory based on the model type. For the flexible mesh model this is 
+`fm/runs` and for the structured grid model it's `structured/runs`. If 
+either Delft3D solver is updated, ensure to delete or move these folders, so 
+that new simulations are conducted.
+
+By default, the study is conducted using just one CPU thread. To reduce 
+simulation time of the `fm` model, assuming additional capacity is available, 
+increase the number of utilised threads using the `--threads` optional argument:
+
+```
+(snld3d) > python comparison.py --threads 8
+```
+
+Note that this study takes a considerable amount of wall-clock time to 
+complete. To run the simulations at lower resolution (and, therefore, more 
+rapidly), use the `--grid-resolution` optional argument. For example:
+
+```
+(snld3d) > python comparison.py --threads 8 --grid-resolution 0.25
+```
+
+Pre-calculated results of the study at the default resolution of 0.0625m is 
+available in the [online documentation][110].
+
 ## Documentation
 
 API documentation, which describes the classes and functions used in the 
@@ -395,3 +462,4 @@ Retrieved 24 January 2022, from https://www.grc.nasa.gov/www/wind/valid/tutorial
 [116]: https://github.com/SNL-WaterPower/SNL-Delft3D-CEC
 [117]: https://www.deltares.nl/en/software/delft3d-4-suite/
 [118]: https://github.com/Data-Only-Greater/SNL-Delft3D-CEC-Verify/releases/latest
+[119]: https://github.com/lierdakil/pandoc-crossref

docs/_assets/gh-pages-redirect.html

@@ -3,7 +3,7 @@
   <head>
     <title>Redirecting to latest version</title>
     <meta charset="utf-8">
-    <meta http-equiv="refresh" content="0; url=./v0.5.3/index.html">
-    <link rel="canonical" href="https://data-only-greater.github.io/SNL-Delft3D-CEC-Verify/v0.5.3/index.html">
+    <meta http-equiv="refresh" content="0; url=./v0.6.0/index.html">
+    <link rel="canonical" href="https://data-only-greater.github.io/SNL-Delft3D-CEC-Verify/v0.6.0/index.html">
   </head>
 </html>

docs/conf.py

@@ -25,7 +25,7 @@
 author = 'Mathew Topper'
 
 # The full version, including alpha/beta/rc tags
-release = '0.5.3'
+release = '0.6.0'
 
 
 # -- General configuration ---------------------------------------------------
@@ -57,7 +57,7 @@
 smv_remote_whitelist = r'^(origin)$'
 smv_tag_whitelist = r'^v(\d+\.\d+\.\d+)$' # r'^v(?!0.4.0|0.4.1|0.4.2)\d+\.\d+\.\d+$'
 smv_released_pattern = r'^refs/tags/.*$'
-smv_latest_version = 'v0.5.3'
+smv_latest_version = 'v0.6.0'
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']

docs/validation/comparison/report.rst

@@ -0,0 +1,167 @@
+Model Comparison (Windows)
+==========================
+
+1 Introduction
+--------------
+
+This is a comparison of the performance of simulations of the Mycek
+flume experiment (Mycek et al. 2014) using the flexible mesh (FM) and
+structured grid solvers for Delft3D. The simulation settings are
+mirrored between the two methods as much as possible. The chosen grid
+resolution for this study is 0.0625m. Axial and radial velocities in the
+horizontal plane intersecting the turbine hub will be examined.
+
+.. _sec:axial:
+
+2 Axial Velocity Comparison
+---------------------------
+
+This section compares axial velocities between the FM and structured
+grid models. Figs. 1, 2 show the axial velocity over the horizontal
+plane intersecting the turbine hub for the FM and structured gird
+models, respectively. The units are non-dimensionalised by the
+free-stream velocity, measured at the hub location without the presence
+of the turbine. If :math:`u` is the dimensional velocity, and
+:math:`u_\infty` is the dimensional free stream velocity, then the
+normalized velocity :math:`u^* = u / u_\infty`. Note the observable
+difference in the wake velocities immediately downstream of the turbine
+between the two simulations.
+
+.. figure:: turb_z_u_fm.png
+   :alt: Figure 1: Axial velocity normalised by the free stream velocity
+         for the fm model type
+   :name: fig:turb_z_u_fm
+   :width: 3.64in
+
+   Figure 1: Axial velocity normalised by the free stream velocity for
+   the fm model type
+
+.. figure:: turb_z_u_structured.png
+   :alt: Figure 2: Axial velocity normalised by the free stream velocity
+         for the structured model type
+   :name: fig:turb_z_u_structured
+   :width: 3.64in
+
+   Figure 2: Axial velocity normalised by the free stream velocity for
+   the structured model type
+
+Fig. 3 shows the error between the non-dimensional axial velocities of
+the structured grid and FM models, relative to the maximum value within
+the two simulations. Three main areas of difference are revealed, the
+increased deficit in the near wake for the structured model, the reduced
+deficit of the structured model in the far wake and the increased
+acceleration around the edges of the turbine of the structured model.
+
+.. figure:: turb_z_u_diff.png
+   :alt: Figure 3: Relative error in normalised axial velocity between
+         the structured and fm models
+   :name: fig:turb_z_u_diff
+   :width: 3.64in
+
+   Figure 3: Relative error in normalised axial velocity between the
+   structured and fm models
+
+Comparing the non-dimensional centerline velocities alongside the
+experimental data (published in (Mycek et al. 2014)) in fig. 4, confirms
+the behavior in the near and far wake shown in fig. 3. Generally, the
+structured model performs better in the near wake compared to the
+experimental data, however the performance in the far wake is better for
+the FM model, where the wake has decayed less. Nonetheless, neither
+model captures the experimental measurements well for the whole
+centerline.
+
+.. figure:: transect_u.png
+   :alt: Figure 4: Comparison of the normalised turbine centerline
+         velocity. Experimental data reverse engineered from (Mycek et al.
+         2014, fig. 11a).
+   :name: fig:transect_u
+   :width: 4in
+
+   Figure 4: Comparison of the normalised turbine centerline velocity.
+   Experimental data reverse engineered from (Mycek et al. 2014, fig.
+   11a).
+
+.. _sec:radial:
+
+3 Radial Velocity Comparison
+----------------------------
+
+This section compares radial velocities between the FM and structured
+grid models. Figs. 5, 6 show the radial velocity over the horizontal
+plane intersecting the turbine hub for the FM and structured gird
+models, respectively. The units are non-dimensionalized by the
+free-stream velocity, (in the axial direction) measured at the hub
+location without the presence of the turbine. If :math:`v` is the
+dimensional velocity, then the normalized velocity
+:math:`v^* = v / u_\infty`. Note the increased radial velocities
+recorded for the structured grid compared to the FM simulation.
+
+.. figure:: turb_z_v_fm.png
+   :alt: Figure 5: Radial velocity normalised by the free stream
+         velocity for the fm model type
+   :name: fig:turb_z_v_fm
+   :width: 3.64in
+
+   Figure 5: Radial velocity normalised by the free stream velocity for
+   the fm model type
+
+.. figure:: turb_z_v_structured.png
+   :alt: Figure 6: Radial velocity normalised by the free stream
+         velocity for the structured model type
+   :name: fig:turb_z_v_structured
+   :width: 3.64in
+
+   Figure 6: Radial velocity normalised by the free stream velocity for
+   the structured model type
+
+Fig. 7 shows the error between the non-dimensional radial velocities of
+the structured grid and FM models, relative to the maximum value within
+the two simulations. The largest errors are seen upstream of the
+turbine, while smaller errors are seen downstream of the turbine. The
+errors in the radial flow are also much higher than for the axial flow,
+with the maximum error in radial velocity being 0.2425, while the error
+is 0.08593 for the axial velocity (from fig. 3).
+
+.. figure:: turb_z_v_diff.png
+   :alt: Figure 7: Relative error in normalised radial velocity between
+         the structured and fm models
+   :name: fig:turb_z_v_diff
+   :width: 3.64in
+
+   Figure 7: Relative error in normalised radial velocity between the
+   structured and fm models
+
+4 Conclusion
+------------
+
+Comparison of simulations of the 2014 Mycek flume experiment (Mycek et
+al. 2014) using the flexible mesh (FM) and structured grid solvers for
+Delft3D, reveals significant differences. As seen in sec. 2, differences
+in the axial velocities between the two methods were seen in the near
+wake, far wake, and at the turbine edges. When comparing to the
+experimental data, as in fig. 3, it was observed that the structured
+grid simulation performs better in the near wake, while the FM
+simulation is better in the far wake. In sec. 3, radial velocities were
+compared with differences seen immediately upstream and downstream of
+the turbine (see fig. 7). Notably, the maximum relative errors between
+the two simulations were much larger for the radial velocities than then
+axial velocities, 0.2425 and 0.08593 respectively. This discrepancy may
+account for some of the differences seen in the axial flows, although
+the underlying mechanisms are not yet known. Other factors may also be
+contributing, including interpretation of the simulation parameters or
+selection of the time step for the structured grid simulations.
+
+References
+----------
+
+.. container:: references csl-bib-body hanging-indent
+   :name: refs
+
+   .. container:: csl-entry
+      :name: ref-mycek2014
+
+      Mycek, Paul, Benoît Gaurier, Grégory Germain, Grégory Pinon, and
+      Elie Rivoalen. 2014. “Experimental Study of the Turbulence
+      Intensity Effects on Marine Current Turbines Behaviour. Part I:
+      One Single Turbine.” *Renewable Energy* 66: 729–46.
+      https://doi.org/10.1016/j.renene.2013.12.036.

docs/validation/index.rst

@@ -18,3 +18,12 @@ Structured
    :maxdepth: 4
 
    structured/linux/report
+
+
+Comparison
+----------
+
+.. toctree::
+   :maxdepth: 4
+
+   comparison/report