Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add how to add a recipe documentation #3080

Closed
wants to merge 2 commits into from
Closed

Add how to add a recipe documentation #3080

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2de61ab
#3074: add documentation first draft
mo-tgeddes Mar 8, 2023
dc33540
#2074: fix spelling mistake
mo-tgeddes Mar 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions esmvaltool/utils/recipe_test_workflow/doc/source/common.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
.. |ESMValCore| replace:: :term:`ESMValCore`
.. |ESMValTool| replace:: :term:`ESMValTool`
.. |Rose| replace:: :term:`Rose`
.. |KGO| replace:: :term:`KGO`

.. Links

Expand Down
3 changes: 3 additions & 0 deletions esmvaltool/utils/recipe_test_workflow/doc/source/glossary.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ Glossary
routine evaluation of |ESMs| in |CMIP|, see the
`ESMValTool Documentation`_.

KGO
Known good output

Rose
Rose is a toolkit for writing, editing and running application
configurations, see the `Rose Documentation`_.
Expand Down
59 changes: 59 additions & 0 deletions esmvaltool/utils/recipe_test_workflow/doc/source/user_guide/adding_a_recipe.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
.. include:: ../common.txt

Adding a Recipe to the RTW
==========================

Can your recipe be added to the RTW?
------------------------------------
The |RTW| will run a recipe and compare the output to a |KGO|.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wider than just this PR, but use of vertical bars with doc/source/common.txt for RTW documentation looks like a divergence from existing conventions in rest of ESMValTool docs. Are we OK with that?

The compare task will fail if the output doesn not exactly match the |KGO|,
therefore, if the recipe output is not reproducible it cannot be compared to a |KGO|, and the compare task will always fail.

It is recommended that recipes are added to the |RTW| as this will make the process of a release easier,
and should a breaking change be introduced ESMValTool, the author will be notified when the recipe on the |RTW| fails.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Author or maintainer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also a wider policy question, but do we plan for maintainers of recipes to be automatically notified by the RTW? Or would the release manager / recent developers be informed first?


Prerequisites
-------------

To add a recipe to the |RTW|, the first thing that must be done is run the recipe.

Take note of the resources required to run the recipe and the time it takes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the audience for this include developers of new recipes? Or will it more generally be an SSE? Some scientists will not know how to do this, particularly if the recipe is cheap enough to run on VDI. Does it need some tips?


Inspect the results of your recipe and verify they are as expected.

Add the verified output of the recipe to the KGO_ROOT_PATH of each site. These can be found in in the
`recipe_test_workflow/opt/rose-suite-<site>.conf` files. Please ensure the |KGO| for the recipe to be
added is placed in the root path for every site.

If for any reason you are unable to access the KGO_ROOT_PATH directory, please contact the maintainer of the |RTW|.

Add your recipe to the workflow
-------------------------------
* Checkout the |RTW|::

git clone [email protected]:ESMValGroup/ESMValTool.git

* Make a branch of the |RTW|::

git checkout recipe_test_workflow_prototype
git checkout -b add_recipe_branch

* Every recipe in the |RTW| is sorted into one of three groups; fast, medium, and slow, depending on what resources
they require and how long they take to run.

* fast - Anything under 2 minutes to run and under 2G memory required.
* medium - Anything between 2 and 6 minutes to run and under 2G memory required.
* slow - Anything above this.


* Open the flow.cylc file and edit the task parameters of the group the added recipe belongs to::
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, code design question rather than a documentation comment... these lists could get very long. It would feel more natural to me to group parameters by task, like

[radiation_budget]
    resource_group = fast

But I've not got visibility of the whole workflow design, so if you have already discussed this and rejected it then I'm happy to trust your collective judgement.


[task parameters]
fast = radiation_budget, <your_recipe_name>
medium = ensclus, heatwaves_coldwaves

The name of the task parameter must match the file name of the recipe to be added
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The name of the task parameter must match the file name of the recipe to be added
The name of the task parameter must match the file name of the recipe to be added, minus the `.yml` extension.


* Run the branch of |RTW| on all sites and ensure the added recipe passes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, does the intended audience know how to do this. Which are "all sites" for now, and does any user have access to the all? Once Levante (DKRZ) and NCI (Australia) included in RTW sites, there will be no user who has access to them all (though I think that GitHub esmvalbot runs the recipe on Levante when asked).


* Make a pull request to merge the branch with the added recipe with `recipe_test_workflow_prototype`
1 change: 1 addition & 0 deletions esmvaltool/utils/recipe_test_workflow/doc/source/user_guide/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ User Guide

quick_start
workflow
adding_a_recipe