Add how to add a recipe documentation

[mo-tgeddes]

Part of #2786.
Add instructions on how to add a recipe to the recipe test workflow, to the documentation

[ehogan]

@alistairsellar's notes for this:

Adding a recipe to RTW:

• Check it works offline on Jasmin (which should have complete set of obs and model data)
  • Offline means outside the RTW
  • Run with time and check time & mem at the end (esmvaltool provides memory estimate)
• Get it running offline on Met Office: download obs and model data
• Document memory and time on each platform in
• Stop RTW on Met Office & Jasmin
• Add to RTW on Met Office (update start date to today) & run
• When compare fails:
  • copy output from $HOME/cylc-run/recipe_test_workflow/runN/share/cycle// to KGO directory (see opt/rose-suite-metoffice.conf)
  • then chmod a+w
• Re-trigger compare task
• If it succeeds, commit and push
• Pull on jasmin and run
• When compare fails:
  • copy output to /gws/nopw/j04/esmeval/KGO/
  • then chmod a+w

[mo-gill]

During completion of issue #3608 using Alistair's instructions, i made some modifications to them to start shaping them further into final instructions. Hopefully they're helpful:

1. Run "esmvaltool run <recipename.yml>" (if it raises error about missing data, then data will need to be retrieved so it can be run).

2. Run the same on jasmin.

3. Take note of output of time and memory taken to run for both by checking cylc review .

4. Use "cylc stop" to stop the running rtw workflows.

5. Add the recipe to the RTW flow.cylc file.

6. Add time/memory usage to the metoffice.cylc and jasmin.cylc if they are larger than the defaults, this can be found in "job.time" in cylc review.

7. Run the workflow with cylc vip -O metoffice (it will fail).

8. Locate it's run folder.

9. Copy the output of that recipe run from that cycle (run/share/bin/cycle into KGO rootpath folder (KGO_ROOT_PATH="/data/users/esmval/KGO").

   cp -r <directory of recipe from it's cycle in run directory>

10. Cd to KGO folder.

11. Use "chmod a+w on recipe you've just added in KGO folder.

12. Run RTW again.

13. If the workflow now succeeds, commit and push.