Switch to data time cycling to support multiple models and multiple case studies or trials #765
Commits on Sep 27, 2024
-
Add multiple models to rose edit configuration
Also gone through and removed some unneeded things, added titles, etc. Still need to go through the diagnostics. Split diagnostics into sections, add input variable table Also link in more models, so 10 are now supported. Improvements to rose edit metadata Fix a bunch of validation issues
-
-
Change how data is fetched to work on data time
Also now use subclasses for getting individual files from the various data sources.
-
Modify data fetching in workflow to fetch per model
Grab rose template variables directly, rather than via jinja context Add workflow exported functions to __all__ Add licence header to jinja_utils Require success for processing tasks in workflow Housekeep only on success, but finish workflow regardless
-
Change recipe format to just use steps, and update infra for running …
…them I've updated one recipe for now, but they will all have to be converted.
-
The final cycle point task are skipped if doing a continuous trial without a defined end date.
-
-
They have all been replaced my the more generic version that work for both the UM and LFRic.
-
-
-
-
-
-
Differentiate output directories for different case dates
Use cycle point for deterministic resolving.
-
Remove duplicated include file
It was also invalid due to referencing non-existent variables.
-
-
-
-
Split rose metadata into multiple files
Also rearrange pressure level metadata.
-
-
-
-
-
-
-
-
-
-
Q-Q plot recipe is partially converted, pending Issue #808.
-
-
-
-
-
-
-
-
-
-
Reference UTC timezone as datetime.timezone.utc
The datetime.UTC alias was only added in python 3.11.
-
Add workaround for old python not supporting ISO 8601 datetimes
This should be removed when we drop python 3.10 support.
-
-
-
-
-
Finish website on compute platform
Now it touches the data to generate the index, it probably shouldn't be run on the cylc scheduler.
-
-
-
Remove parallel and collate steps
We've now gone back to just steps.
-
-
-
-
-
-
-
Change datetime examples to use YYYYMMDDThhmmZ format
It is more familiar to users.
-
-
Clarify that CSET_ANALYSIS_OFFSET
Note that if fixes differences between the initiation time and the first validity time.
-
-
Number models from 0 to allow directly using as index
This removes the -1 from all usage in include files.
-
-
-
-
-
-
-
-
-
-
-
Default to False when setting doesn't exist
This is useful if the setting is not in the rose-suite.conf
-
-
-
-
-
-
The output_dir was in the wrong place, so it was making a POST request.
-
-
-
-
-
-
-
-
Bundle recipes and example data into documentation
This prevents needing to rely on external hosts for these. I did need to drop the example commands to fetch the resources via curl, but they were probably confusing anyway.
-
They are not implemented yet, and can be re-added when they are.
-
-
Make area constraint much stricter, and handle date line
The operator now requires all arguments to either be numbers, or all be None if you don't want to constrain the area. The test has been mildly improved, so at least it checks for both of the named functions on the coordinates, though it still doesn't test that they do anything.
-
-
-
Remove extra space from rose-suite.conf.example
It is invalid rose syntax. Also use !! to disable showing the option in the GUI.
-
