ODAS updates to allow runing reanalysis on Milan #6
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes the following issues:
Pagepool memory limitation in SLES17 to read observation files.
Users without priority stuck in queue with observer jobs.
Model BOMBING when ADT sigo variance is below a threshold. This is expected, but the user needs to either correct altimeter observations or remove the problematic altimeter for the entire cycle.
Hardcoded path dependencies for utilities like ncap2 and mpi4py requires code modification every time a software change is implemented.
Modifications:
UMD_utils/ocean_obs.py replaces old code stored as
UMD_utils/ocean_obs.legacy.py
Legacy is conserved because the new xarray readers were not implemented for some legacy observer readers.
New readers might be implemented as needed.
be executed within the same job. This option is slower than spawning multiple observer jobs, but in the long run, it minimizes queue wait times for users.
NOTE: IN_LINE=False (old configuration) should not be used on Milan nodes due to cpu per node quota.
scripts/oda_run.j replaces old code stored as
scripts/oda_run.legacy.j
scripts/ocean_observer.py replaces old code stored as
scripts/ocean_observer.legacy.py
An option BOMB_ON_SIGO was implemented in ocean_das_config so when set to false, it will skip the altimeter data for
the single observer, while conserving the ADT obs for the rest of the cycle. BOMB_ON_SIGO default is set to True, to be kept for observation people to correct observation files.
Hard coded paths where removed and new modules were aded to g5_modules.