Reorganize infrastructure code #2628
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Split the share lib into several libs (e.g., field, grid, remap..) * Move diags lib inside the share folder * Move share/io/tests to share/tests/io * Add subfolders in share/tests, one for each small lib * Keep share lib as a shortcut: links against all other libs, including IO but excluding diagnostics * Split IO lib, to make scorpio interface into its own lib, so that code that only needs them does not have to link against all IO lib (which needs also Field/Grids managers, creating circular deps) * Renamed scream_share->eamxx_share, scream_io->eamxx_io
|
bartgol
force-pushed
the
bartgol/infrastructure-code-reorg
branch
from
53869e0
to
17f90d8
Compare
* Vertical interpolation moved to utils * Time interpolation moved to io (it's only used as a way to interpolate data from file)
bartgol
force-pushed
the
bartgol/infrastructure-code-reorg
branch
from
17f90d8
to
b815e08
Compare
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Failure: Timed out waiting for job SCREAM_PullRequest_Autotester_Mappy to start: Total Wait = 1803
|
|
Status Flag 'Pull Request AutoTester' - Failure: Timed out waiting for job SCREAM_PullRequest_Autotester_Weaver to start: Total Wait = 1803
|
|
Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Console Output (last 100 lines) : SCREAM_PullRequest_Autotester_Mappy # 4663 (click to expand)Console Output (last 100 lines) : SCREAM_PullRequest_Autotester_Weaver # 5143 (click to expand) |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Console Output (last 100 lines) : SCREAM_PullRequest_Autotester_Mappy # 4665 (click to expand)Console Output (last 100 lines) : SCREAM_PullRequest_Autotester_Weaver # 5145 (click to expand) |
tcclevenger
reviewed
Dec 8, 2023
Ok, then I may move fwd with this, although not a big priority right now.
I don't know if changing all names should be done in this PR. I did change some of them, but maybe I should hold back, and we could do a single scream->eamxx PR when there's not much work going on (to avoid hard-to-fix conflicts). OTOH, this PR may already create conflicts for other development, so maybe we should kill two birds with a stone? |
|
Closing, becasue this has fallen so much behind that it's too complicated to sync up with master. If we decide to do something like this again, it will be from scratch, and probably done in steps. |
@bartgol , did you mean to close this PR with this comment? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
First of all, don't get discouraged by the number of files changed in this PR. Most mods are just changing the location of files included.
The main goal of this PR is to reduce the spaghetti-ness of the share folder, which has grown to a very large amount of files. In particular, I tried to split it into smaller libs, which should not have cyclic dependencies (hopefully). The libs are the following (the real name of the lib has the
eamxx_prefix), with their deps listed in parentheses (they should depend only on stuff above).core(ekat): the config defs and basic types of eamxxutils(core,gptl): some common utilsfield(core,utils): the field stack (except for field manager)grid(field): the grid stack (no remappers, not grid manager)scorpio_interface(utils,pioc,piof): all low-level interfaces to scorpio (performing I/O calls with low-level data structures)remap(grid,scorpio_interface): all the remappers infrastructureproperty_checks('field,grid,utils): the property checks classesmanagers(field,grid,remap): the grid and field manager classesatm_process(field,managers,util,property_check): the atm process interfacesio(scorpio_interface,atm_process,remap,managers): the high-level input/output classes (performing I/O calls with eamxx data structures)fvphyshack(remap): hacks to handle pg2 grids in the ad (hopefully to go away at some point)diagnostics(atm_process): all diagnostic classesunit_test_support(managers): the mesh-free GM, the test session, the random test setup.All the tests for these libs (if any) are in the homoninous folder in
share/tests. In particular, io and diags tests are no longer inside those folders.The
scream_sharelib has also been renamed toeamxx_shareand it is an INTERFACE lib, linking to all the above (except forunit_test_supportanddiagnostics, but includingio).Note: I moved the time interpolation utility in the
share/iofolder, along with other IO stuff. That's because it is mainly used to interpolate data from a set of input files. At some point, we may as well call itInputFileTimeInterpolationor something...