Additional variables specific to the scheme are included in the init phase of a _diagnostics scheme in src/physics/ncar_ccpp/diagnostics Each call to history_add_field adds a new field to the end of the possible_field_list_head linked list. At the end of cam_init, the possible field list linked list is used to print the list (to the atm.log* file) and then is converted to the possible_field_list hash table. A sample of the history field list is seen below.
***************** HISTORY FIELD LIST ******************\n T K avg air_temperature\n ZM m avg geopotential_height_wrt_surface\n PHIS m2 s-2 ins surface_geopotential\n PMID Pa avg air_pressure\n PDELDRY Pa avg air_pressure_thickness_of_dry_air\n Q kg kg-1 avg water_vapor_mixing_ratio_wrt_moist_air_and_condensed_water\n CLDLIQ kg kg-1 avg cloud_liquid_water_mixing_ratio_wrt_moist_air_and_condensed_water \n RAINQM kg kg-1 avg rain_mixing_ratio_wrt_moist_air_and_condensed_water\n TTEND K s-1 avg tendency_of_air_temperature_due_to_model_physics\n *************** END HISTORY FIELD LIST ****************\n
"},{"location":"design/history/#capturing-history-output","title":"Capturing history output","text":"Outside of CAM-SIMA init and finalize time, history buffers can be populated with data via a call to history_out_field (found in src/history/cam_history.F90)
The subroutine history_out_field iterates over the hist_configs array and populates the buffer(s) of the hist_field_info_t object of the hist_file_t object if the field name in the call is active on that file (e.g. the field was configured via the namelist to be output for that volume).
- Within the CCPP physics, calls to
history_out_field can exist anywhere except _init and _final - In dynamics, calls to
history_out_field can exist anywhere except in dyn_init, stepon_init, and stepon_final - For physics variables,
history_out_field calls are included in the run phase of the same schemes described in the section above.
"},{"location":"design/history/#defining-new-history-files","title":"Defining new history files","text":"The cam_history.F90 subroutine history_write_files (which is called during cam_timestep_final) does three main actions for each of the user-defined history volumes:
- Determine if it\u2019s time to write to the file based on the user-supplied namelist option hist_output_frequency
- If it's time to write, determine if we need to open a new file
- Write the variables to the file(s)
The bolded step #2 above is what determines if we need to define a new history file. The situations where we would need to define a new file are:
- It's the first sample we're outputting to a specific volume.
- The last time we output to the volume, we \"filled\" the file (number of samples written to the file met the user-configured
hist_max_frames) and closed it
We determine if it's time for a new file with the following line of code:
mod(num_samples, hist_configs(file_idx)%max_frame()) == 0\n
If it is indeed time to define a new file, we call the config_define_file subroutine (which is found in src/history/cam_hist_file.F90) for the volume:
call hist_configs(file_idx)%define_file(restart, logname, host, model_doi_url)\n
- This subroutine opens the netcdf file(s) and writes the necessary time-independent metadata (for the file and for the history fields) and grid information.
- Note that there will be a maximum of TWO (2) files opened at this stage. If there are both instantaneous fields AND accumulated fields (average, minimum, maximum, standard deviation) on the volume, two files will be created (hXi AND hXa); otherwise, only the relevant file will be created (hXi OR hXa)
"},{"location":"design/history/#writing-a-history-file","title":"Writing a history file","text":"The cam_history.F90 subroutine history_write_files (which is called during cam_timestep_final) does three main actions for each of the user-defined history volumes:
- Determine if it\u2019s time to write to the file based on the user-supplied namelist option hist_output_frequency
- If it's time to write, determine if we need to open a new file
- Write the variables to the file(s)
The bolded step #3 above occurs any time the criteria for #1 is satisfied. At this point, the following call is made to write the history fields (whose data has been stored in their buffers via calls to history_out_field):
call hist_configs(file_idx)%write_time_dependent_variables(file_idx, restart)\n
It is during this call that we increment the number of samples written for this volume and actually write the data held within the buffer(s) to the netcdf file (as well as the time-dependent metadata for the fields).
"},{"location":"design/history/#defining-history-restart-files","title":"Defining history restart files","text":"Restarts not yet implemented in CAM-SIMA
"},{"location":"design/history/#writing-a-history-restart-file","title":"Writing a history restart file","text":"Restarts not yet implemented in CAM-SIMA
"},{"location":"design/history/#reading-a-history-restart-file","title":"Reading a history restart file","text":"Restarts not yet implemented in CAM-SIMA
"},{"location":"design/sima-design-goals/","title":"Design goals & features","text":"Motivated by the Singletrack project (a precursor to SIMA), the CAM-SIMA project was created to build a new CAM infrastructure to meet the SIMA science and computational needs. This page documents those needs and some of the features that implement them.
"},{"location":"design/sima-design-goals/#cam-needs-to-be-more-run-time-configurable","title":"CAM needs to be more run-time configurable","text":" - To make CAM and CESM more available, e.g., for usage in containers and the cloud, a build of CAM should be more configurable than it is at present.
- One feature that makes CAM more run-time configurable is moving physics suites to the CCPP. By allowing CAM to compile in more than one physics suite, the physics suite can be selected at run time (e.g., as a namelist variable).
- Another feature needed to make CAM more run-time configurable is making dycores themselves more run-time configurable. For instance, the SE dycore will no longer require the number of advected constituents to be specified at compile time.
"},{"location":"design/sima-design-goals/#remove-obstacles-to-use-of-specialized-high-performance-processing-units-eg-gpus-fpgas","title":"Remove obstacles to use of specialized high-performance processing units (e.g., GPUs, FPGAs)","text":"The chunk structure in CAM physics served its purpose when threading was the only way to accelerate code. However, to make the best use of both threading and modern accelerators, a flexible chunking mechanism is required. The new infrastructure enables this by using flat arrays for all fields.
- Moving to flexible precision of data is important for being able to test both performance improvements and the affect on model quality. The CCPP is explicitly designed to allow for compile-time selection of precision at the physics suite level as well as for individual fields. In addition, the new infrastructure is explicitly designed to handle the case where the dycore is running at a different precision than the physcs (i.e., by using proper field promotion and demotion primitives).
- Pointers in Fortran are less efficient because they prevent some optimization techniques. The new infrastructure avoids pointers as much as possible by making use of the automatic data management capability of the CCPP (which does not create pointers).
- The new infrastructure provides greater flexibility in that the model can be built with multiple physics suites to increase run-time flexibility. There is a tradeoff in that building more physics suites will often increase build time. Builds with a single suite should be faster than now since only the schemes that are required for the suite are compiled (currently most schemes are compiled all the time even if they will not be used).
"},{"location":"design/sima-design-goals/#modularity","title":"Modularity","text":"In order to continue to allow CAM to grow without an ever increasing cost of bringing in new features, CAM must be more modular. A classic example is chemistry which ACOM would like to make modular but which is currently entwined with CAM in many areas (e.g., code in CAM repository, extensive configuration code dedicated to chemistry, extensive namelist building code and data dedicated to chemistry, large number of use cases containing chemistry configuration data). The new CAM infrastructure contains features to increase modularity.
- Support for multiple namelists. This allows modular components to contain their own namelist (run-time configuration options). The active namelists are combined to produce atm_in.
- Flexible handling of constituent information. Modular components can provide constituent information via metadata (if component is a CCPP scheme) or at run time.
Modularity will allow CAM to move components to external repositories. This process cuts development costs for both CAM and for the component (e.g., as has happened with PUMAS). Some ways this is accomplished are listed here:
- Code reviews are more efficient since CAM SEs do not have to review every routine in the external module so they can just focus on the interfaces. The external developers do not have to be involved in CAM modifications.
- Externals can develop and maintain they own namelist definition files, they do not have to coordinate with the larger CAM namelist (which itself has been broken into several smaller namelists).
- Namelists associated with physics schemes do not have to have separate namelist-reading code. The new infrastructure automatically creates an appropriate Fortran module to read in the runtime data from atm_in. The system then also ensures that all active namelists are called at run time. This process ensures that namelists are always read correctly while not requiring coding or reviews to keep up to date with namelist changes.
Use of the CCPP to build physics suites also makes CAM more modular because the CCPP treats physics schemes as modular which allows flexibility in building physics suites. The CCPP takes care of making sure variables are available before they are used and also builds the code currently handled via hand-written code in the various versions of physpkg.F90.
"},{"location":"design/sima-design-goals/#run-time-data-checking","title":"Run-time data checking","text":"CAM needs data to run but the data needs vary with the simulation. The new infrastructure facilitates this.
- Before running physics, the new infrastructures queries the physics suite as to what input fields are required (using a CCPP interface). Then it makes sure that all of these fields have been initialized or reads the values from the initial data file. Any uninitialized fields that are not found on the initial data file will trigger a run-time error.
"},{"location":"design/sima-design-goals/#efficient-offline-testing-and-simulation","title":"Efficient offline testing and simulation","text":"CAM currently has a few ways to run offline testing (e.g., SCAM, PORT). The new infrastructure builds these capabilities in for more efficient and flexible use.
- The new infrastructure has the ability to run without a dycore.
- Offline mode (NULL dycore) can be run with any number of columns.
- Offline mode does not required gridded input.
"},{"location":"design/sima-design-goals/#software-quality-control","title":"Software quality control","text":"To enable efficient quality control, the new infrastructure implements a number of continuous integration (CI) techniques.
- To implement all the flexibility mentioned above, the new infrastructure makes extensive use of python scripts.
- Python scripts make extensive use of python doctests (for simpler tests).
- There are python unit tests to cover more complicated situations.
- The GitHub CI also runs a static analysis tool (pylint) to provide feedback on potential coding issues.
- Python tests can easily be run by hand but are also automatically run on GitHub
- The new infrastructure will use offline mode (see above) to run many physics configurations quickly without requiring large machines.
- This will enable quick testing during development on a laptop.
- We hope many of these tests can also be run automatically on GitHub.
"},{"location":"development/cam-coding-standards/","title":"Coding standards","text":"The standards in this document are largely prescriptive, i.e., they should be followed.
- MUST: Exceptions must be discussed and agreed to by the CAM SEs and noted in this document.
- SHOULD: Exceptions must be approved by the CAM SEs and documented in the ChangeLog.
While some legacy code will not follow these rules, efforts SHOULD be made to improve the code whenever you are working on it (e.g., bug fixes, enhancements).
"},{"location":"development/cam-coding-standards/#general-coding-standards","title":"General coding standards","text":""},{"location":"development/cam-coding-standards/#must","title":"MUST","text":" - Always use spaces instead of tabs
- No trailing spaces (i.e., no spaces at the end of a line)
See tips for configuring editors
"},{"location":"development/cam-coding-standards/#should","title":"SHOULD","text":" - Use comments to explain the purpose of the following code and/or include any important but non-obvious information. When working with code, always check the comments to make sure they are still correct and useful.
- Do not use comments to 'save code for later in case it might be useful'.
- Do not include a comment that merely restates the following code logic (e.g., 'Loop over variables')
"},{"location":"development/cam-coding-standards/#python-coding-standards","title":"Python coding standards","text":"We expect all python code to pass with a perfect score (10) using pylint with this version of pylintrc. However, external repos may have their own pylintrc version depending on their needs.
We also expect all python files to follow the black code style and format.
Finally, one can also follow the Google Python Style Guide.
"},{"location":"development/cam-coding-standards/#fortran-coding-standards","title":"Fortran coding standards","text":"The standards described in this section represent the CAM Fortran standards. Other Fortran standards:
- CTSM Fortran Standards
- MOM Fortran Standards
"},{"location":"development/cam-coding-standards/#must_1","title":"MUST","text":" - No naked
use statements - No continued single-line
if statements (i.e., all if statements should have a then if the statement is on more than one line) - Every namelist variable in each active namelist group is present in the namelist file. An active namelist group is one which may be read during the current run.
- All namelist variables except for logical quantities are initialized to invalid values (integer:
-HUGE(1), real: NaN, character: 'UNSET'). - Functions may not have side effects, and should include the
pure keyword. - Do not combine statements on a single line (i.e., avoid use of the semi-colon to combine statements).
- Use
intent for dummy arguments except for pointers. - All variables of type real must have a specified kind, including literals. For example, use
1.5_r8, not 1.5 or 1.5D0. Literals must also include the decimal point. - All character declarations must use Fortran 90+ syntax (e.g.,
character(len=*) or character(len=CL)). - All variable declarations must use Fortran 90+ syntax (i.e., must include the double colon between the attributes and the variable name).
- All type and procedure declarations must use Fortran 90+ syntax (i.e., must include the double colon before the type or procedure name).
- All modules should include an
implicit none statement in the preamble (after the use statements). Module routines then do not need this statement. - All optional arguments must be passed via keyword (e.g. use
call subroutine(x, optional_y=y) instead of call subroutine(x, y) for the optional variable optional_y). - Initialize local (non-parameter) variables in subroutines and functions at the top of the executable code, NOT on a variable declaration lines.
- Initializing a local variable on a declaration line invokes the
SAVE attribute and is not thread safe. - Local pointer variables MUST be initialized before other (non-initialization) statements. By default, use the
nullify statement.
- All variables that are on the physics grid must have their horizontal dimension declared with
pcols, even if only a subset of the variable is used in the subroutine or function.
"},{"location":"development/cam-coding-standards/#should_1","title":"SHOULD","text":" - Avoid use of preprocessor directives (e.g.,
#if, #ifdef). Always try for runtime variable logic instead. - Keep formula statements relatively short. Use temporary variables to break long formulas into easier-to-read sections.
- Use subroutines to avoid repeated (cut and paste) code logic.
- Avoid side effects in subroutines. Pass variables to routines instead of 'using' them from elsewhere.
- Use the
pure keyword if a subroutine has no side effects. - List dummy arguments one per line, however, related items may be grouped.
- Dummy argument order should match the order in the argument list.
- Use symbolic numerical comparison operators (e.g.,
==, /=, <, >=) not old character versions (e.g., .eq.). - Avoid the use of pointers as dummy arguments (exceptions must be discussed in design or code review)
- Modules should be default
private. Public interfaces are declared after the private declaration. private module interfaces (i.e., subroutines and functions) should be declared private in the module header.- Module names should conform to their filename (i.e., the module name should be the filename without the
.F90). - Functions should use the
pure attribute. If they cannot, the reason should be included as a comment in the function's preamble. - All functions and subroutines should avoid un-necessary statements (e.g. a blank
return at the end of a subroutine). use statements should be brought in at the smallest scope possible (e.g. inside individual subroutines instead of at the module level).
"},{"location":"development/cam-coding-standards/#indentation-and-style","title":"Indentation and style","text":" - Scoping: Indentation should follow scope. That is, whenever entering a new scope (e.g.,
module, subroutine, if, do), indent that scope relative to the scoping statement (recommended 3 spaces but each module should at least be self consistent). - A single line should be less than 133 characters long.
- Continue lines: Indent continue lines 5 spaces or align with similar lines in statement.
- Use spaces to ease reading statements (e.g., before and after operators, after commas except in a dimensions list)
- Include a space after
if, else, end, do, and while. - Include a space before and after
:: - No space after
only, i.e., only:, not only :. - When aligning code for readability, commas go immediately after a symbol (no space).
"},{"location":"development/cam-coding-standards/#tips-for-configuring-editors","title":"Tips for configuring editors","text":""},{"location":"development/cam-coding-standards/#emacs-add-to-your-emacs-file","title":"emacs (add to your .emacs file)","text":" - To automatically remove trailing spaces whenever you save a file:
(add-hook 'before-save-hook 'delete-trailing-whitespace)\n
- To automatically indent with spaces instead of tabs:
(setq-default indent-tabs-mode nil)\n
- To use 4 spaces for each indent:
(setq tab-width 4)\n
"},{"location":"development/cam-coding-standards/#vi-add-to-your-vimrc-file","title":"vi (add to your .vimrc file)","text":" - To automatically remove trailing spaces whenever you save a file:
autocmd BufWritePre * :%s/\\s\\+$//e\n
"},{"location":"development/cam-testing/","title":"Testing","text":"This page describes the various automated and manual tests that are run for CAM-SIMA whenever the code is modified, as well as instructions for how to add new tests.
"},{"location":"development/cam-testing/#python-unit-testing","title":"Python unit testing","text":"CAM-SIMA supports two kinds of python unit tests, doctest and unittest tests, both of which are part of the standard python library.
All unittest tests should be in:
CAM-SIMA/test/unit
while all files used by the tests should be in:
CAM-SIMA/test/unit/sample_files
All unittest tests are automatically run via Github Actions whenever a Pull Request (PR) is opened, modified, or merged.
All doctest tests are also run automatically as long as the scripts they are located in are under CAM-SIMA/cime_config or CAM-SIMA/src/data.
To manually run all of the unit tests at any time, simply run the following shell script:
CAM-SIMA/test/run_tests.sh
Finally, when adding new tests, determine if the test can be done in only a few lines with minimal interaction with external files or variables. If so, then it would likely be best as a doctest. Otherwise it should be a unittest test. Failure to follow this rule of thumb could result in test failures in the Github Actions workflow. Also remember to add your new tests to the run_tests.sh script so future users can easily run the tests manually.
"},{"location":"development/cam-testing/#static-source-code-analysis","title":"Static Source Code Analysis","text":""},{"location":"development/cam-testing/#python","title":"Python","text":"Any python script which is added or modified via a PR will automatically be analyzed using pylint, and must have a score of 9.5 or greater in order to not be marked as a test failure. The hidden pylintrc file used for the analysis can be found here:
CAM-SIMA/test/.pylintrc
Users can also manually run pylint against the core python build scripts by running the following shell script:
CAM-SIMA/test/pylint_test.sh
Please note that pylint is not part of the standard python library, and so it may need to be installed before being able to run the shell script.
"},{"location":"development/cam-testing/#regression-testing","title":"Regression Testing","text":""},{"location":"development/cam-testing/#running-the-regression-tests-manual","title":"Running the regression tests (manual)","text":"NOTE: Regression testing on Derecho should be done for every PR before merging!
Users can manually run regression tests on Derecho to ensure that the model builds correctly in various configurations. The tests can be run with a local copy of CAM-SIMA by using the test_driver.sh script under $CAM-SIMA/test/system. To run the tests associated with a particular compiler option one can do the following commands:
For running GNU tests*:
env CAM_FC=gnu ./test_driver.sh -f\n
For running Intel tests*:
env CAM_FC=intel ./test_driver.sh -f\n
Running the script will produce a directory in your scratch space labeled aux_sima_<CAM_FC>_<timestamp>, where <CAM_FC> is the compiler you chose, and <timestamp> is the timestamp (starting with the date) of when the tests were started, along with a job submitted to the local batch system.
Inside the directory you should see an executable labeled cs.status.*. Running that command after the submitted job has finished will display the test results. Currently for all tests everything should be labeled PASS except the SUBMIT step, which should be labeled PEND. Any other label indicates that a test may have failed, and should be investigated.
Finally, the tests themselves are listed in <CAM-SIMA>/cime_config/testdefs/testlist_cam.xml. Any files that need to be included in order for the tests to run properly are located in <CAM-SIMA/cime_config/testdefs/testmods_dirs/cam/outfrq_XXX, where XXX is the name of the test. Additional information on the CIME testing system, which is what this testing infrastructure is built on, can be found online here.
*Note: you may also have to include the environment variable CAM_ACCOUNT on derecho, which points to your account key
"},{"location":"development/cam-testing/#adding-a-new-regression-test","title":"Adding a new regression test","text":"The test list can be found here: $CAM-SIMA/cime_config/testdefs/testlist_cam.xml
- If you are adding a new machine, compiler or category for an existing test, add a new
<machine> XML entry - If you are adding a fully new test, add a new
<test> XML entry with the following structure:
<test compset=\"<COMPSET_NAME>\" grid=\"<GRID_ALIAS>\" name=\"<TEST_TYPE>_<TEST_MOD>\" testmods=\"<RELPATH_TO_TESTMODS_DIR>\">\n <machines>\n <machine name=\"<MACH_NAME>\" compiler=\"<COMPILER>\" category=\"<TEST_CATEGORY>\"/>\n </machines>\n <options>\n <option name=\"comment\">COMMENT HERE</option>\n <option name=\"wallclock\">WALLCLOCK_TIME</option>\n </options>\n</test>\n
<COMPSET_NAME>: component set alias (or long name) - you can see more about compsets here<GRID_ALIAS>: model grid/resolution you'd like to run on - you can see more about grids here<TEST_TYPE>: type of test to be run. You can find the testing types here.<TEST_MOD>: test modifier that changes the default behavior of the test type. More here<RELPATH_TO_TESTMODS_DIR>: relative path to the testmods directory for this run; usually looks something like \"cam/some_directory_name/\" - The testmods directory will contain any namelist mods and XML configuration variable changes for the test (
user_nl_cam and/or shell_commands) - testmods directories can be found in
$CAM-SIMA/cime_config/testdefs/testmods_dirs/cam/
<MACH_NAME>: machine name - will almost definitely be either derecho or izumi<COMPILER>: compiler to be used (options: gnu, nag, intel, nvhpc)<TEST_CATEGORY>: group of tests that this test belongs to - the default run by test_driver.sh is aux_sima (which is run for each PR to CAM-SIMA)WALLCLOCK_TIME: maximum amount of time that the job will be allowed to run
Here is an example test entry for a 2-timestep smoke test of kessler physics on the MPAS grid, run with both intel and gnu
<test compset=\"FKESSLER\" grid=\"mpasa480_mpasa480\" name=\"SMS_Ln2\" testmods=\"cam/outfrq_kessler_mpas_derecho_nooutput/\">\n <machines>\n <machine name=\"derecho\" compiler=\"intel\" category=\"aux_sima\"/>\n <machine name=\"derecho\" compiler=\"gnu\" category=\"aux_sima\"/>\n </machines>\n <options>\n <option name=\"wallclock\">00:10:00</option>\n <option name=\"comment\">GNU build test for MPAS dycore (with Kessler physics)</option>\n </options>\n </test>\n
"},{"location":"development/debugging/","title":"Debugging techniques","text":"Start with the CAM wiki.
"},{"location":"development/debugging/#cam-sima-specific-debugging","title":"CAM-SIMA-specific debugging","text":""},{"location":"development/debugging/#build-errors","title":"Build errors","text":"Debugging tips if you get build errors:
- If the output indicates that the error message or failure is coming from somewhere within $CAM-SIMA/ccpp_framework:
- If you're getting a clear error message, it's likely that you have something wrong with your metadata
- If you're getting an error message that indicates that something is breaking in the framework code itself (something went uncaught) - consult the AMP SEs
- If the error happens during the atm build, you can see the full output of the atm build in the build log here:
bld/atm.bldlog.*
"},{"location":"development/debugging/#run-time-errors","title":"Run-time errors","text":" - Start with the atm.log* - if the issue occurred during the execution of the CAM code, it will hopefully have a clear and concise error message
- Move to the cesm.log* - it will hopefully include a stack trace for the error in question; if the error did not occur in the CAM code (or CAM did not properly trap the error), it will help you identify the source of the issue.
- If neither log file contains helpful information, a few first steps:
- Resubmit the case; it could be a machine hiccup
- Turn on DEBUG mode (if it's not on already) and rebuild/rerun
- Look in your run directory for any log files called
PETXXX - if there was an issue on the ESMF side of things, it will show up in one of these (there will be one PET file per processor) - Try a different compiler - maybe it'll give you a more helpful error message
- set NTASKS=1 (
./xmlchange NTASKS=1), do a clean rebuild (as instructed), and run again; maybe running in serial will identify the error - Look for the
***************** HISTORY FIELD LIST ****************** in the atm.log* file; if it's not there, the error occurred at init time - If the error occurred during init time, try a new case with a different grid and/or dycore
- If the model ran for a few timesteps before dying (look for the
CAM-SIMA time step advanced message in the atm.log* file), it's likely that one or more variable that you introduced or modified has gone off the rails (value has become very large or very small or zero) - Update your user_nl_cam to output all possible suspected variables to a history file at some point shortly before the model dies, then inspect the output to see if any are obviously wrong
- If the model completed all timesteps, try running a shorter case to see if the problem persists; if so, it's an error during the model finalization
- Run the TotalView debugger on izumi
- Use the old standard - print statements - to narrow down where the code is stopping
- Ask for help!
"},{"location":"development/debugging/#unexpected-answer-changes","title":"Unexpected answer changes","text":" - Two paths here:
- You're getting unexpected DIFFs from the regression testing
- Consult with a scientist about whether differences are expected and for which configurations (compsets, resolutions, namelists parameters, etc)
- If the differences are very small (look like round-off), consult with the other AMP SEs on whether we're ok with this
- If the differences are indeed unexpected and larger than round-off, create a case using the code from the head of
development and: - place print statements in both code bases (your development branch and the head of
development) to identify where the numbers are going awry OR - run the TotalView debugger OR
- use the comparison tool described below (
$CAM-SIMA/tools/find_max_nonzero_index.F90)
- You're getting unexpected answer changes compared with CAM
- Consult with other AMP SEs about whether the differences appear to be due to round-off error
- Use the comparison tool (LINK ONCE IT EXISTS):
$CAM-SIMA/tools/find_max_nonzero_index.F90 - This tool can help you narrow down where the issue begins by printing out values at a specific index and comparing those with the \"truth\" (from CAM)
"},{"location":"development/debugging/#totalview","title":"TotalView","text":"(COURTNEY - ASK CHERYL TO HELP WITH THIS)
- Create and configure a new case (using gnu and only 1 task)
- Build the case (
./case.build) - Run command
bash to change to bash (if not already) - Run the following commands:
np=1\nnthreads=1\n\nsource .env_mach_specific.sh\n\nRUNDIR=`./xmlquery RUNDIR -value`\nEXEROOT=`./xmlquery EXEROOT -value`\nLID=`date '+%y%m%d-%H%M%S'`\n\ncd $RUNDIR\nmkdir timing\nmkdir timing/checkpoints\necho `pwd`\nexport OMP_NUM_THREADS=$nthreads\ntotalview ${EXEROOT}/cesm.exe\n
exit to exit the totalview window and give up the node
"},{"location":"development/git-basics/","title":"Working with git and GitHub","text":""},{"location":"development/git-basics/#getting-started-with-git","title":"Getting started with git","text":" - For a quick, visual introduction to git, try a 5 Minute Overview of Git on Youtube
- Need an introduction to version control along with git and GitHub? Try Git and GitHub for Poets, a Youtube series
- For a glossary of common terms used in git and GitHub documentation, see the GitHub glossary.
- GitHub's Git Handbook offers a brief overview of git and GitHub, including example workflows and links to more in-depth resources.
- Software Carpentry offers a hands-on introduction to git.
- A highly-regarded, comprehensive git reference is the book Pro Git (available as a book or a free pdf).
- The first three chapters of Pro Git, along with sections 6.1 and 6.2 on GitHub, are particularly useful if you're getting started with git. [1]
- The site also contains links to git reference pages along with introductory videos on git.
- You can skip the last section of chapter 3, on rebasing, until you have more experience with git. It's easy to cause problems for yourself or others with rebasing so only use rebasing in CAM-SIMA following the workflow instructions below as rebasing rewrites history, which can make it impossible to reproduce exactly what you have done before.
- For a clear introduction to git internals, try Dissecting Git Guts by Emily Xie.
- Git from the inside out is a blog post with accommpaning video from Mary Rose Cook that looks under the covers a bit to show what happens when you execute git commands.
- Git from the Bottom Up by Josh Wiegley provides a quick glossary of the main terms describing a git repository with links to detailed descriptions. Good site for reminders about a git concept.
- Need to really understand the science behind how git works? Try Advanced Git: Graphs, Hashes, and Compression, Oh My! by Matthew McCullough
Git also offers extensive built-in (i.e., command line) help, although it can be hard to understand until you are familiar with basic git concepts:
git help
git help COMMAND
man gittutorial
man giteveryday
"},{"location":"development/git-basics/#how-to-set-up-your-git-development-environment","title":"How to set up your git development environment","text":"There are several stages in setting up your git and GitHub development environment. Below, they are broken into three sections which represent the different stages:
- One time GitHub setup
- Set up git environment on new machine
- Working with clones
"},{"location":"development/git-basics/#one-time-github-setup","title":"One time GitHub setup","text":" - Set-up personal github account (if one does not already exist): https://github.com
-
Create a new personal CAM-SIMA fork of the ESCOMP/CAM-SIMA repo:
-
Set up SSH keys on GitHub (optional):
If you will be pushing changes to GitHub (either to your own fork or to shared forks), or if you will be pulling changes from private repositories on GitHub, then it's worth the time to set up ssh keys for each machine you'll be using. By doing so, you won't have to enter your password when pushing to or pulling from GitHub.
See here for instructions. After doing this, you can use the ssh form of GitHub URLs (e.g., git@github.com:ESCOMP/cam-sima.git) in place of the https form.
-
Configure GitHub notifications (optional): It is important to keep track of activity on GitHub but there are ways to manage how you are notified.
- Controlling how you receive notifications: Click on your profile picture in the upper-right of any GitHub page, then click on \"Settings\", and then on \"Notifications\". You will see several options, here are a few recommended settings:
- Automatically watch repositories: Yes (check box)
- Participating: Email
- Watching: Web
- To see and manage Web notifications: Click on the bell in the upper right of any GitHub page to see Web notifications and to manage which repositories you are watching. There will be a blue dot on the bell if there are new (unread) notifications).
"},{"location":"development/git-basics/#set-up-git-environment-on-new-machine","title":"Set up git environment on new machine","text":"git has global settings which apply to all repository clones on your machine. These settings reside in a file called .gitconfig in your home directory. Below are some required and some optional but recommended global git settings to apply to any new machine where you will do CAM-SIMA development (e.g., Derecho, Izumi, personal laptop). Apply the settings below or simply copy a .gitconfig file from a machine that is already configured.
"},{"location":"development/git-basics/#required-git-global-configuration-settings","title":"Required git global configuration settings","text":"git config --global user.name \"Your Name\"\ngit config --global user.email <GitHub email address>\n
We recommend that you use your UCAR email address as your GitHub email address but if you use another address, you can add your UCAR email address by clicking on your profile picture in the upper-right of any GitHub page, then clicking on \"Settings\", and then on \"Emails\".
"},{"location":"development/git-basics/#recommended-git-global-configuration-settings","title":"Recommended git global configuration settings","text":"You can set which editor to use for log messages, etc., with:
git config --global core.editor <editor of your choice: emacs, vi, vim, etc>\n
(See http://swcarpentry.github.io/git-novice/02-setup for specific settings to use for many common editor choices.)
The following setting generates better patches than the default:
git config --global diff.algorithm histogram\n
The following setting makes it easier to resolve conflicts if you're doing conflict resolution by hand as opposed to with a dedicated conflict resolution tool. Without this setting, you'll just see your version and the version you're merging in delimited by conflict markers. With this setting, you'll also see the common ancestor of the two sides of the merge, which can make it much easier to figure out how to resolve the conflict:
git config --global merge.conflictstyle diff3\n
Alternatively, look into using a graphical conflict-resolution tool such as kdiff3 or the Emacs built-in M-x vc-resolve-coflicts.
We recommend that you set git to not push anything by default:
git config --global push.default nothing\n
This can help prevent you from accidentally pushing work to the wrong repository.
"},{"location":"development/git-basics/#configuring-git-on-shared-machines","title":"Configuring git on shared machines","text":"If using git on shared resources, such as on the login nodes for CISL machines, then one may find their git commands being killed by sys admins due to git spawning too many threads and thus blocking (or at least slowing down) other users. To avoid this situation, you can limit the number of threads git spawns for various activities by setting the following git config variables:
git config --global --add index.threads 8\ngit config --global --add grep.threads 8\ngit config --global --add pack.threads 8\n
Please note that a limit of 8 threads was chosen specifically for CISL machines. If you are using a separate shared system you may find it beneficial to choose a different thread limit.
"},{"location":"development/git-basics/#git-tools-for-bash","title":"git tools for Bash","text":"There are two helpful things you can do to streamline your use of git if you use the bash shell. These are completely optional, but improve your git experience. These are documented in the appendix of the excellent Pro Git book.
"},{"location":"development/git-basics/#updating-your-login-to-a-github-token","title":"Updating your login to a GitHub token","text":"If you have been denied access to push to a GitHub repo, note that the old password system has been deprecated and no longer works. If you do not make changes, you will no longer be able to work with GitHub on personal forks and private repos.
If you only ever git clone from public repositories like ESCOMP and ESMCI, you may ignore the rest of this wiki page.
If you've already created and are using a GitHub token, you may ignore the rest of this wiki page.
GitHub will soon be requiring that you use a GitHub generated token (basically a long password that they autogenerate for you).
- If you are on a Mac and do not need to use your password, but still receive an error message, you may have stored your password in the keychain. Remove a password on Macs
- Create a GitHub password token. Make sure you copy it to a safe place as they only show it to you once.
-
Store your credentials on every machine you use GitHub on.
-
On a private / secure Linux system:
git config --global credential.helper store
-
Note that the store option to git's credential helper stores your GitHub token in plain text. If you are on a public machine, you should store your GitHub token in a password manager and use the credential cache mechanism to open command-line access to your GitHub repositories:
git config --global credential.helper cache --timeout <seconds>
where <seconds> is the number of seconds git 'remembers' your token. For example, to only have to enter your token once per day:
git config --global credential.helper cache --timeout 86400
-
On Windows:
git config --global credential.helper wincred - On Macs (make sure you select the Mac tab in the middle of the window): see this documentation
-
On each machine, do a git clone of your personal repo. It will ask for your username and password. The password needs to be your saved token. You should only need to do this once.
- To test that your credentials are successfully cached, do a new
git clone of your personal repo. If it clones without asking for your username/password you are done.
"},{"location":"development/git-basics/#working-with-clones","title":"Working with clones","text":"Here are some commands for creating and working with clones:
"},{"location":"development/git-basics/#create-a-new-clone","title":"Create a new clone","text":"git clone https://github.com/<GitHub userid>/CAM-SIMA\ncd CAM-SIMA\n
or
git clone git@github.com:<GitHub userid>/CAM-SIMA\ncd CAM-SIMA\n
where <GitHub userid> is your GitHub account login ID. Some useful options to the clone command are:
--origin <origin name>: A clone knows where it came from and by default, calls that location, \"origin\". You can change that name with the --origin option. This can come in handy when dealing with multiple upstream repositories. Change the clone directory name: By default, the clone is created in a directory with the same name as the repository. You can change this by adding a directory name to the clone command. Use the appropriate example below:
git clone https://github.com/<GitHub userid>/CAM-SIMA <clone_dir_name>\ncd <clone_dir_name>\n
or
git clone git@github.com:<GitHub userid>/CAM-SIMA <clone_dir_name>\ncd <clone_dir_name>\n
"},{"location":"development/git-basics/#checking-out-a-tag-or-switching-to-a-new-tag","title":"Checking out a tag or switching to a new tag","text":" git checkout <tag>\n
note that <tag> can also be the name of a branch or a commit hash. If you specify the name of a branch, you will check out the head of the branch. If you name a remote branch (e.g., origin/branch_name), you will create a detached HEAD but you can still use the code. Please note that if you plan on changing the code, first create a branch (see Working with branches
"},{"location":"development/git-basics/#working-with-branches","title":"Working with branches","text":"When you create a clone, your clone will contain pointers all the branches that existed at the clone's origin (e.g., the repository at GitHub). While you can check out these branches, however, before attempting to make any changes, you should first create a local version branch (so git can keep track of the local commits).
- To create a new local branch that starts at a certain point:
git branch <new branch name> <tag or branch name>\n
for example
git branch new_feature cam6_2_024\n
- To check out a local branch:
git checkout <new branch name>\n
- If you are working with a repository that uses
git-fleximod (e.g., CAM-SIMA, CESM), always run that tool after checking out a new branch or tag:
bin/git-fleximod update\n
"},{"location":"development/git-basics/#working-with-remotes-upstream-repository-locations","title":"Working with remotes (upstream repository locations)","text":"While working with clones created using the methods above will be sufficient for most if not all of your development needs, there may be times when you will want to access or compare your code with code from a different repository. git has no problem storing revisions from multiple repositories in a single clone!
To begin, your clone probably has a single remote (also known as an upstream repository). To see the current status of which upstream repositories are configured for your clone, use the git remote command:
git remote\n
To see the location of the remote repositories in your current directory:
git remote -v\n
You should see something like:
origin https://github.com/gituser/CAM-SIMA (fetch)\norigin https://github.com/gituser/CAM-SIMA (push)\n
This tells you the \"upstream\" location from where new code is downloaded (when you run git fetch origin) or where code is uploaded (when you run git push origin <branch>). Note that most git commands are purely local, using only information in the .git directory of your clone.
You can rename an existing remote:
git remote rename origin ESCOMP\n
You can set the remote name as part of a clone command (the default is 'origin'):
git clone -o ESCOMP https://github.com/ESCOMP/cam-sima\n
"},{"location":"development/git-basics/#adding-remote-new-upstream-repository-locations","title":"Adding remote (new upstream repository locations)","text":"To add a new upstream repository, use the remote add command. For example:
git remote add ESCOMP https://github.com/ESCOMP/CAM-SIMA\ngit fetch --tags ESCOMP\n
You should see messages much like a new clone when you execute the git fetch command. Note that you can call the new remote anything, in this example we are calling it ESCOMP.
"},{"location":"development/git-basics/#updating-your-branch-to-latest-development","title":"Updating your branch to latest development","text":"Note that while this section explains how to update your local branch to the ESCOMP/CAM-SIMA/development branch, the instructions can easily be generalized for any branch from any upstream remote.
Before starting, you should have either:
- A fresh clone of your fork with the branch you wish to update checked out (see Create a new clone and Working with branches.
- An existing clone with the branch you wish to update checked out and in a clean state (i.e., make sure you do a
git commit and that git status shows no modified files).
Add the upstream remote, if you have not already done so (see Adding remotes).
Merge the specific remote/branch into your branch. In this example, it is ESCOMP/development
git fetch ESCOMP\ngit merge ESCOMP/development\n
"},{"location":"development/git-basics/#comparing-differences-using-git-diff","title":"Comparing differences using git diff","text":"If you have a git clone, you can view differences between commits or tags. As far as git diff is concerned, a commit hash is the same as a tag so in the examples below will use <tag>.
- To see the full difference between two tags (i.e., a changeset):
git diff <tag1> <tag2>\n
- To see the full difference between the current checkout (sandbox) and a tag:
git diff <tag>\n
- To see only the names of files that are different:
git diff --name-only <tag> [ <tag2> ]\n
- To see the difference in one or more specific files:
git diff <tag> [ <tag2> ] -- <path_to_file1> <path_to_file2>\n
"},{"location":"development/git-basics/#configuring-and-using-a-graphical-difference-tool","title":"Configuring and using a graphical difference tool","text":"git has a command, difftool, that can run a graphical tool on each file that is different between two commits.
- To configure
opendiff as the graphical difference tool:
git config --global diff.tool opendiff\n
- To see the available graphical difference tools:
git difftool --tool-help\n
- To run
difftool on <file1> and <file2>
git difftool <tag> [ <tag2> ] -- <path_to_file1> <path_to_file2>\n
- To optionally run
difftool on all files in a changeset (answer 'y' or 'n' for each file):
git difftool <tag> [ <tag2> ]\n
- To run
difftool on all files in a changeset (i.e., same as 'y' for every file):
yes | git difftool <tag> [ <tag2> ]\n
"},{"location":"development/git-basics/#using-diffmerge","title":"Using diffmerge","text":"This section contains the complete modifications needed for using the graphical tool diffmerge for \"git difftool\" and \"git mergetool\"
There is a tool called diffmerge which enables both side-by-side comparison of edited files in git as well as providing a three way-pane for editing merge conflicts. This tool is available for download at: https://sourcegear.com/diffmerge/. It has been installed on izumi and derecho by the system administrators in public areas, so you don't need to download it for those machines.
To use the differencing tool type: git difftool
If after a git merge the git command says there are conflicts, then you may type git mergetool to allow you to resolve the conflicts and complete the merge. The mergetool with diffmerge properly installed will display three panes. From left to right these panes are:
Code being merged in Merged code Your code\n
The panes are opened in a visual editor, and any changes you make in the middle pane, may be saved by selecting the save icon at the top of the screen and then exiting the window. This will finalize the merge for that particular file.
For a presentation which can be used as a tutorial, you may refer to: Presentation on diffmerge tool for git
The following modifications may be copy/pasted into the user's .gitconfig file which resides in the home directory. Since the potential for making an editing mistake is possible, it is recommended that a copy be made of the .gitconfig file prior to these edits in case an error is discovered.
If you have problems, check out the tips at the bottom of this page
[diff]\n\u00a0 \u00a0 \u00a0 \u00a0 tool = diffmerge\n\u00a0 \u00a0 \u00a0 \u00a0 algorithm = histogram\n[difftooldiffmerge]\n\u00a0 \u00a0 \u00a0 \u00a0 cmd = diffmerge \\\"$LOCAL\\\" \\\"$REMOTE\\\"\n[difftool \"diffmerge\"]\n\u00a0 \u00a0 \u00a0 \u00a0 cmd = diffmerge \\\"$LOCAL\\\" \\\"$REMOTE\\\"\n[difftool]\n\u00a0 \u00a0 \u00a0 \u00a0 prompt = false\n[push]\n\u00a0 \u00a0 \u00a0 \u00a0 default = simple\n[merge]\n\u00a0 \u00a0 \u00a0 \u00a0 tool = diffmerge\n\u00a0 \u00a0 \u00a0 \u00a0 ff = false\n[mergetool \"diffmerge\"]\n\u00a0 \u00a0 \u00a0 \u00a0 cmd = diffmerge --merge --result=$MERGED $REMOTE $BASE $LOCAL\n[mergetool]\n\u00a0 \u00a0 \u00a0 \u00a0 keepBackup = false\n
Useful tips
- On derecho, if you get an error message about diffmerge not being loaded, make sure you are using
ncarenv/23.09 or newer (to check say \"module list\") - If you do not get a response when clicking on the \"git mergetool\" window, make sure you find the \"conflict\" popup and hit \"exit\" on it
- If your git mergetool gives blank windows and says: \"Files are identical or equivalent under the current RuleSet\", hit OK and then go to the File dropdown menu and select \"Reload\". This filled in the blank windows for the user with this error.
- CAUTION: Check your file carefully if you do this, as a second user who did this discovered that all of the changes which were automatically merged did not reside in the final saved version.
"},{"location":"development/git-faq/","title":"git & GitHub FAQ","text":""},{"location":"development/git-faq/#q-how-can-i-clone-someones-git-clone-to-my-directory-on-the-same-machine","title":"Q: How can I clone someone's git clone to my directory on the same machine?","text":"A: git clone <path_name> [<local_dir>] where <path_name> is the location of the source git clone and the optional <local_dir> is the name of the clone (default is a local directory of the same name as the original).
"},{"location":"development/git-faq/#q-how-can-i-clone-someones-git-clone-to-my-directory-on-a-different-machine","title":"Q: How can I clone someone's git clone to my directory on a different machine?","text":"A: git clone ssh://<user>@<machine>:<path_name> [<local_dir>] where <user> is your user name on the remote machine, <machine> is the machine name (e.g., derecho.hpc.ucar.edu), <path_name> is the location of the source git clone on the remote machine, and the optional <local_dir> is the name of the clone (default is a local directory of the same name as the original).
"},{"location":"development/git-faq/#q-how-can-i-look-at-someones-pr-code","title":"Q: How can I look at someone's PR code?","text":"A: There a a few ways to do this:
- On GitHub (like looking at any other code on GitHub)
- Add the PR fork to my remote (allows using tools such as
git diff or git difftool with your existing branches or development) - As a clone (standalone clone on your machine).
A first step is to find the link to the fork's branch. Just below the PR is a line that starts with a colored oval (e.g., \"Open\" in green) and looks something like:
Octocat wants to merge 123 commits into ESCOMP:development from Octocat:amazing-new-feature\n
Clicking on the last part (Octocat:amazing-new-feature) will take you to the correct branch where you can browse the code (the first method above). If you want to download that code, click the green \"Code\" button and then click the clipboard icon. Be sure to take note of the branch name.
- To load this code into your clone, cd to your clone directory, add the PR fork as a new remote, and checkout the branch. For instance:
git remote add octocat https://github.com/octocat/CAM-SIMA.git\n git fetch --no-tags octocat\n git checkout octocat/amazing-new-feature\n
Instead of the checkout you can also do diffs:
git difftool origin/development octocat/amazing-new-feature\n
- If you want to make a new clone with the PR code, simply do:
git clone -b amazing-new-feature octocat https://github.com/octocat/CAM-SIMA.git octocat_cam\n cd octocat_cam-sima\n
"},{"location":"development/git-faq/#q-why-do-pull-request-pr-code-review-take-so-long","title":"Q: Why do Pull Request (PR) code review take so long?","text":"A: A code review must fulfill three purposes:
- Code reviewers must make sure that new and/or changed code does not affect any currently-supported functionality (i.e., it cannot break anything).
- While regression tests will catch many of these issues, reviewers must also check for usage of or reliance on deprecated code, and also for any code which is not supported on all platforms and compilers used by the CESM community.
- Code reviewers must make sure that any new functionality or changes are implemented correctly and at least somewhat efficiently. They must also ensure that important changes are tested against future regressions.
- The CAM SE team is almost always engaged in several projects to implement new CAM functionality along with supporting infrastructure. Each CAM SE usually looks at each PR in order to prevent new code from interfering with those plans.
The first two steps are usually completed by a single SE although SEs engaged in a final review will often find missed errors. This is similar to peer reviewers finding problems with a paper even after reviews done by colleagues.
"},{"location":"development/git-faq/#q-how-do-i-update-my-branch-to-the-current-cam_development","title":"Q: How do I update my branch to the current cam_development?","text":"A: see this section
"},{"location":"development/git-fleximod/","title":"git-fleximod","text":""},{"location":"development/git-fleximod/#populate-andor-update-your-externals","title":"Populate and/or update your externals","text":"Run bin/git-fleximod update
The process will either:
- complete and you're good to go
- indicate to you that modifications have been made in certain modules
- this means you should either go commit those mods or remove them
- you can also (if you're sure!) run
bin/git-fleximod update -f to force overwrite all externals
"},{"location":"development/git-fleximod/#update-external-repo-or-tag","title":"Update external repo or tag","text":"To add or update an external repo to CAM-SIMA, the following steps must be done:
- Modify the
.gitmodules file at the head of CAM-SIMA to add or update the external. Explanations for what each entry in the .gitmodules file is can be found on Github here - Once the
.gitmodules file has been updated, go to the head of CAM-SIMA and run bin/git-fleximod update. This will bring in the new external code into your local repo (but will not commit them). - Once you are ready to officially commit the changes, then make sure to commit both the modified
.gitmodules file, and the updated submodule itself. An easy way to make sure you have commited everything is to run git status and make sure there are no files or directories that have been modified but are still un-staged.
Once all of that is done then congrats! A new external has been successfully added/updated.
"},{"location":"development/tool-recommendations/","title":"Tool recommendations","text":"This page lists recommendations for various tools one might use during CAM-SIMA development.
Please note that these standards are currently in development, and thus any of them could change at any time.
"},{"location":"development/tool-recommendations/#version-control","title":"Version Control","text":"The recommended version control tool is git.
"},{"location":"development/tool-recommendations/#repository-hosting","title":"Repository Hosting","text":"We recommend Github for hosting software repositories
"},{"location":"development/tool-recommendations/#cicd","title":"CI/CD","text":"When possible, we recommend running any CI/CD workflows using Github Actions.
"},{"location":"development/tool-recommendations/#container-software","title":"Container Software","text":"We recommend using Docker for building and running containers.
More to Come!
"},{"location":"usage/creating-a-case/","title":"Creating, configuring, and running a case","text":"Because CAM-SIMA uses the CIME build system, creating and configuration a case is mostly the same as it was in CAM7 (and in CESM). But! We're going to assume you know nothing.
"},{"location":"usage/creating-a-case/#1-run-create_newcase","title":"1. Run create_newcase","text":"Before you create a case:
- Choose the correct machine to run your case on (if this is a high-resolution or long run, you will likely want to choose derecho!)
- Consult the git basics if you need a new clone
- Make sure you have run
bin/git-fleximod update to update and/or populate your externals - Navigate to
$CAM-SIMA/cime/scripts
Assuming you have completed the above steps and have a nice checkout of a development branch of CAM-SIMA, let's go!
Here is the (basic) structure of the command for create_newcase:
./create_newcase --case <CASEDIR> --compset <COMPSET_NAME> --res <RESOLUTION> --compiler <COMPILER> --project <PROJECT KEY> --run-unsupported\n
To break that down piece by piece:
--case <CASEDIR> (optional): The case argument specifies the full path to the directory you would like to be created, in which your case will reside. - pro tip: if you choose to put your case directory in your scratch directory, your
run and bld directories will be automatically placed within the case directory - on derecho, this means:
--case /glade/derecho/scratch/<username>/<case_name> - on izumi, this means:
--case /scratch/cluster/<username>/<case_name>
- if you do not specify a case directory, it will default to your
$CAM-SIMA/cime/scripts directory (and, eventually, your run and bld directories will default to a new directory in your scratch space)
--compset <COMPSET_NAME> (required): the compset (or \"component set\") tells CIME which base-level configurations you want to run with (including model versions and input data sets) - The compset supplied to the
create_newcase command can be either: - The long name of a compset OR
- The alias (if it exists) for the compset
- Compset options and aliases can be found in here (
$CAM-SIMA/cime_config/config_compsets.xml) - The structure of a compset long name is
<initialization time>_<atmosphere model>_<land model>_<sea-ice model>_<ocean model>_<river runoff model>_<land ice model>_<wave model> - Additional context: for each of the seven model components of the compset configuration, there can be three different implementations:
- active (prognostic, full): solves a complex set of equations to describe model's behavior (must solve all equations on a numerical grid)
- data: version that sends/receives same variables to/from other models, but read from files rather than computed from equations (reduces feedback within a system)
- stub: occupies the required place in the driver but does not send or receive data
--res <RESOLUTION> (required): the resolution specifies both the dycore being run and the grid. - The structure of the resolution argument is:
<atmosphere>_<ocean>_<mask> - The resolution determines the dycore:
- A grid resolution that looks like \"ne*\" indicates that this is spectral element (SE) cube sphere grid (SE dycore)
- A grid resolution that looks like \"C*\" indicates that this is the FV3 (finite volume cubed sphere) dycore [not yet implemented in CAM-SIMA]
- A grid resolution that looks like \"mpasa*\" indicates that this is an MPAS (Model for Prediction Across Scales) grid (MPAS dycore) [work-in-progress in CAM-SIMA]
- You can find the options for these model grid aliases in
$CAM-SIMA/ccs_config/modelgrid_aliases_nuopc.xml - Most often, for testing CAM-SIMA, we use a coarse SE grid like
ne3pg3_ne3pg3_mg37 or ne5_ne5_mg37
--compiler <COMPILER> (optional): the compiler you wish to use - Options:
- on derecho:
- intel (default)
- gnu
- nvhpc
- on izumi:
- intel (default) - NOTE: intel is not to be trusted on izumi (it's a known-buggy version)
- gnu
- nag - NOTE: until the nag version is updated beyond 7.0, CAM-SIMA won't work with nag
--project <PROJECT KEY>: a project key to charge to - Only needed on derecho
- You can see the project keys you can use here (click on your primary group)
- You can also see how much of the allocation has been used/is left by navigating to Reports -> Project Search and searching for the key
--run-unsupported: this flag is mostly in place to indicate to scientists that the configuration they're choosing to run is not scientifically supported.
Given all that, let's say you run the following command on izumi (assuming you're this mysterious person \"courtneyp\"):
./create_newcase --case /scratch/cluster/courtneyp/kessler-ne5-gnu-0722 --compset FKESSLER --res ne5_ne5_mg37 --compiler gnu --run-unsupported\n
What will happen is that a new case directory will be created here: /scratch/cluster/courtneyp/kessler-ne5-gnu-0722 that will be configured to run the FKESSLER compset (long name: 2000_CAM%KESSLER_SLND_SICE_SOCN_SROF_ SGLC_SWAV) with the SE dycore and the ne5 grid.
"},{"location":"usage/creating-a-case/#2-configure-the-case","title":"2. Configure the case","text":"Navigate into your shiny new case directory and run ./case.setup
From here, there are A LOT of configuration options. We'll highlight a few here that you'll use often.
"},{"location":"usage/creating-a-case/#xml-configurations","title":"XML configurations","text":"Many configurable settings can be found within the env_*.xml files in your case directory. In order to protect again typos, it's not advised to edit those directly. Instead, you will run ./xmlchange commands to alter those settings (and ./xmlquery can give you the current setting). You can find additional information on the configurable xml variables here. A few to highlight:
CAM_CONFIG_OPTS: This is where we tell CAM-SIMA what physics scheme(s) we wish to run, as well as if we wish to run the null dycore. - FKESSLER will default to
--physics-suites kessler --analytic_ic, which means we're running the suite_kessler.xml SDF with the SE dycore (with analytic initial conditions - no need to supply an ncdata initial conditions file) - If you instead want to run a test with the null dycore, you'll want to change this with
./xmlchange CAM_CONFIG_OPTS = \"--physics-suites kessler --dyn none\"
CAM_LINKED_LIBS: Hopefully, you won't have to change this often; however, if you are getting linking errors during your build, try turning off the linked libraries (./xmlchange CAM_LINKED_LIBS='')DOUT_S: This is the flag to archive log files of successful runs. During development, we advise turning this off so your log files don't disappear on you (./xmlchange DOUT_S=False)STOP_OPTION & STOP_N: How long you are going to run the model. If STOP_N is 8 and STOP_OPTION is \"ndays\", you are setting the model up to run for 8 days. - the options for
STOP_OPTION are: 'nsteps', 'nseconds', 'nminutes', 'nhours', 'ndays', 'nmonths', 'nyears' STOP_N is an integer- NOTE: if you are running the ncdata_check with snapshot files, keep the number of timesteps you are running at or below the number of timesteps on the snapshot files
- NOTE #2: These configurations can be updated without rebuilding the case
DEBUG: A flag to turn on debug mode for the run (runs without compiler optimizations) - this is very useful during development! It defaults to \"False\", so you can turn it on with ./xmlchange DEBUG=TrueRUNDIR: this is the path to the run directory for your case. The bld directory will exist one level up.
If you run ./xmlchange, the output will tell you if you need to re-set up and/or do a clean build of your case.
"},{"location":"usage/creating-a-case/#namelist-configurations","title":"Namelist configurations","text":"There are countless namelist configuration options. These can be modified by updating the user_nl_cam file in your case directory. Namelist options can be updated without rebuilding the case.
- How to configure the namelist for history/output
- Information on namelist reader generation
- You can see the full CAM6 list of namelist options here, keeping in mind that most of those options have not yet been ported to CAM-SIMA
"},{"location":"usage/creating-a-case/#3-build-the-case","title":"3. Build the case","text":"Run ./case.build
The console output will tell you the progress of the build. If you get to MODEL BUILD HAS FINISHED SUCCESSFULLY, hooray!
Debugging tips
"},{"location":"usage/creating-a-case/#4-run-the-case","title":"4. Run the case","text":"Run ./case.submit
The job will be submitted to the system's queue. You can see the status of your job with the qstat command. Once it is finished, the log files will be in the run directory (unless it ran successfully AND your archiver is on)
Debugging tips
"},{"location":"usage/history/","title":"History & model output","text":""},{"location":"usage/history/#configuring-the-namelist","title":"Configuring the namelist","text":"The syntax for updating a the configuration for a history file is:
<namelist_option>;<volume>: <setting>\n
Possible namelist options for your user_nl_cam:
hist_add_avg_fields: add average fields to a specified history filehist_add_inst_fields: add instantaneous fields to a specified history filehist_add_min_fields: add minimum fields to a specified history filehist_add_max_fields: add maximum fields to a specified history filehist_remove_fields: remove a given field from a specified history filehist_file_type: type of file (options are \"history\", \"satellite\", and \"initial_value\") - defaults to \"history\"hist_max_frames: maximum number of samples written to a specified history file (after which a new one will be created)hist_output_frequency: frequency with which to write samples to a specified history file (syntax is <integer>*<multiplier> like 3*nhours)hist_precision: precision of the specified history file (options are \"REAL32\" and \"REAL64\") - defaults to \"REAL32\"hist_write_nstep0: logical for whether or not to write the nstep = 0 sample (defaults to .false.)hist_filename_template: template filename for the specified history file - Defaults to \"%c.cam.%u.%y-%m-%d-%s.nc\" where \"%c\" is the case name, \"%u\" is the volume, \"%y\" is the year, \"%m\" is the month, \"%d\" is the day, and \"%s\" is the number of seconds since midnight GMT, with the timestamp itself representing the model time when the file is created.
Example Take the following sample user_nl_cam:
hist_output_frequency;h1: 5*ndays\nhist_max_frames;h1: 3\nhist_add_inst_fields;h1: U\nhist_add_inst_fields;h1: V, Q\nhist_precision;h1: REAL64\nhist_filename_spec;h1: my-history-file%m-%d\nhist_write_nstep0;h1: .false.\n
It will be parsed by hist_config.py and this will be the relevant section of atm_in:
&hist_config_arrays_nl\n hist_num_inst_fields = 3\n hist_num_avg_fields = 2\n hist_num_min_fields = 0\n hist_num_max_fields = 0\n hist_num_var_fields = 0\n/\n\n&hist_file_config_nl\n hist_volume = 'h0'\n hist_avg_fields = 'T', 'Q'\n hist_max_frames = 1\n hist_output_frequency = '1*month'\n hist_precision = 'REAL32'\n hist_file_type = 'history'\n hist_filename_spec = '%c.cam.%u.%y-%m-%d-%s.nc'\n hist_write_nstep0 = .false.\n/\n\n&hist_file_config_nl\n hist_volume = 'h1'\n hist_inst_fields = 'U', \u2018V\u2019, \u2018Q\u2019\n hist_max_frames = 3\n hist_output_frequency = '5*ndays'\n hist_precision = 'REAL64'\n hist_file_type = 'history'\n hist_filename_spec = 'my-history-file%m-%d'\n hist_write_nstep0 = .false.\n/\n
In plain English, a one-month run with these history configuration will result in a total of three files that will look something like these:
- my-history-file01-06.nc
- This file will contain instantaneous output for U, V, and Q (eastward_wind, northward_wind, and water vapor)
- It will contain three frames, one at each of the following times:
- 0001-01-06 (time=5)
- 0001-01-11 (time=10)
- 0001-01-16 (time=15)
- my-history-file01-21.nc
- This file will contain instantaneous output for U, V, and Q (eastward_wind, northward_wind, and water vapor)
- It will contain three frames, one at each of the following times:
- 0001-01-21 (time=20)
- 0001-01-26 (time=25)
- 0001-01-31 (time=30)
- .cam.h0a.0001-02-01-00000.nc
- This file will contain averaged output for T and Q (air_temperature and water vapor)
- It will have one frame with the time calculated at the midpoint of the month
"},{"location":"usage/history/#adding-a-diagnostic-field-to-the-cam-sima-source-code","title":"Adding a diagnostic field to the CAM-SIMA source code","text":"During init time, fields can be added to the possible field list with a call to history_add_field:
history_add_field(diagnostic_name, standard_name, vdim_name, avgflag, units, gridname, flag_xyfill, mixing_ratio)
Field Optional? Type Description diagnostic_name No string diagnostic name for the field - will be the name in netcdf output file standard_name No string CCPP standard name for the variable vdim_name No string vertical dimension: 'horiz_only' for no vertical dimension; 'lev' for vertical_layer_dimension; 'ilev' for vertical_interface_dimension avgflag No string default average flag; options: 'avg', 'lst' (instantaneous), 'min', 'max', 'var' (standard deviation) units No string variable units gridname Yes string gridname on which the variable's data is mapped (defaults to the physics grid) flag_xyfill Yes string fill value for variable values mixing_ratio Yes string constituent mixing ratio type ('wet' or 'dry'); not set to anything if not passed in Example:
call history_add_field('Q', 'water_vapor_mixing_ratio_wrt_moist_air_and_condensed_water', 'lev', 'avg', 'kg kg-1', mixing_ratio='wet')\n
It's important to avoid adding calls to history_add_field to the CCPP-ized physics schemes (to keep them portable). Instead, create a new diagnostics scheme and place that in the diagnostics folder of the atmospheric_physics repository. The history_add_field call will be in the init phase.
"},{"location":"usage/history/#outputting-a-diagnostic-field-to-the-cam-sima-source-code","title":"Outputting a diagnostic field to the CAM-SIMA source code","text":"After init time, a variable's current values can be captured for output with a call to history_out_field:
history_out_field(diagnostic_name, field_values)
Field Optional? Type Description diagnostic_name No string diagnostic name for the field - will cause an error if the diagnostic name was not added via history_add_field field_values No real variable values to be stored in the history buffer(s) Example:
call history_out_field('Q', const_array(:,:,const_idx))\n
It's important to avoid adding calls to history_add_field to the CCPP-ized physics schemes (to keep them portable). Instead, create a new diagnostics scheme and place that in the diagnostics folder of the atmospheric_physics repository. The history_out_field call(s) will likely be in the run phase.
"},{"location":"usage/history/#using-the-output","title":"Using the output","text":"The output files can be found in your run directory. They are in netCDF format.
See the ADF for lots that you can do with your results!
A few useful commands for inspecting netCDF data:
- To output the header/metadata information for the file (includes list of variables on the file):
ncdump -h file.nc\n
- To output the data for a specific variable on the file:
ncdump -v <varname> file.nc\n
cprnc <file1_path> <file2_path>\n
- To get a slice of one file (subset of time samples):
ncks -F -d time,<start time>,<end time>,<step> <snapshot_file> <output_file_name>\n
Example to get time samples 3-5 from a file onto a new file called \"split-file.nc\":
ncks -F -d time,3,5,1 file.nc split-file.nc\n
- To diff a variable between two files (and output the results to a new netcdf file):
ncdiff -v <field_name> -o <output_file> <file1> <file2>\n
*cprnc can be found:
- on derecho:
/glade/p/cesmdata/cseg/tools/cime/tools/cprnc/cprnc - on izumi:
/fs/cgd/csm/tools/cime/tools/cprnc/cprnc
"}]}
\ No newline at end of file
diff --git a/site/sitemap.xml.gz b/site/sitemap.xml.gz
index 1eed6191..ab1d30e8 100644
Binary files a/site/sitemap.xml.gz and b/site/sitemap.xml.gz differ
diff --git a/site/stylesheets/extra.css b/site/stylesheets/extra.css
index 6bf886cf..7eacbd87 100644
--- a/site/stylesheets/extra.css
+++ b/site/stylesheets/extra.css
@@ -1,3 +1,3 @@
.md-grid {
- max-width: initial;
+ max-width: 90%
}
diff --git a/site/usage/creating-a-case/index.html b/site/usage/creating-a-case/index.html
new file mode 100644
index 00000000..77a91bd0
--- /dev/null
+++ b/site/usage/creating-a-case/index.html
@@ -0,0 +1,1150 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Creating, configuring, and running a case - CAM-SIMA
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Creating, configuring, and running a case
+Because CAM-SIMA uses the CIME build system, creating and configuration a case is mostly the same as it was in CAM7 (and in CESM). But! We're going to assume you know nothing.
+1. Run create_newcase
+Before you create a case:
+
+- Choose the correct machine to run your case on (if this is a high-resolution or long run, you will likely want to choose derecho!)
+- Consult the git basics if you need a new clone
+- Make sure you have run
bin/git-fleximod update to update and/or populate your externals
+- Navigate to
$CAM-SIMA/cime/scripts
+
+Assuming you have completed the above steps and have a nice checkout of a development branch of CAM-SIMA, let's go!
+Here is the (basic) structure of the command for create_newcase:
+./create_newcase --case <CASEDIR> --compset <COMPSET_NAME> --res <RESOLUTION> --compiler <COMPILER> --project <PROJECT KEY> --run-unsupported
+
+To break that down piece by piece:
+
+--case <CASEDIR> (optional): The case argument specifies the full path to the directory you would like to be created, in which your case will reside.
+- pro tip: if you choose to put your case directory in your scratch directory, your
run and bld directories will be automatically placed within the case directory
+- on derecho, this means:
--case /glade/derecho/scratch/<username>/<case_name>
+- on izumi, this means:
--case /scratch/cluster/<username>/<case_name>
+
+
+- if you do not specify a case directory, it will default to your
$CAM-SIMA/cime/scripts directory (and, eventually, your run and bld directories will default to a new directory in your scratch space)
+
+--compset <COMPSET_NAME> (required): the compset (or "component set") tells CIME which base-level configurations you want to run with (including model versions and input data sets)
+- The compset supplied to the
create_newcase command can be either:
+- The long name of a compset OR
+- The alias (if it exists) for the compset
+
+
+- Compset options and aliases can be found in here (
$CAM-SIMA/cime_config/config_compsets.xml)
+- The structure of a compset long name is
<initialization time>_<atmosphere model>_<land model>_<sea-ice model>_<ocean model>_<river runoff model>_<land ice model>_<wave model>
+- Additional context: for each of the seven model components of the compset configuration, there can be three different implementations:
+- active (prognostic, full): solves a complex set of equations to describe model's behavior (must solve all equations on a numerical grid)
+- data: version that sends/receives same variables to/from other models, but read from files rather than computed from equations (reduces feedback within a system)
+- stub: occupies the required place in the driver but does not send or receive data
+
+
+
+
+
+--res <RESOLUTION> (required): the resolution specifies both the dycore being run and the grid.
+- The structure of the resolution argument is:
<atmosphere>_<ocean>_<mask>
+- The resolution determines the dycore:
+- A grid resolution that looks like "ne*" indicates that this is spectral element (SE) cube sphere grid (SE dycore)
+- A grid resolution that looks like "C*" indicates that this is the FV3 (finite volume cubed sphere) dycore [not yet implemented in CAM-SIMA]
+- A grid resolution that looks like "mpasa*" indicates that this is an MPAS (Model for Prediction Across Scales) grid (MPAS dycore) [work-in-progress in CAM-SIMA]
+
+
+- You can find the options for these model grid aliases in
$CAM-SIMA/ccs_config/modelgrid_aliases_nuopc.xml
+- Most often, for testing CAM-SIMA, we use a coarse SE grid like
ne3pg3_ne3pg3_mg37 or ne5_ne5_mg37
+
+--compiler <COMPILER> (optional): the compiler you wish to use
+- Options:
+- on derecho:
+- intel (default)
+- gnu
+- nvhpc
+
+
+- on izumi:
+- intel (default) - NOTE: intel is not to be trusted on izumi (it's a known-buggy version)
+- gnu
+- nag - NOTE: until the nag version is updated beyond 7.0, CAM-SIMA won't work with nag
+
+
+
+
+
+--project <PROJECT KEY>: a project key to charge to
+- Only needed on derecho
+- You can see the project keys you can use here (click on your primary group)
+- You can also see how much of the allocation has been used/is left by navigating to Reports -> Project Search and searching for the key
+
+
+
+--run-unsupported: this flag is mostly in place to indicate to scientists that the configuration they're choosing to run is not scientifically supported.
+Given all that, let's say you run the following command on izumi (assuming you're this mysterious person "courtneyp"):
+./create_newcase --case /scratch/cluster/courtneyp/kessler-ne5-gnu-0722 --compset FKESSLER --res ne5_ne5_mg37 --compiler gnu --run-unsupported
+
+What will happen is that a new case directory will be created here: /scratch/cluster/courtneyp/kessler-ne5-gnu-0722 that will be configured to run the FKESSLER compset (long name: 2000_CAM%KESSLER_SLND_SICE_SOCN_SROF_ SGLC_SWAV) with the SE dycore and the ne5 grid.
+
+Navigate into your shiny new case directory and run ./case.setup
+From here, there are A LOT of configuration options. We'll highlight a few here that you'll use often.
+XML configurations
+Many configurable settings can be found within the env_*.xml files in your case directory. In order to protect again typos, it's not advised to edit those directly. Instead, you will run ./xmlchange commands to alter those settings (and ./xmlquery can give you the current setting). You can find additional information on the configurable xml variables here. A few to highlight:
+
+CAM_CONFIG_OPTS: This is where we tell CAM-SIMA what physics scheme(s) we wish to run, as well as if we wish to run the null dycore.
+- FKESSLER will default to
--physics-suites kessler --analytic_ic, which means we're running the suite_kessler.xml SDF with the SE dycore (with analytic initial conditions - no need to supply an ncdata initial conditions file)
+- If you instead want to run a test with the null dycore, you'll want to change this with
./xmlchange CAM_CONFIG_OPTS = "--physics-suites kessler --dyn none"
+
+CAM_LINKED_LIBS: Hopefully, you won't have to change this often; however, if you are getting linking errors during your build, try turning off the linked libraries (./xmlchange CAM_LINKED_LIBS='')DOUT_S: This is the flag to archive log files of successful runs. During development, we advise turning this off so your log files don't disappear on you (./xmlchange DOUT_S=False)STOP_OPTION & STOP_N: How long you are going to run the model. If STOP_N is 8 and STOP_OPTION is "ndays", you are setting the model up to run for 8 days.
+- the options for
STOP_OPTION are: 'nsteps', 'nseconds', 'nminutes', 'nhours', 'ndays', 'nmonths', 'nyears'
+STOP_N is an integer- NOTE: if you are running the ncdata_check with snapshot files, keep the number of timesteps you are running at or below the number of timesteps on the snapshot files
+- NOTE #2: These configurations can be updated without rebuilding the case
+
+DEBUG: A flag to turn on debug mode for the run (runs without compiler optimizations) - this is very useful during development! It defaults to "False", so you can turn it on with ./xmlchange DEBUG=TrueRUNDIR: this is the path to the run directory for your case. The bld directory will exist one level up.
+If you run ./xmlchange, the output will tell you if you need to re-set up and/or do a clean build of your case.
+Namelist configurations
+There are countless namelist configuration options. These can be modified by updating the user_nl_cam file in your case directory.
+Namelist options can be updated without rebuilding the case.
+
+3. Build the case
+Run ./case.build
+The console output will tell you the progress of the build. If you get to MODEL BUILD HAS FINISHED SUCCESSFULLY, hooray!
+Debugging tips
+4. Run the case
+Run ./case.submit
+The job will be submitted to the system's queue. You can see the status of your job with the qstat command. Once it is finished, the log files will be in the run directory (unless it ran successfully AND your archiver is on)
+Debugging tips
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+History & model output
+Configuring the namelist
+The syntax for updating a the configuration for a history file is:
+<namelist_option>;<volume>: <setting>
+
+Possible namelist options for your user_nl_cam:
+
+hist_add_avg_fields: add average fields to a specified history filehist_add_inst_fields: add instantaneous fields to a specified history filehist_add_min_fields: add minimum fields to a specified history filehist_add_max_fields: add maximum fields to a specified history filehist_remove_fields: remove a given field from a specified history filehist_file_type: type of file (options are "history", "satellite", and "initial_value") - defaults to "history"hist_max_frames: maximum number of samples written to a specified history file (after which a new one will be created)hist_output_frequency: frequency with which to write samples to a specified history file (syntax is <integer>*<multiplier> like 3*nhours)hist_precision: precision of the specified history file (options are "REAL32" and "REAL64") - defaults to "REAL32"hist_write_nstep0: logical for whether or not to write the nstep = 0 sample (defaults to .false.)hist_filename_template: template filename for the specified history file
+- Defaults to "%c.cam.%u.%y-%m-%d-%s.nc" where "%c" is the case name, "%u" is the volume, "%y" is the year, "%m" is the month, "%d" is the day, and "%s" is the number of seconds since midnight GMT, with the timestamp itself representing the model time when the file is created.
+
+
+Example
+Take the following sample user_nl_cam:
+hist_output_frequency;h1: 5*ndays
+hist_max_frames;h1: 3
+hist_add_inst_fields;h1: U
+hist_add_inst_fields;h1: V, Q
+hist_precision;h1: REAL64
+hist_filename_spec;h1: my-history-file%m-%d
+hist_write_nstep0;h1: .false.
+
+It will be parsed by hist_config.py and this will be the relevant section of atm_in:
+&hist_config_arrays_nl
+ hist_num_inst_fields = 3
+ hist_num_avg_fields = 2
+ hist_num_min_fields = 0
+ hist_num_max_fields = 0
+ hist_num_var_fields = 0
+/
+
+&hist_file_config_nl
+ hist_volume = 'h0'
+ hist_avg_fields = 'T', 'Q'
+ hist_max_frames = 1
+ hist_output_frequency = '1*month'
+ hist_precision = 'REAL32'
+ hist_file_type = 'history'
+ hist_filename_spec = '%c.cam.%u.%y-%m-%d-%s.nc'
+ hist_write_nstep0 = .false.
+/
+
+&hist_file_config_nl
+ hist_volume = 'h1'
+ hist_inst_fields = 'U', ‘V’, ‘Q’
+ hist_max_frames = 3
+ hist_output_frequency = '5*ndays'
+ hist_precision = 'REAL64'
+ hist_file_type = 'history'
+ hist_filename_spec = 'my-history-file%m-%d'
+ hist_write_nstep0 = .false.
+/
+
+In plain English, a one-month run with these history configuration will result in a total of three files that will look something like these:
+
+- my-history-file01-06.nc
+- This file will contain instantaneous output for U, V, and Q (eastward_wind, northward_wind, and water vapor)
+- It will contain three frames, one at each of the following times:
+- 0001-01-06 (time=5)
+- 0001-01-11 (time=10)
+- 0001-01-16 (time=15)
+
+
+
+
+- my-history-file01-21.nc
+- This file will contain instantaneous output for U, V, and Q (eastward_wind, northward_wind, and water vapor)
+- It will contain three frames, one at each of the following times:
+- 0001-01-21 (time=20)
+- 0001-01-26 (time=25)
+- 0001-01-31 (time=30)
+
+
+
+
+- .cam.h0a.0001-02-01-00000.nc
+- This file will contain averaged output for T and Q (air_temperature and water vapor)
+- It will have one frame with the time calculated at the midpoint of the month
+
+
+
+Adding a diagnostic field to the CAM-SIMA source code
+During init time, fields can be added to the possible field list with a call to history_add_field:
+history_add_field(diagnostic_name, standard_name, vdim_name, avgflag, units, gridname, flag_xyfill, mixing_ratio)
+
+
+
+| Field |
+Optional? |
+Type |
+Description |
+
+
+
+
+| diagnostic_name |
+No |
+string |
+diagnostic name for the field - will be the name in netcdf output file |
+
+
+| standard_name |
+No |
+string |
+CCPP standard name for the variable |
+
+
+| vdim_name |
+No |
+string |
+vertical dimension: 'horiz_only' for no vertical dimension; 'lev' for vertical_layer_dimension; 'ilev' for vertical_interface_dimension |
+
+
+| avgflag |
+No |
+string |
+default average flag; options: 'avg', 'lst' (instantaneous), 'min', 'max', 'var' (standard deviation) |
+
+
+| units |
+No |
+string |
+variable units |
+
+
+| gridname |
+Yes |
+string |
+gridname on which the variable's data is mapped (defaults to the physics grid) |
+
+
+| flag_xyfill |
+Yes |
+string |
+fill value for variable values |
+
+
+| mixing_ratio |
+Yes |
+string |
+constituent mixing ratio type ('wet' or 'dry'); not set to anything if not passed in |
+
+
+
+Example:
+call history_add_field('Q', 'water_vapor_mixing_ratio_wrt_moist_air_and_condensed_water', 'lev', 'avg', 'kg kg-1', mixing_ratio='wet')
+
+It's important to avoid adding calls to history_add_field to the CCPP-ized physics schemes (to keep them portable). Instead, create a new diagnostics scheme and place that in the diagnostics folder of the atmospheric_physics repository. The history_add_field call will be in the init phase.
+Outputting a diagnostic field to the CAM-SIMA source code
+After init time, a variable's current values can be captured for output with a call to history_out_field:
+history_out_field(diagnostic_name, field_values)
+
+
+
+| Field |
+Optional? |
+Type |
+Description |
+
+
+
+
+| diagnostic_name |
+No |
+string |
+diagnostic name for the field - will cause an error if the diagnostic name was not added via history_add_field |
+
+
+| field_values |
+No |
+real |
+variable values to be stored in the history buffer(s) |
+
+
+
+Example:
+call history_out_field('Q', const_array(:,:,const_idx))
+
+It's important to avoid adding calls to history_add_field to the CCPP-ized physics schemes (to keep them portable). Instead, create a new diagnostics scheme and place that in the diagnostics folder of the atmospheric_physics repository. The history_out_field call(s) will likely be in the run phase.
+Using the output
+The output files can be found in your run directory. They are in netCDF format.
+See the ADF for lots that you can do with your results!
+A few useful commands for inspecting netCDF data:
+
+- To output the header/metadata information for the file (includes list of variables on the file):
+
+ncdump -h file.nc
+
+
+- To output the data for a specific variable on the file:
+
+ncdump -v <varname> file.nc
+
+
+- To compare two files*:
+
+cprnc <file1_path> <file2_path>
+
+
+- To get a slice of one file (subset of time samples):
+
+ncks -F -d time,<start time>,<end time>,<step> <snapshot_file> <output_file_name>
+
+Example to get time samples 3-5 from a file onto a new file called "split-file.nc":
+ncks -F -d time,3,5,1 file.nc split-file.nc
+
+
+- To diff a variable between two files (and output the results to a new netcdf file):
+
+ncdiff -v <field_name> -o <output_file> <file1> <file2>
+
+*cprnc can be found:
+
+- on derecho:
/glade/p/cesmdata/cseg/tools/cime/tools/cprnc/cprnc
+- on izumi:
/fs/cgd/csm/tools/cime/tools/cprnc/cprnc
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+