Enable PIO to Add Attributes to Existing NetCDF Files #1234
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit removes some debugging log writes around calls to sfclayrev in the driver_sfclayer subroutine. These writes led to messages like the following appearing in the log file for every model timestep: --- enter subroutine sfclayrev: --- end subroutine sfclayrev: --- enter subroutine sfclayrev seaice: --- end subroutine sfclayrev seaice:
…) arrays This commit adds a new routine to the dmpar module for broadcasing real(kind=R4KIND) arrays. The new mpas_dmpar_bcast_real4s routine mirrors the mpas_dmpar_bcast_reals routine, with the dummy array argument being of R4KIND rather than RKIND kind.
…sics.F The mpas_atmphys_init_microphysics module contains a routine, table_ccnAct, that broadcasts an R4KIND real array, tnccn_act, from the Thompson microphysics scheme. This broadcast was previously performed by a call to mpas_dmpar_bcast_reals through the macro DM_BCAST_MACRO. However, because the mpas_dmpar_bcast_reals routine broadcasts arrays of kind RKIND, when MPAS-A is compiled with double-precision reals, the mpas_dmpar_bcast_reals routine can no longer be used with the tnccn_act array. With this commit, the DM_BCAST_MACRO in mpas_atmphys_init_microphysics.F has been defined to mpas_dmpar_bcast_real4s, which broadcasts arrays of R4KIND reals. This works for both single- and double-precision builds of MPAS-A, since the kind type of the tnccn_act array is independent of the MPAS-A build precision.
This commit restores CMake support for the atmosphere core, which broke due to changes made prior to the version 8.2 release. The breaking changes included moving the physics_mmm directory to its own git repository. This is accounted for in CMake by using FetchContent to clone the correct version of the physics_mmm source code directory based on the MPAS model git version. Additionally, this commit builds the noahmp library that was added to the code base. Previously, this library only supported being built with a hardcoded GNU Makefile. This commit also accounts for the new registry preprocessing arguments that make the registry processing core-agnostic. These changes re-enable a seamless build process for the atmosphere core and improve modularity and maintainability.
This commit restores CMake support for the atmosphere core, which broke due to changes made prior to the version 8.2 release. The breaking changes were a result of the addition of mpas_init_atm_thompson_aerosols.F to the init_atmosphere core without updating the CMakeLists.txt. To fix this problem, this file was appended to the list of source files in the init_atmosphere core's CMakeLists.txt.
…l instances when spaces were used instead of tabs (lines 41-43, and lines 104, 108, 111, and 114.)
MPI implementations not compiled with mpi_f08 enabled still pass the check. Using OpenMPI 4.0.0 and gfortran 11.4.1 compiling without mpi_f08 features the MPI wrapper will still link and supply mpi_f08 (-lmpi_usempif08) despite not being valid or usable. This causes the `use mpi_f08` and some features to compile fine, but other mpi_f08 features to not exist and fail compilation.
This follows the same logic as before with the added check that a libpnetcdf.* file exists in the checked directory. The change in logic will now account for situations where PNETCDF is set to a directory containing both lib/ and lib64/ directories and libpnetcdf.* exists in just the lib/ folder. Should libpnetcdf.* exist in both directories, the lib64/ still maintains precedence.
As the netcdf library logic mirrors the pnetcdf one, it would suffer from the same issues noted in 0ca4208 about lib/lib64 checking. This updates the corresponding issue in the netcdf library detection before it becomes an issue.
…ize.o Add the object dependencies in the makefile for mpas_atmphys_lsm_noahmpfinalize.o specific to the objects created under core_physics, i.e. any use statement that reference modules compiled as part of the core_physics target. Modules originating from NoahMP and framework are ignored as they should be handled separately before the core_physics target is started.
When building MPAS-A as a dycore, all physics-related components are disabled. An example scenario of this is for use with CAM/CAM-SIMA. However, the MPAS-A registry file in version 8.2.0 contains the following regressions, which cause the dycore-only build to break: * The MPAS-A registry file introduced a new stream, `da_state`, which unconditionally includes variables that exist only when physics are enabled. * The MPAS-A registry file unconditionally includes a new registry file from Noah-MP, which is applicable only when physics are enabled. Fix MPAS-A dycore-only build by guarding the above regions with the `DO_PHYSICS` macro.
…F, corrected the calculation of the height Above Ground Level (AGL) prior to computing radar in subroutine compute_radar_reflectivity.
-> removed package bl_mynn_in from packages included for nc and tend_nc. the
array declaration for nc, nifa, nwfa, tend_nc, tend_nifa, and tend_nwfa
now includes the same packages.
-> in ./physics/mpas_atmphys_interface.F, added arrays nifa_p, and nwfa_p for
the PBL option "bl_mynn".
-> in ./physics/mpas_atmphys_todynamics.F, only updates tend_scalars(index_nc,
tend_scalars(index_nifa, and tend_scalars(index_nwfa due to PBL processes
when the microp_scheme option is "mp_thompson_aerosols".
These changes are needed to run the MYNN PBL scheme without the aerosol-aware
option of the Thompson cloud microphysics scheme.
…anged print statements "enter/end subroutine init_atm_gocart" with "enter/end subroutine init_atm_thompson_aerosols.
…, removed extra deallocate for variable tao5503d_p in subroutine deallocate_radiation_sw.F.
- Adds support for adding new attributes to existing NetCDF files by minimizing expensive mode switches between data and define modes. - Introduces `put_att_pio` interface with try-fail logic, handling scalar and 1D attributes of various data types (int, real, double, string). - Enhances performance by avoiding unnecessary transitions and includes extensive logging for better traceability. - Ensures backward compatibility for NetCDF files generated by earlier MPAS versions.
amstokely
force-pushed
the
framework/pio_put_att
branch
from
228b58c
to
7b75c62
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add Ability to Write New Attributes to Existing NetCDF Files in PIO
Overview:
Currently, PIO in MPAS lacks the ability to add new attributes to existing NetCDF files. This limitation arises because when PIO attempts to write a new attribute, the file remains in data mode, which does not allow for the addition of attributes. Instead, attributes can only be added when the file is in define mode.
Unfortunately, the public APIs of both PIO and NetCDF do not provide a way to inquire about a file’s current mode. Switching to define mode each time a new attribute is needed is expensive, as transitioning between data and define modes introduces significant I/O costs.