E3SM building properly but when executing it is looking for cesm.exe

[chprsandeep]

Dear Scientist,

I am able to build E3SM successfully by running ./case.build and e3sm.exe file is getting generated.

However, when I run ./cas.submit, the code is looking for cesm.exe. I was able to run the case by just linking ln -sf e3sm.exe cesm.exe. Also, the log file is also generated as cesm.log file.

Could you help me figure out the possible reason for which this is happening?

I have attached the log files.

Thank you.
cesm.log.447643.220519-133726.txt
e3sm.bldlog.220519-165715.txt
lnd.log.447643.220519-133726.txt
run.nst.447643.txt

[rljacob]

What is the output of ./preview_run in your case directory?

[chprsandeep]

The output is as follows:

CASE INFO:
nodes: 1
total tasks: 1
tasks per node: 1
thread count: 1
ngpus per node: 0

BATCH INFO:
FOR JOB: case.run
ENV:
Setting Environment NETCDF_PATH= /home/software/intel/intel-2017.0.1/pkg/netcdf/4
Setting Environment NetCDF_C_PATH=/home/software/intel/intel-2017.0.1/pkg/netcdf/4/include
Setting Environment NetCDF_Fortran_PATH=/home/software/intel/intel-2017.0.1/pkg/netcdf/4/bin
Setting Environment OMP_NUM_THREADS=1

SUBMIT CMD:
  sbatch --time 24:00:00 -p fdr --time 24:00:00 .case.run --resubmit

MPIRUN (job=case.run):
  mpirun  -n 1 /net/fs05/d1/sandeepc/e3sm/cases/nst/bld/cesm.exe   >> cesm.log.$LID 2>&1

[rljacob]

What is the environment variable CIME_MODEL set to? Also which version of the E3SM is this? (git hash)

[chprsandeep]

I have used the following code to download E3SM. I think I am using v2.

git clone [email protected]:E3SM-Project/E3SM.git
cd E3SM
git submodule update --init --recursive

/net/fs05/d1/sandeepc/e3sm/cases/nst/bld/cmake-bld/CMakeCache.txt:27:CIME_MODEL:UNINITIALIZED=e3sm

Could you please let me know where else to check for CIME_model, I have CESM model installed too.

[rljacob]

You may have set it in a .login or .bashrc file. The only thing that makes sense is that CIME_MODEL got redefined between build and submit.

[chprsandeep]

I have checked my .bashrc file. I have not defined an environmental variable with CIME_MODEL. Do I need to change something in the files below?

(base) [sandeepc@svante-login E3SM]$ grep -rnw pwd -e 'CIME_MODEL'
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/.git/modules/cime/objects/pack/pack-cad7726296fd9bd033337cc61da1e74539cb17f1.pack matches
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/.github/workflows/srt.yml:56: CIME_MODEL: cesm
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/.github/workflows/srt_nuopc.yml:36: CIME_MODEL: cesm
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/Tools/Makefile:94: ifneq ($(CIME_MODEL),e3sm)
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/Tools/Makefile:868: ifeq ($(CIME_MODEL),cesm)
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/Tools/Makefile:892:ifeq ($(CIME_MODEL),cesm)
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/XML/machines.py:93: "Could not initialize machine object from {} or {}. This machine is not available for the target CIME_MODEL. The supported CIME_MODELS that can be used are: {}".format(
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/XML/machines.py:138: Return a list of machines defined for a given CIME_MODEL
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/XML/pycache/machines.cpython-38.pyc matches
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/XML/pycache/machines.cpython-39.pyc matches
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/XML/pycache/machines.cpython-36.pyc matches
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/build.py:207: make_args = "CIME_MODEL={} ".format(case.get_value("MODEL"))
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/scripts/configure:47: "from. Overrides the CIME_MODEL environment variable, "
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/scripts/configure:112: "Either --mach-dir or the CIME_MODEL environment "
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/tests/test_unit_case.py:128: @mock.patch.dict(os.environ, {"CIME_MODEL": "cesm"})
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/tests/test_unit_case.py:209: @mock.patch.dict(os.environ, {"CIME_MODEL": "cesm"})
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:217: CIME_MODEL=e3sm,cesm,ufs
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:491: "model {} not recognized. The acceptable values of CIME_MODEL currently are {}".format(
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:495: cime_config.set("main", "CIME_MODEL", model)
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:501: The CIME_MODEL env variable may or may not be set
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:503: >>> os.environ["CIME_MODEL"] = "garbage"
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:508: >>> del os.environ["CIME_MODEL"]
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:518: model = os.environ.get("CIME_MODEL")
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:521: logger.debug("Setting CIME_MODEL={} from environment".format(model))
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:525: "model {} not recognized. The acceptable values of CIME_MODEL currently are {}".format(
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:530: if cime_config.has_option("main", "CIME_MODEL"):
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:531: model = cime_config.get("main", "CIME_MODEL")
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:533: logger.debug("Setting CIME_MODEL={} from ~/.cime/config".format(model))
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:548: # logger.debug("Guessing CIME_MODEL={}, set environment variable if this is incorrect".format(model))
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:556: msg = ".cime/config or environment variable CIME_MODEL must be set to one of: "
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/utils.py:1261: >>> os.environ["CIME_MODEL"] = "e3sm" # Set the test up don't depend on external resources
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/pycache/utils.cpython-38.pyc matches
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/pycache/build.cpython-38.pyc matches
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/pycache/utils.cpython-36.pyc matches
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/pycache/build.cpython-36.pyc matches
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/pycache/utils.cpython-39.pyc matches
Binary file /net/fs05/d1/sandeepc/e3sm/E3SM/cime/CIME/pycache/build.cpython-39.pyc matches
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/ChangeLog:5047: - Fixed and cleaned-up scripts_regression_tests.py test Q_TestBlessTestResults with CIME_MODEL = cesm.
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/ChangeLog:5576: CIME_MODEL=cesm; CIME_DRIVER=nuopc; ./scripts_regression_tests.py
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/ChangeLog:6278: - Latest developments from the CIME_MODEL=ufs project.
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/ChangeLog:8303: - Avoid an error message when CIME_MODEL is cesm - CAM_TARGET is not defined.
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/doc/source/users_guide/cime-config.rst:15: * CIME_MODEL=[e3sm, cesm]
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/doc/source/users_guide/cime-config.rst:42: CIME_MODEL=cesm
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/doc/source/users_guide/grids.rst:199: You will need to modify $CIMEROOT/config/cesm/config_grids.xml or $CIMEROOT/config/e3sm/config_grids.xml depending on the value of $CIME_MODEL.
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/doc/source/users_guide/introduction-and-overview.rst:101:Before using any CIME commands, set the CIME_MODEL environment variable. In bash, use export as shown and replace
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/doc/source/users_guide/introduction-and-overview.rst:105: > export CIME_MODEL=
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/scripts/fortran_unit_testing/run_tests.py:63: "from. Overrides the CIME_MODEL environment variable, "
/net/fs05/d1/sandeepc/e3sm/E3SM/cime/tools/Readme.configure:26: Overrides the CIME_MODEL environment variable, and
/net/fs05/d1/sandeepc/e3sm/E3SM/cime_config/machines/orgnl/config_workflow.xml:11: in the config/$CIME_MODEL/machines directory.
/net/fs05/d1/sandeepc/e3sm/E3SM/cime_config/machines/config_workflow.xml:11: in the config/$CIME_MODEL/machines directory.
/net/fs05/d1/sandeepc/e3sm/E3SM/components/cmake/build_model.cmake:251: set(TARGET_NAME "${CIME_MODEL}.exe")
/net/fs05/d1/sandeepc/e3sm/E3SM/components/cmake/common_setup.cmake:70: if (CIME_MODEL STREQUAL "cesm")
/net/fs05/d1/sandeepc/e3sm/E3SM/driver-mct/cime_config/namelist_definition_drv.xml:280:
/net/fs05/d1/sandeepc/e3sm/E3SM/driver-moab/cime_config/namelist_definition_drv.xml:248:

[rljacob]

No those are fine.

[chprsandeep]

I am able to identify the issue. In config_machines.xml, the following code is present:

<default_run_suffix>
<default_run_exe>${EXEROOT}/cesm.exe </default_run_exe>
<default_run_misc_suffix> >> cesm.log.$LID 2>&1 </default_run_misc_suffix>
</default_run_suffix>

The config_machines.xml I was using was copied from the CESM model, hence cesm. I replaced cesm with e3sm in the above code, and it is working now.

Thank you.