Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merra2 data used in control_csawmg test #699

Closed
junwang-noaa opened this issue Jul 20, 2021 · 7 comments · Fixed by #724
Closed

merra2 data used in control_csawmg test #699

junwang-noaa opened this issue Jul 20, 2021 · 7 comments · Fixed by #724
Labels
bug Something isn't working

Comments

@junwang-noaa
Copy link
Collaborator

Description

In the ufs-weather-model develop branch, the control_csawmg test has the option IAER=1111, it was using merra2 data in /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/input-data-20210614/FV3_input_data_INCCN_aeroclim/MERRA2. The test passed on all the UFS supported platforms.

[Jun.Wang@hfe01 NEMSfv3gfs]$ cd input-data-20210614/FV3_input_data_INCCN_aeroclim/MERRA2
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m01.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m02.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m03.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m04.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m05.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m06.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m07.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m08.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m09.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m10.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m11.nc
-rw-rw-r-- 1 emc.nemspara nems 1018900216 Jan 16  2021 merra2.aerclim.2003-2014.m12.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m01.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m02.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m03.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m04.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m05.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m06.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m07.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m08.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m09.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m10.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m11.nc
-rw-rw-r-- 1 emc.nemspara nems   64217932 Jan 16  2021 merra2C.aerclim.2003-2014.m12.nc

Anning updated MERRA2 data and found that the data in input-data-20210614/FV3_input_data_INCCN_aeroclim/MERRA2 causes issues in control_merra2 test (with IAER=1011) and the P7b coupled test. So the latest merra2 data in input-data-20210717/FV3_input_data_INCCN_aeroclim/MERRA2 is now updated to use data in /scratch1/NCEPDEV/global/glopara/fix/fix_aer as Anning suggested.

-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:22 merra2.aerclim.2003-2014.m01.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:22 merra2.aerclim.2003-2014.m02.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m03.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m04.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m05.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m06.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m07.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m08.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m09.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m10.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m11.nc
-rw-rw-r-- 1 emc.nemspara nems 1018901352 Jul  1 15:23 merra2.aerclim.2003-2014.m12.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m01.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m02.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m03.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m04.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m05.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m06.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m07.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m08.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m09.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m10.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m11.nc
-rw-rw-r-- 1 emc.nemspara nems   64218936 Jul  1 15:23 merra2C.aerclim.2003-2014.m12.nc

However using the new merra2 data causes the control_csawmg test to fail on jet with error message:

FATAL from PE   140: NaN in input field of mpp_reproducing_sum(_2d), this indicates numerical instability

To Reproduce:

  1. Checkout the the latest ufs-weather-model develop branch (revision: 22613e8 )
  2. run control_csawmg test on jet

Additional context

The issue showed up in PR#639.
Now we need to figure out what merra2 data the control_csawmg test with (IAER=1111) should use

@junwang-noaa junwang-noaa added the bug Something isn't working label Jul 20, 2021
@junwang-noaa
Copy link
Collaborator Author

@SMoorthi-emc @AnningCheng-NOAA May I ask if the control_csawmg (with IAER=1111) should use the same MERRA2 data as the control_merra2 test (with IAER=1011)? Any additional code changes are required to use the new merra2 data under /scratch1/NCEPDEV/global/glopara/fix/fix_aer (now in input-data-20210717/FV3_input_data_INCCN_aeroclim/MERRA2)? Thanks

@SMoorthi-emc
Copy link
Contributor

SMoorthi-emc commented Jul 20, 2021 via email

@AnningCheng-NOAA
Copy link
Contributor

AnningCheng-NOAA commented Jul 20, 2021 via email

@junwang-noaa
Copy link
Collaborator Author

@AnningCheng-NOAA would you please test the control_csawmg with iaer=1011 and let us know if it is working?

@AnningCheng-NOAA
Copy link
Contributor

AnningCheng-NOAA commented Jul 28, 2021 via email

@junwang-noaa
Copy link
Collaborator Author

Thanks, Anning. I will test the control_csawmg with iaer=1011 on jet to see if it passes.

@junwang-noaa
Copy link
Collaborator Author

@AnningCheng-NOAA @SMoorthi-emc The control_csawmg test passed with iaer=1011 on jet. So I will change iaer to 1011 for this test.

epic-cicd-jenkins pushed a commit that referenced this issue Apr 17, 2023
…ol file issue (#699)

The March 24, 2023 update to the ufs-weather-model (36d6e16), updated UPP and corrected issues associated with the post control file. The ufs-weather-model hash has been updated to 36d6e16 and the UPP hash has been updated to 22cfb88.

The changes required to allow the previous versions of ufs-weather-model and UPP to run the run_post tasks were removed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
3 participants