Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[production/RRFS.v1] The final smoke update PR to RRFS.v1 #2298

Merged
merged 13 commits into from
Aug 6, 2024
Merged

[production/RRFS.v1] The final smoke update PR to RRFS.v1 #2298

merged 13 commits into from
Aug 6, 2024

Conversation

haiqinli
Copy link
Contributor

Commit Queue Requirements:

  • Fill out all sections of this template.
  • All sub component pull requests have been reviewed by their code managers.
  • Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
  • Commit 'test_changes.list' from previous step

Description:

Commit Message:

* UFSWM - 
 
  * FV3 - 
    * ccpp-physics - 


### Priority:
<!--
Please provide the priority you would prefer this pull request to have.

* High: Time-sensitive project.

Please delete the ones that are not applicable
-->

* High: For the RRFS.v1.


## Git Tracking
### UFSWM:

### Sub component Pull Requests:
<!--
Please provide a list of sub-components involved with this pull request.
Please provide links to the sub-component pull requests as shown below.
Please delete what is not needed.

* FV3: NOAA-EMC/fv3atm#841
  * ccpp-physics: ufs-community/ccpp-physics#211


### UFSWM Blocking Dependencies:


---
## Changes
### Regression Test Changes (Please commit test_changes.list):
<!--
Please let us know if this PR creates new baselines, changes baselines or not.
Please delete what is not needed.
Please make sure you have properly submitted test_changes.list
-->

* PR Updates/Changes Baselines.


### Input data Changes:
<!--
If there are any changes to input-data for a test, please provide information here.
Please delete what is not needed.
-->
* None.


### Library Changes/Upgrades:


  
---
<!-- STOP!!! THE FOLLOWING IS FOR CODE MANAGERS ONLY. PLEASE DO NOT FILL OUT -->
## Testing Log:
- RDHPCS
  - [ x] Hera
  - [ ] Orion
  - [ ] Hercules
  - [ ] Jet
  - [ ] Gaea
  - [ ] Derecho
- WCOSS2
  - [ ] Dogwood/Cactus
  - [ ] Acorn
- [ ] CI
- [ ] opnReqTest (complete task if unnecessary)

@jkbk2004
Copy link
Collaborator

@MatthewPyle-NOAA I am going to create new baseline on hera/hercules.

@MatthewPyle-NOAA MatthewPyle-NOAA changed the title The final physics update PR to RRFS.v1 [production/RRFS.v1] The final physics update PR to RRFS.v1 May 30, 2024
@MatthewPyle-NOAA
Copy link
Collaborator

@jkbk2004 I will do the same on WCOSS

@grantfirl
Copy link
Collaborator

@MatthewPyle-NOAA @jkbk2004 @haiqinli is on leave until next Tuesday.

@jkbk2004
Copy link
Collaborator

@MatthewPyle-NOAA @jkbk2004 @haiqinli is on leave until next Tuesday.

@grantfirl thanks for the note!

@MatthewPyle-NOAA
Copy link
Collaborator

Not sure what the issue is, but seeing errors like this one in a few tests on WCOSS (such as conus13km_debug_intel)

forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, position 20
Image              PC                Routine            Line        Source
fv3.exe            000000000BD1336B  Unknown               Unknown  Unknown
fv3.exe            000000000BD4E779  Unknown               Unknown  Unknown
fv3.exe            000000000BD4C830  Unknown               Unknown  Unknown
fv3.exe            000000000646DBC1  gfs_typedefs_mp_c        4465  GFS_typedefs.F90
fv3.exe            0000000006220886  gfs_init_mp_gfs_i          68  GFS_init.F90
fv3.exe            00000000021F6405  atmos_model_mod_m         709  atmos_model.F90
fv3.exe            0000000001C242E4  module_fcst_grid_         787  module_fcst_grid_comp.F90
fv3.exe            0000000000E2A194  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
fv3.exe            0000000000E2DDCA  ESMCI_FTableCallE         824  ESMCI_FTable.C
fv3.exe            000000000142E95B  _ZN5ESMCI3VMK5ent        1125  ESMCI_VMKernel.C
fv3.exe            0000000001417DD9  _ZN5ESMCI2VM5ente        1216  ESMCI_VM.C
fv3.exe            0000000000E2B5D7  c_esmc_ftablecall         981  ESMCI_FTable.C
fv3.exe            0000000000A0A710  esmf_compmod_mp_e        1223  ESMF_Comp.F90
fv3.exe            00000000008110B6  esmf_gridcompmod_        1412  ESMF_GridComp.F90
fv3.exe            0000000001BDB3A2  fv3atm_cap_mod_mp         393  fv3_cap.F90
fv3.exe            0000000000C79A4F  _ZNK5ESMCI13Metho         377  ESMCI_MethodTable.C
fv3.exe            0000000000C799CD  _ZN5ESMCI11Method         563  ESMCI_MethodTable.C
fv3.exe            0000000000C7A381  c_esmc_methodtabl         347  ESMCI_MethodTable.C
fv3.exe            0000000000B7496B  esmf_attachmethod        1280  ESMF_AttachMethods.F90
fv3.exe            0000000005081068  nuopc_modelbase_m         687  NUOPC_ModelBase.F90
fv3.exe            0000000000E2A194  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
fv3.exe            0000000000E2DDCA  ESMCI_FTableCallE         824forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, position 20

@BrianCurtis-NOAA
Copy link
Collaborator

it doesn't like a variable in input.nml. Did you change any of them recently?

@MatthewPyle-NOAA
Copy link
Collaborator

@haiqinli Is it ready to test again?

@haiqinli
Copy link
Contributor Author

haiqinli commented Jun 5, 2024

Not sure what the issue is, but seeing errors like this one in a few tests on WCOSS (such as conus13km_debug_intel)

forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, position 20
Image              PC                Routine            Line        Source
fv3.exe            000000000BD1336B  Unknown               Unknown  Unknown
fv3.exe            000000000BD4E779  Unknown               Unknown  Unknown
fv3.exe            000000000BD4C830  Unknown               Unknown  Unknown
fv3.exe            000000000646DBC1  gfs_typedefs_mp_c        4465  GFS_typedefs.F90
fv3.exe            0000000006220886  gfs_init_mp_gfs_i          68  GFS_init.F90
fv3.exe            00000000021F6405  atmos_model_mod_m         709  atmos_model.F90
fv3.exe            0000000001C242E4  module_fcst_grid_         787  module_fcst_grid_comp.F90
fv3.exe            0000000000E2A194  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
fv3.exe            0000000000E2DDCA  ESMCI_FTableCallE         824  ESMCI_FTable.C
fv3.exe            000000000142E95B  _ZN5ESMCI3VMK5ent        1125  ESMCI_VMKernel.C
fv3.exe            0000000001417DD9  _ZN5ESMCI2VM5ente        1216  ESMCI_VM.C
fv3.exe            0000000000E2B5D7  c_esmc_ftablecall         981  ESMCI_FTable.C
fv3.exe            0000000000A0A710  esmf_compmod_mp_e        1223  ESMF_Comp.F90
fv3.exe            00000000008110B6  esmf_gridcompmod_        1412  ESMF_GridComp.F90
fv3.exe            0000000001BDB3A2  fv3atm_cap_mod_mp         393  fv3_cap.F90
fv3.exe            0000000000C79A4F  _ZNK5ESMCI13Metho         377  ESMCI_MethodTable.C
fv3.exe            0000000000C799CD  _ZN5ESMCI11Method         563  ESMCI_MethodTable.C
fv3.exe            0000000000C7A381  c_esmc_methodtabl         347  ESMCI_MethodTable.C
fv3.exe            0000000000B7496B  esmf_attachmethod        1280  ESMF_AttachMethods.F90
fv3.exe            0000000005081068  nuopc_modelbase_m         687  NUOPC_ModelBase.F90
fv3.exe            0000000000E2A194  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
fv3.exe            0000000000E2DDCA  ESMCI_FTableCallE         824forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, position 20

@MatthewPyle-NOAA Yes, there is a namelist change fo smoke_forecast to hwp_method. We can simply delete the smoke_forecast in the input.nml of RT cases, and it worked well after deleting the smoke_forecast in /scratch2/BMC/acomp/Haiqin.Li/stmp4/Haiqin.Li/FV3_RT/rt_3775248/conus13km_debug_intel.

@haiqinli
Copy link
Contributor Author

haiqinli commented Jun 5, 2024

@haiqinli Is it ready to test again?

@MatthewPyle-NOAA Yes, it is ready to test again now. Thanks.

@jkbk2004
Copy link
Collaborator

jkbk2004 commented Jun 7, 2024

Not sure what the issue is, but seeing errors like this one in a few tests on WCOSS (such as conus13km_debug_intel)

forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, position 20
Image              PC                Routine            Line        Source
fv3.exe            000000000BD1336B  Unknown               Unknown  Unknown
fv3.exe            000000000BD4E779  Unknown               Unknown  Unknown
fv3.exe            000000000BD4C830  Unknown               Unknown  Unknown
fv3.exe            000000000646DBC1  gfs_typedefs_mp_c        4465  GFS_typedefs.F90
fv3.exe            0000000006220886  gfs_init_mp_gfs_i          68  GFS_init.F90
fv3.exe            00000000021F6405  atmos_model_mod_m         709  atmos_model.F90
fv3.exe            0000000001C242E4  module_fcst_grid_         787  module_fcst_grid_comp.F90
fv3.exe            0000000000E2A194  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
fv3.exe            0000000000E2DDCA  ESMCI_FTableCallE         824  ESMCI_FTable.C
fv3.exe            000000000142E95B  _ZN5ESMCI3VMK5ent        1125  ESMCI_VMKernel.C
fv3.exe            0000000001417DD9  _ZN5ESMCI2VM5ente        1216  ESMCI_VM.C
fv3.exe            0000000000E2B5D7  c_esmc_ftablecall         981  ESMCI_FTable.C
fv3.exe            0000000000A0A710  esmf_compmod_mp_e        1223  ESMF_Comp.F90
fv3.exe            00000000008110B6  esmf_gridcompmod_        1412  ESMF_GridComp.F90
fv3.exe            0000000001BDB3A2  fv3atm_cap_mod_mp         393  fv3_cap.F90
fv3.exe            0000000000C79A4F  _ZNK5ESMCI13Metho         377  ESMCI_MethodTable.C
fv3.exe            0000000000C799CD  _ZN5ESMCI11Method         563  ESMCI_MethodTable.C
fv3.exe            0000000000C7A381  c_esmc_methodtabl         347  ESMCI_MethodTable.C
fv3.exe            0000000000B7496B  esmf_attachmethod        1280  ESMF_AttachMethods.F90
fv3.exe            0000000005081068  nuopc_modelbase_m         687  NUOPC_ModelBase.F90
fv3.exe            0000000000E2A194  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
fv3.exe            0000000000E2DDCA  ESMCI_FTableCallE         824forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, position 20

@MatthewPyle-NOAA Yes, there is a namelist change fo smoke_forecast to hwp_method. We can simply delete the smoke_forecast in the input.nml of RT cases, and it worked well after deleting the smoke_forecast in /scratch2/BMC/acomp/Haiqin.Li/stmp4/Haiqin.Li/FV3_RT/rt_3775248/conus13km_debug_intel.

@haiqinli @MatthewPyle-NOAA what about https://github.com/haiqinli/ufs-weather-model/blob/production/RRFS.v1-final/tests/parm/rrfs_conus13km_hrrr.nml.IN#L152 ? I mean to address the failed issue.

@haiqinli
Copy link
Contributor Author

haiqinli commented Jun 7, 2024

Not sure what the issue is, but seeing errors like this one in a few tests on WCOSS (such as conus13km_debug_intel)

forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, position 20
Image              PC                Routine            Line        Source
fv3.exe            000000000BD1336B  Unknown               Unknown  Unknown
fv3.exe            000000000BD4E779  Unknown               Unknown  Unknown
fv3.exe            000000000BD4C830  Unknown               Unknown  Unknown
fv3.exe            000000000646DBC1  gfs_typedefs_mp_c        4465  GFS_typedefs.F90
fv3.exe            0000000006220886  gfs_init_mp_gfs_i          68  GFS_init.F90
fv3.exe            00000000021F6405  atmos_model_mod_m         709  atmos_model.F90
fv3.exe            0000000001C242E4  module_fcst_grid_         787  module_fcst_grid_comp.F90
fv3.exe            0000000000E2A194  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
fv3.exe            0000000000E2DDCA  ESMCI_FTableCallE         824  ESMCI_FTable.C
fv3.exe            000000000142E95B  _ZN5ESMCI3VMK5ent        1125  ESMCI_VMKernel.C
fv3.exe            0000000001417DD9  _ZN5ESMCI2VM5ente        1216  ESMCI_VM.C
fv3.exe            0000000000E2B5D7  c_esmc_ftablecall         981  ESMCI_FTable.C
fv3.exe            0000000000A0A710  esmf_compmod_mp_e        1223  ESMF_Comp.F90
fv3.exe            00000000008110B6  esmf_gridcompmod_        1412  ESMF_GridComp.F90
fv3.exe            0000000001BDB3A2  fv3atm_cap_mod_mp         393  fv3_cap.F90
fv3.exe            0000000000C79A4F  _ZNK5ESMCI13Metho         377  ESMCI_MethodTable.C
fv3.exe            0000000000C799CD  _ZN5ESMCI11Method         563  ESMCI_MethodTable.C
fv3.exe            0000000000C7A381  c_esmc_methodtabl         347  ESMCI_MethodTable.C
fv3.exe            0000000000B7496B  esmf_attachmethod        1280  ESMF_AttachMethods.F90
fv3.exe            0000000005081068  nuopc_modelbase_m         687  NUOPC_ModelBase.F90
fv3.exe            0000000000E2A194  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
fv3.exe            0000000000E2DDCA  ESMCI_FTableCallE         824forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, position 20

@MatthewPyle-NOAA Yes, there is a namelist change fo smoke_forecast to hwp_method. We can simply delete the smoke_forecast in the input.nml of RT cases, and it worked well after deleting the smoke_forecast in /scratch2/BMC/acomp/Haiqin.Li/stmp4/Haiqin.Li/FV3_RT/rt_3775248/conus13km_debug_intel.

@haiqinli @MatthewPyle-NOAA what about https://github.com/haiqinli/ufs-weather-model/blob/production/RRFS.v1-final/tests/parm/rrfs_conus13km_hrrr.nml.IN#L152 ? I mean to address the failed issue.

@jkbk2004 Yes, "smoke_forecast" in that file should be changed to "hwp_method". I will update it soon. Thanks.

@MatthewPyle-NOAA
Copy link
Collaborator

@haiqinli Did you have a chance to update it?

@haiqinli
Copy link
Contributor Author

@MatthewPyle-NOAA Ravan will give me some smoke updates tomorrow, and I will update this PR tomorrow. Thanks.

@haiqinli
Copy link
Contributor Author

@MatthewPyle-NOAA @grantfirl @jkbk2004 Per our discussion with Ming and Curtis, we will split this PR into two seperate PRs. The new PR only includes the FMS and minor RUC LSM deep soil temperature updates, and this new PR could be merged into RRFS.v1 easily. The smoke updates will be kept in the current PR. However, we need to do some retro runs to guarantee there will be very minor impacts on the meteorology forecast, and may take more time. Thanks.

@haiqinli haiqinli changed the title [production/RRFS.v1] The final physics update PR to RRFS.v1 [production/RRFS.v1] The final smoke update PR to RRFS.v1 Jun 20, 2024
@jkbk2004
Copy link
Collaborator

@MatthewPyle-NOAA @grantfirl @jkbk2004 Per our discussion with Ming and Curtis, we will split this PR into two seperate PRs. The new PR only includes the FMS and minor RUC LSM deep soil temperature updates, and this new PR could be merged into RRFS.v1 easily. The smoke updates will be kept in the current PR. However, we need to do some retro runs to guarantee there will be very minor impacts on the meteorology forecast, and may take more time. Thanks.

@haiqinli thanks for the update!

@haiqinli
Copy link
Contributor Author

@MatthewPyle-NOAA Should we move this PR (the smoke updates) forward? Thanks.

@MatthewPyle-NOAA
Copy link
Collaborator

@haiqinli If it has been confirmed that the impact on meteorological forecasts is small, I'd say sure.

@haiqinli
Copy link
Contributor Author

@haiqinli If it has been confirmed that the impact on meteorological forecasts is small, I'd say sure.

@MatthewPyle-NOAA
t1
t2
t3
t4

We conducted sensitivity simulations with our smoke PR for the RRFS-A. We've used the input and boundary conditions from the real-time RRFS_A system in these simulations. The main goals of this PR are the following:

  • to fix a bug affecting diurnal cycle of agriculture fire emissions
  • to add a new namelist option to scale the fire emissions (based on NESDIS's RAVE satellite fire dataset), when we switch to RAVEv2. The current operational RAVE data (v1.3) ingested into RRFS_A has a major bug, which leads to overestimation of fire emissions. We'll need to apply a scaling factor to the RAVE data, when we switch to ingesting RAVEV2. NOAA's current operational AQM model (UFS-CMAQ) is also using the RAVE data, and is going to update the AQM to ingest the RAVE v2 data sometime late summer.

The figures are the averaged temperature and wind speed differences (averaged over the CONUS domain) between the simulations with the current RRFS_A code and our changes. As we expected, the differences are negligible. The reason for this is that the smoke impact on meteorology is local, in areas with dense smoke. Usually agricultural fires don't emit tons of smoke. ((This PR is revised to remove the microphysics changes).

@MatthewPyle-NOAA
Copy link
Collaborator

@haiqinli Can you clean up the conflicts so we can look to move this forward?

@haiqinli
Copy link
Contributor Author

@MatthewPyle-NOAA Sure, I will sync the code with the authoritative repository and clean up the conflicts.

@MatthewPyle-NOAA
Copy link
Collaborator

@haiqinli Any idea when this will be ready to move forward?

@haiqinli
Copy link
Contributor Author

@MatthewPyle-NOAA Yes, the PRs for ccpp-physics, fv3atm, and ufs-weather-model are updated, and are ready to go. Thanks.

@jkbk2004
Copy link
Collaborator

@haiqinli I see some conus13km cases crash.

 27: forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, p
osition 20
 27: Image              PC                Routine            Line        Source
 27: fv3.exe            000000000399ADC8  Unknown               Unknown  Unknown
 27: fv3.exe            00000000039D6ED7  Unknown               Unknown  Unknown
 27: fv3.exe            00000000039D4EB0  Unknown               Unknown  Unknown
 27: fv3.exe            0000000002D37651  gfs_typedefs_mp_c        4465  GFS_typedefs.F90

@haiqinli
Copy link
Contributor Author

@haiqinli I see some conus13km cases crash.

 27: forrtl: severe (19): invalid reference to variable in NAMELIST input, unit -5, file Internal Formatted NML Read, line -1, p
osition 20
 27: Image              PC                Routine            Line        Source
 27: fv3.exe            000000000399ADC8  Unknown               Unknown  Unknown
 27: fv3.exe            00000000039D6ED7  Unknown               Unknown  Unknown
 27: fv3.exe            00000000039D4EB0  Unknown               Unknown  Unknown
 27: fv3.exe            0000000002D37651  gfs_typedefs_mp_c        4465  GFS_typedefs.F90

@jkbk2004 An input namelist variable of "smoke_forecast" is changed to "hwp_method". Or we can just simply remove "smoke_forecast" from the input.nml for regression test. Thanks.

@MatthewPyle-NOAA
Copy link
Collaborator

@jkbk2004 Thanks for updating the namelist template - looks good for me on WCOSS2 now.

@jkbk2004
Copy link
Collaborator

jkbk2004 commented Aug 2, 2024

@MatthewPyle-NOAA We have Derecho resource issue. As Hera/Hercules/Wcoss2 tests are done, we can start merging this PR. We can merge physics pr ufs-community/ccpp-physics#211

@jkbk2004 jkbk2004 mentioned this pull request Aug 2, 2024
Merged
@jkbk2004
Copy link
Collaborator

jkbk2004 commented Aug 6, 2024

@MatthewPyle-NOAA you can merge this pr with cleanup when you squash merge and add commit message.

@MatthewPyle-NOAA MatthewPyle-NOAA merged commit ce43a6f into ufs-community:production/RRFS.v1 Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MatthewPyle-NOAA MatthewPyle-NOAA Awaiting requested review from MatthewPyle-NOAA

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@haiqinli @jkbk2004 @MatthewPyle-NOAA @grantfirl @BrianCurtis-NOAA