SCM with DEPHY format

[hertneky]

I am having some issues with running an SCM case using the DEPHY format and was hoping you could enlighten me as to what I am doing wrong. This is my first time working with the DEPHY format, so I don't have anything to compare to.

The issue is related to the surface flux variable. I have set the global attribute to 'surfaceForcing = "ts"', since that is the data I have and 'ts' is included in the forging file, but I get an error:

File "./run_scm.py", line 291, in setup_rundir
if surface_flux_spec:
UnboundLocalError: local variable 'surface_flux_spec' referenced before assignment

Looking through the scm source code - it's not clear what I have wrong. I am working on Hera and the processed file can be found here: /scratch1/BMC/gmtb/Tracy.Hertneky/scm/ccpp-scm/scm/data/processed_case_input/MAGIC_20130720_SCM_driver.nc
The namelist here: /scratch1/BMC/gmtb/Tracy.Hertneky/scm/ccpp-scm/scm/etc/case_config/MAGIC_20130720.nml

[grantfirl]

Thanks for reporting @hertneky. I'll have a look on Hera and report back.

[grantfirl]

Tracy,

There is an error in the run script where "surface_flux_spec" is not being set when using the DEPHY format. You can get past this error by adding a couple of lines in scm/src/run_scm.py. Look for

if (surfaceForcing.lower() == 'flux' or surfaceForcing.lower() == 'surfaceflux'):
surface_flux_spec = True

around line 280 and add

else:
surface_flux_spec = False

This is a bug and I'll fix it in main too. After getting past this, in the SCM executable, there are further error(s) when trying to read the MAGIC_20130720_SCM_driver.nc file. Do you have a script somewhere where this file was generated? If so, I could play around with it until the file can be read error-free.

[hertneky]

This did fix that specific issue. I am not surprised that there are other errors as this is new territory for me.

[grantfirl]

OK, let me know if you need any further help getting through the remaining issues. The next one that I ran into after fixing the original issue had to do with the start/end dates not being the right type and format.

[hertneky]

Thanks Grant - I did not get that same error - perhaps you can take a look at this point now that I have tried debugging it myself.

The current one I am struggling with is " NetCDF: Attempt to convert between text & numbers". This isn't a very helpful error for me. I commented all Attributes out and added each one back as required. I did get this NETCDF error when I added the startDate Attribute, but I don't see what is wrong with it, or if it is hung up on something past that.

My data file is here: /scratch1/BMC/gmtb/Tracy.Hertneky/scm/ccpp-scm/scm/data/processed_case_input/MAGIC_20130720_SCM_driver.nc
My script is here: /scratch1/BMC/gmtb/Tracy.Hertneky/scm/scripts/ncl/MAGIC_write_dephy_netcdf.ncl

Note: There is some questionable data (e.g. some extremely strong near-sfc winds, high pressure values in the 1st couple levels, etc) I will address those after I get the SCM to read the file.

[grantfirl]

I should also say that right now, the SCM code was written for DEPHY v0. They have moved on to v1, which now breaks the SCM DEPHY functionality. It shouldn't be too hard to fix it, but I don't think it is a very high priority right now. Was your MAGIC*SCM_driver.nc file created assuming DEPHY v0 or v1?

[hertneky]

I was following the document you pointed me to awhile back. It looks like it is v1.

[grantfirl]

OK. It looks like what the DEPHY folks call "v1" has changed pretty dramatically in the past year or so. What the SCM was coded to use was apparently an early version of "v1" when the format was first being discussed. Here is the document that I coded to: https://docs.google.com/document/d/118xP04jB9HO7Y2LqWk3HZpZ9n3CFujgzimLI7Ug8vO4/edit?pli=1. Here is the latest standard on their GitHub page: https://github.com/GdR-DEPHY/DEPHY-SCM/blob/master/DEPHY_Format_V1.pdf. There are a lot of subtle differences in naming, etc. that have changed such that the SCM code will no longer read in current "v1" standard files in the DEPHY repo. If you create a file that corresponds to the first document, it should still work though.

This shouldn't be that hard to fix/update since the underlying ideas are the same, but I've been told not to make this a priority right now. When @ligiabernardet gets back, perhaps a way forward can be discussed.

[hertneky]

@grantfirl You mentioned a DEPHY repo. If there are any DEPHY files in that repo (that work with the SCM) or if you have a sample SCM DEPHY file, I can take a look at it and compare to what I have done too.