Update the bufr2ioda yamls

[delippi]

This PR updates the bufr2ioda yaml configurations to address issues identified in the original files. Some of those issues include:

change height data type from int to float

• change unit and/or longName to more accurately reflect the data
• separate multiple subset types (e.g., aircft, aircar, adpsfc, sfcshp, etc.) and create single ioda file for each subset type.
• separate ObsError/wind to ObsError/windEastward and ObsError/windNorthward
• using stationPressure instead of pressure for when referring to surface pressure
• clean up extra white space
• re-organize to make comparisons easier

[ShunLiu-NOAA]

Opps, it is a draft PR...

[ShunLiu-NOAA]

Looks good to me.

[spanNOAA]

After reviewing the YAML files I'm currently using for my FV3JEDI case, I've noticed inconsistencies in the data types for station elevation and height across different files. I usually use integers for surface observations and floats for others. I'm not sure why this is the case, as these settings appear to be derived from the previous YAML examples provided by JCSDA. I will look into this issue further.

[delippi]

After reviewing the YAML files I'm currently using for my FV3JEDI case, I've noticed inconsistencies in the data types for station elevation and height across different files. I usually use integers for surface observations and floats for others. I'm not sure why this is the case, as these settings appear to be derived from the previous YAML examples provided by JCSDA. I will look into this issue further.

@spanNOAA @guoqing-noaa @ShunLiu-NOAA @hu5970 @HuiLiu-NOAA, I know Sijie was going to look into the inconsistencies noted above. But I've reverted heights/stationElevation to integers and don't seem to have an issue at the moment--I can't reproduce the issue I thought I had before. If it comes up we can always just make that change later. Is there anything else anyone would like to see different? Otherwise, I think this PR is ready to go.

[delippi]

Perhaps one thing could be a bash script that can help generate the ioda files using the yamls? Would that be something to add here?

[ShunLiu-NOAA]

Looks good to me.
Let's consider adding a bufr2ioda ctest in another PR with including a bash script to generate the IODA files.

[spanNOAA]

@delippi Have you ever looked at the bufr2ioda outputs? When I review my adpsfc output, I consistently find two values for each variable at the same location and time offset. Do you have any insights on this? Here’s a snippet of the obs:

	latitude	longitude	time offset	pressure	airTemperature
141	24.030001	255.330002	-299.988007	81410.0	NaN
142	24.030001	255.330002	-299.988007	81410.0	290.549988
168	24.549999	278.250000	-1559.988037	101180.0	NaN
169	24.549999	278.250000	-1559.988037	101180.0	301.449982
170	24.549999	278.250000	-839.987976	101210.0	NaN
171	24.549999	278.250000	-839.987976	101210.0	301.449982
172	24.549999	278.250000	-420.011993	101210.0	NaN
173	24.549999	278.250000	-420.011993	101210.0	301.449982
10060	24.556110	278.240570	-420.011993	101150.0	NaN
10061	24.556110	278.240570	-420.011993	101150.0	301.449982
174	24.570000	278.320007	-420.011993	101200.0	NaN
175	24.570000	278.320007	-420.011993	101200.0	299.850006
159	24.629999	252.570007	-299.988007	100870.0	NaN
160	24.629999	252.570007	-299.988007	100870.0	293.149994
176	24.730000	278.950012	-420.011993	101170.0	NaN
177	24.730000	278.950012	-420.011993	101170.0	301.449982
178	25.049999	282.529999	0.000000	101210.0	NaN
179	25.049999	282.529999	0.000000	101210.0	301.149994
180	25.650000	279.570007	-420.011993	101180.0	NaN
181	25.650000	279.570007	-420.011993	101180.0	299.850006

[delippi]

@delippi Have you ever looked at the bufr2ioda outputs? When I review my adpsfc output, I consistently find two values for each variable at the same location and time offset. Do you have any insights on this? Here’s a snippet of the obs:

latitude longitude time offset pressure airTemperature
141 24.030001 255.330002 -299.988007 81410.0 NaN
142 24.030001 255.330002 -299.988007 81410.0 290.549988
168 24.549999 278.250000 -1559.988037 101180.0 NaN
169 24.549999 278.250000 -1559.988037 101180.0 301.449982
170 24.549999 278.250000 -839.987976 101210.0 NaN
171 24.549999 278.250000 -839.987976 101210.0 301.449982
172 24.549999 278.250000 -420.011993 101210.0 NaN
173 24.549999 278.250000 -420.011993 101210.0 301.449982
10060 24.556110 278.240570 -420.011993 101150.0 NaN
10061 24.556110 278.240570 -420.011993 101150.0 301.449982
174 24.570000 278.320007 -420.011993 101200.0 NaN
175 24.570000 278.320007 -420.011993 101200.0 299.850006
159 24.629999 252.570007 -299.988007 100870.0 NaN
160 24.629999 252.570007 -299.988007 100870.0 293.149994
176 24.730000 278.950012 -420.011993 101170.0 NaN
177 24.730000 278.950012 -420.011993 101170.0 301.449982
178 25.049999 282.529999 0.000000 101210.0 NaN
179 25.049999 282.529999 0.000000 101210.0 301.149994
180 25.650000 279.570007 -420.011993 101180.0 NaN
181 25.650000 279.570007 -420.011993 101180.0 299.850006

@spanNOAA, I think that might be because wind is reported separately from the other variables. Check the corresponding ObsType with the NaN's. I bet they start with a 2. Here is a snip from the ioda_adpsfc_dc.nc:

group: ObsType {
  data:
   airTemperature = 287, 187, 287, 187, 287, 187, 287, 187, 287, 187, 287, 


group: ObsValue {
  data:
   airTemperature = _, 303.15, _, 302.15, _, 302.15, _, 302.15, _, 302.15, _, 

[delippi]

I'm trying to follow this https://github.com/JCSDA-internal/ioda/blob/develop/share/ioda/yaml/validation/ObsSpace.yaml (which Praveen provided).