Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

resolve inconsistencies between bufr2ioda python converters and jcb templates #1241

Merged
merged 10 commits into from
Aug 12, 2024
Merged

resolve inconsistencies between bufr2ioda python converters and jcb templates #1241

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
26e4464
resolve inconsistencies between bufr2ioda python converters and jcb t…
RussTreadon-NOAA Jul 31, 2024
412307b
Merge branch 'develop' into feature/bufr2ioda
RussTreadon-NOAA Aug 4, 2024
d5ef559
Merge branch 'NOAA-EMC:develop' into feature/bufr2ioda
RussTreadon-NOAA Aug 7, 2024
1466e09
add tm00 to adp prepbufr bufr2ioda converters, use nc instead of nc4 …
RussTreadon-NOAA Aug 7, 2024
4e43edd
update parm/jcb-gdas hash (#1227)
RussTreadon-NOAA Aug 7, 2024
182086d
Merge branch 'develop' into feature/bufr2ioda
RussTreadon-NOAA Aug 8, 2024
7fef398
update jcb-gdas hash to point at authoritaive repo (#1227)
RussTreadon-NOAA Aug 8, 2024
6fc0341
remove bufr from gnssro bufr2ioda converter and json filenames (#1227)
RussTreadon-NOAA Aug 9, 2024
8d842b2
remove obsolete gpsro bufr2ioda converter and json files (#1227)
RussTreadon-NOAA Aug 9, 2024
f44cf3b
Merge branch 'develop' into feature/bufr2ioda
RussTreadon-NOAA Aug 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions parm/atm/jcb-prototype_3dvar.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,27 @@ algorithm: 3dvar
# Observation things
# ------------------
observations:
- satwnd.abi_goes-16
# - ADPSFC
# - ADPUPA
# - SFCSHP
- aircraft
- ascatw.ascat_metop-b
#- conv_ps # Turned off until naming convention of obs files is resolved.
#- gnssro # Turned off until naming convention of obs files is resolved.
# - ascatw.ascat_metop-c
- atms_n20
- atms_npp
- conventional_ps
- gnssro
# - gpsro
# - mtiasi_metop-a
# - mtiasi_metop-b
# - ompsnp_n20
- ompsnp_npp
# - ompstc_n20
- ompstc_npp
- omi_aura
- satwnd.abi_goes-16
- satwnd.abi_goes-18
- satwnd.leogeo_multi
- satwnd.viirs_n20
- satwnd.viirs_npp
# - scatwind_ascat_metop-a
# - snowcvr
28 changes: 24 additions & 4 deletions parm/atm/jcb-prototype_lgetkf.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Use observations for lgetkf
# ---------------------------
app_path_observations: {{PARMgfs}}/gdas/jcb-gdas/observations/atmosphere-lgetkf
app_path_observation_chronicle: {{PARMgfs}}/gdas/jcb-gdas/observation_chronicle/atmosphere-lgetkf
app_path_observation_chronicle: {{PARMgfs}}/gdas/jcb-gdas/observation_chronicle/atmosphere

# Algorithm
# ---------
Expand All @@ -14,7 +14,27 @@ atmosphere_background_ensemble_path: ./bkg/mem%mem%
# Observation things
# ------------------
observations:
- satwnd.abi_goes-16
# - ADPSFC
# - ADPUPA
# - SFCSHP
- aircraft
- ascatw.ascat_metop-b
#- conv_ps # Turned off until naming convention of obs files is resolved.
#- gnssro # Turned off until naming convention of obs files is resolved.
# - ascatw.ascat_metop-c
- atms_n20
- atms_npp
- conventional_ps
- gnssro
# - gpsro
# - mtiasi_metop-a
# - mtiasi_metop-b
# - ompsnp_n20
- ompsnp_npp
# - ompstc_n20
- ompstc_npp
- satwnd.abi_goes-16
- satwnd.abi_goes-18
- satwnd.leogeo_multi
- satwnd.viirs_n20
- satwnd.viirs_npp
# - scatwind_ascat_metop-a
# - snowcvr
2 changes: 1 addition & 1 deletion parm/jcb-gdas
Open in desktop
RussTreadon-NOAA marked this conversation as resolved.
Show resolved Hide resolved
Submodule jcb-gdas updated 24 files
+0 −0 observation_chronicle/atmosphere/mtiasi_metop-a.yaml
+0 −0 observation_chronicle/atmosphere/mtiasi_metop-b.yaml
+365 −0 observations/atmosphere-lgetkf/aircraft.yaml.j2
+546 −0 observations/atmosphere-lgetkf/atms_n20.yaml.j2
+564 −0 observations/atmosphere-lgetkf/atms_npp.yaml.j2
+373 −0 observations/atmosphere-lgetkf/conventional_ps.yaml.j2
+588 −0 observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2
+588 −0 observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2
+339 −0 observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2
+144 −0 observations/atmosphere-lgetkf/ompstc_npp.yaml.j2
+610 −0 observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2
+270 −0 observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2
+298 −0 observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2
+298 −0 observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2
+7 −7 observations/atmosphere/atms_n20.yaml.j2
+7 −7 observations/atmosphere/atms_npp.yaml.j2
+0 −0 observations/atmosphere/conventional_ps.yaml.j2
+7 −7 observations/atmosphere/cris-fsr_n20.yaml.j2
+7 −7 observations/atmosphere/cris-fsr_npp.yaml.j2
+7 −7 observations/atmosphere/mtiasi_metop-a.yaml.j2
+7 −7 observations/atmosphere/mtiasi_metop-b.yaml.j2
+601 −0 observations/atmosphere/satwnd.abi_goes-18.yaml.j2
+7 −7 observations/atmosphere/ssmis_f17.yaml.j2
+7 −7 observations/atmosphere/ssmis_f18.yaml.j2
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_acft_profiles_prepbufr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,7 @@ def bufr_to_ioda(config, logger):
# Create the dimensions
dims = {'Location': np.arange(0, lat.shape[0])}

iodafile = f"{cycle_type}.t{hh}z.acft_profiles.{data_format}.nc"
iodafile = f"{cycle_type}.t{hh}z.acft_profiles.tm00.nc"
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.debug(f" ... ... Create OUTPUT file: {OUTPUT_PATH}")

Expand Down
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_adpsfc_prepbufr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ def bufr_to_ioda(config, logger):
# Create the dimensions
dims = {'Location': np.arange(0, lat.shape[0])}

iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
iodafile = f"{cycle_type}.t{hh}z.{data_type}.tm00.nc"
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.debug(f" ... ... Create OUTPUT file: {OUTPUT_PATH}")

Expand Down
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_adpupa.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,7 @@ def bufr_to_ioda(config, logger):
dims = {'Location': np.arange(0, clat.shape[0])}

# Create IODA ObsSpace
iodafile = f"{cycle_type}.t{hh}z.{data_type}.nc"
iodafile = f"{cycle_type}.t{hh}z.{data_type}.tm00.nc"
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.info(f"Create output file: {OUTPUT_PATH}")
obsspace = ioda_ospace.ObsSpace(OUTPUT_PATH, mode='w', dim_dict=dims)
Expand Down
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_adpupa_prepbufr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,7 @@ def bufr_to_ioda(config, logger):
dims = {'Location': np.arange(0, lat.shape[0])}

# Create IODA ObsSpace
iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
iodafile = f"{cycle_type}.t{hh}z.{data_type}.tm00.nc"
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.info(f"Create output file: {OUTPUT_PATH}")
obsspace = ioda_ospace.ObsSpace(OUTPUT_PATH, mode='w', dim_dict=dims)
Expand Down
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_conventional_prepbufr_ps.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -864,7 +864,7 @@ def bufr_to_ioda(config, logger):
# Create the dimensions
dims = {'Location': np.arange(0, lat.shape[0])}

iodafile = f"{cycle_type}.t{hh}z.{data_description}.{data_format}.nc"
iodafile = f"{cycle_type}.t{hh}z.{data_description}.tm00.nc"
RussTreadon-NOAA marked this conversation as resolved.
Show resolved Hide resolved
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.debug(f" ... ... Create OUTPUT file: {OUTPUT_PATH}")

Expand Down
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_gnssro_bufr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -403,7 +403,7 @@ def bufr_to_ioda(config, logger):
# Create the dimensions
dims = {'Location': np.arange(0, clath.shape[0])}

iodafile = f"{cycle_type}.t{hh}z.{data_type}.tm00.{data_format}.nc"
iodafile = f"{cycle_type}.t{hh}z.{data_type}.tm00.nc"
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.debug(f" ... ... Create OUTPUT file: {OUTPUT_PATH}")

Expand Down
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_gpsro_bufr_combined.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ def bufr_to_ioda(config, logger):
# Create the dimensions
dims = {'Location': np.arange(0, clath.shape[0])}

iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
iodafile = f"{cycle_type}.t{hh}z.{data_type}.tm00.nc"
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.debug(f" ... ... Create OUTPUT file: {OUTPUT_PATH}")

Expand Down
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_sfcshp_prepbufr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ def bufr_to_ioda(config, logger):
# Create the dimensions
dims = {'Location': np.arange(0, lat.shape[0])}

iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
iodafile = f"{cycle_type}.t{hh}z.{data_type}.tm00.nc"
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.debug(f" ... ... Create OUTPUT file: {OUTPUT_PATH}")

Expand Down
2 changes: 1 addition & 1 deletion ush/ioda/bufr2ioda/bufr2ioda_snocvr_bufr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ def bufr_to_ioda(config, logger):
# Create the dimensions
dims = {'Location': snod.shape[0]}

iodafile = f"{cycle_type}.t{hh}z.{data_type}.{data_format}.nc"
iodafile = f"{cycle_type}.t{hh}z.{data_type}.nc"
OUTPUT_PATH = os.path.join(ioda_dir, iodafile)
logger.debug(f" ... ... Create OUTPUT file: {OUTPUT_PATH}")

Expand Down
4 changes: 2 additions & 2 deletions ush/run_satbias_conv.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def run_satbias_conv(config):
f.write('output:\n')
for sat in satlist:
f.write(f'- sensor: {sat}\n')
f.write(f' output file: {sat}_satbias.nc4\n')
f.write(f' output file: {sat}_satbias.nc\n')
f.write(' predictors: *default_preds\n')
# run executable
runcmd = f'./satbias2ioda.x satbias_converter.yaml'
Expand All @@ -110,7 +110,7 @@ def run_satbias_conv(config):
# copy output files to output directory
outdir = os.path.join(ufo_bc_root, f'{cdump}.{pdy}', cyc, 'atmos')
os.makedirs(outdir, exist_ok=True)
ncfiles = glob.glob(os.path.join(workdir, '*.nc4'))
ncfiles = glob.glob(os.path.join(workdir, '*.nc'))
txtfiles = glob.glob(os.path.join(workdir, '*.txt'))
allfiles = ncfiles + txtfiles
for f in allfiles:
Expand Down
Loading