Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the capability to assimilate the MADIS snow depth for global-workflow #602

Merged
merged 81 commits into from
Sep 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
c2f5659
Update to new stable build on 20230424
CoryMartin-NOAA Apr 24, 2023
5f5f39d
Update to new stable build on 20230428
CoryMartin-NOAA Apr 28, 2023
f9c7e44
Update to new stable build on 20230428
CoryMartin-NOAA Apr 28, 2023
a36ac33
Update CMakeLists.txt
CoryMartin-NOAA Apr 28, 2023
fee9a6f
Update to new stable build on 20230428
CoryMartin-NOAA Apr 28, 2023
7746838
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Apr 28, 2023
0058ca3
Update to new stable build on 20230429
CoryMartin-NOAA Apr 29, 2023
dd9f838
Update to new stable build on 20230430
CoryMartin-NOAA Apr 30, 2023
766b4d0
Update to new stable build on 20230501
CoryMartin-NOAA May 1, 2023
da646d2
Update to new stable build on 20230501
CoryMartin-NOAA May 1, 2023
50740f1
Update to new stable build on 20230502
CoryMartin-NOAA May 2, 2023
f592e46
Update to new stable build on 20230503
CoryMartin-NOAA May 3, 2023
e98bf88
Update to new stable build on 20230504
CoryMartin-NOAA May 4, 2023
bd68e1b
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA May 9, 2023
71ffa84
Update to new stable build on 20230509
CoryMartin-NOAA May 9, 2023
2586a7c
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA May 10, 2023
4ac96b9
Update to new stable build on 20230510
CoryMartin-NOAA May 10, 2023
e77c556
Update to new stable build on 20230511
CoryMartin-NOAA May 11, 2023
ab6f06d
Update to new stable build on 20230512
CoryMartin-NOAA May 12, 2023
56bcbf2
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA May 16, 2023
81675c9
Update to new stable build on 20230516
CoryMartin-NOAA May 16, 2023
27cf69d
Update to new stable build on 20230517
CoryMartin-NOAA May 17, 2023
1696cda
Update to new stable build on 20230518
CoryMartin-NOAA May 18, 2023
98edc36
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA May 19, 2023
551d4c7
Update to new stable build on 20230519
CoryMartin-NOAA May 19, 2023
328cc4a
Update to new stable build on 20230520
CoryMartin-NOAA May 20, 2023
18b4a76
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA May 23, 2023
f38549e
Update to new stable build on 20230523
CoryMartin-NOAA May 23, 2023
8bf4bdd
Update to new stable build on 20230524
CoryMartin-NOAA May 24, 2023
2bd4c46
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA May 25, 2023
9176046
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA May 26, 2023
afb3a93
Update to new stable build on 20230526
CoryMartin-NOAA May 26, 2023
2df6e96
Update to new stable build on 20230527
CoryMartin-NOAA May 27, 2023
6282fdf
Update to new stable build on 20230531
CoryMartin-NOAA May 31, 2023
d171588
Update to new stable build on 20230601
CoryMartin-NOAA Jun 1, 2023
7d13e0d
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jun 2, 2023
87e071f
Update to new stable build on 20230602
CoryMartin-NOAA Jun 2, 2023
7a2f299
Update to new stable build on 20230603
CoryMartin-NOAA Jun 3, 2023
9623436
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jun 6, 2023
c9cd07f
Update to new stable build on 20230606
CoryMartin-NOAA Jun 6, 2023
bd9cf6a
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jun 9, 2023
1da2e63
Update to new stable build on 20230609
CoryMartin-NOAA Jun 9, 2023
e7fa85b
Update to new stable build on 20230610
CoryMartin-NOAA Jun 10, 2023
8f223da
Update to new stable build on 20230613
CoryMartin-NOAA Jun 13, 2023
d01c5a7
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jul 1, 2023
3f6d0e2
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jul 7, 2023
333337b
Update to new stable build on 20230707
CoryMartin-NOAA Jul 7, 2023
a46df8d
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jul 8, 2023
e61e33c
Update to new stable build on 20230708
CoryMartin-NOAA Jul 8, 2023
65618a9
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jul 14, 2023
4452b0a
Update to new stable build on 20230714
CoryMartin-NOAA Jul 14, 2023
7e3d694
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jul 14, 2023
fed5c4d
Update to new stable build on 20230715
CoryMartin-NOAA Jul 15, 2023
b3f800e
Update to new stable build on 20230717
CoryMartin-NOAA Jul 17, 2023
ada6239
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Jul 18, 2023
fd8aa2f
Update to new stable build on 20230718
CoryMartin-NOAA Jul 18, 2023
a8e878a
Update to new stable build on 20230720
CoryMartin-NOAA Jul 20, 2023
b7c8a4b
Update to new stable build on 20230721
CoryMartin-NOAA Jul 21, 2023
40621ca
Update to new stable build on 20230722
CoryMartin-NOAA Jul 22, 2023
846c9ce
Prepare GTS snow depth observations for JEDI-based Land DA (#541)
jiaruidong2017 Jul 24, 2023
597b970
Filtering of the offline ensemble members, and more ... (#539)
CoryMartin-NOAA Jul 25, 2023
7966501
Merge in develop for Jiarui's PR to have a commit hash (#543)
CoryMartin-NOAA Jul 25, 2023
96c694e
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Aug 4, 2023
f743728
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Aug 4, 2023
0f8ed3d
Revert to stable hashes for all
CoryMartin-NOAA Aug 4, 2023
09757ce
back to crtm 2.4
CoryMartin-NOAA Aug 4, 2023
ac097da
Merge branch 'develop' into feature/stable-nightly
CoryMartin-NOAA Aug 16, 2023
2774a10
Update to new stable build on 20230816
CoryMartin-NOAA Aug 16, 2023
84f6565
test
jiaruidong2017 Aug 23, 2023
4381583
Initialize the madis snow DA in GDASApp for Global-Workflow.
jiaruidong2017 Aug 23, 2023
a1a1a5b
Merge branch 'develop' into feature/madis_da
jiaruidong2017 Aug 23, 2023
986a4c9
Merge branch 'develop' into feature/madis_da
jiaruidong2017 Aug 28, 2023
a1f59eb
Merge branch 'develop' into feature/madis_da
jiaruidong2017 Sep 4, 2023
bb2e60f
Merge develop for the CMakeLists.txt
jiaruidong2017 Sep 5, 2023
9825f94
Made changes for global-workflow.
jiaruidong2017 Sep 9, 2023
a4e02ea
Add temporal thinning filter for madis data
jiaruidong2017 Sep 12, 2023
41b60fd
Update the temporal thinning filter.
jiaruidong2017 Sep 12, 2023
d6f5b59
Change '$(' to '{{' as suggested.
jiaruidong2017 Sep 12, 2023
ca456a1
Made changes from $() to {{ }}
jiaruidong2017 Sep 12, 2023
5d9968c
Remove the comments
jiaruidong2017 Sep 12, 2023
671f785
Merge branch 'develop' into feature/madis_da
CoryMartin-NOAA Sep 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions parm/land/obs/config/snocvr_snow.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
obs space:
name: snocvr_snow
distribution:
name: Halo
halo size: 250e3
obsdatain:
engine:
type: H5File
obsfile: '{{ DATA }}/obs/{{ OPREFIX }}snocvr_snow.nc4'
obsdataout:
engine:
type: H5File
obsfile: '{{ DATA }}/diags/diag_snocvr_snow.nc4'
simulated variables: [totalSnowDepth]
obs operator:
name: Composite
components:
- name: Identity
- name: BackgroundErrorIdentity
obs error:
covariance model: diagonal
obs localizations:
- localization method: Horizontal SOAR
lengthscale: 250e3
soar horizontal decay: 0.000021
max nobs: 50
- localization method: Vertical Brasnett
vertical lengthscale: 700
obs filters:
- filter: Bounds Check
filter variables:
- name: totalSnowDepth
minvalue: 0.0
maxvalue: 2000.0
action:
name: reject
- filter: Domain Check
where:
- variable:
name: MetaData/height
minvalue: -999.0
- filter: Domain Check # land only
where:
- variable:
name: GeoVaLs/slmsk
minvalue: 0.5
maxvalue: 1.5
- filter: RejectList # no land-ice
where:
- variable:
name: GeoVaLs/vtype
minvalue: 14.5
maxvalue: 15.5
- filter: Temporal Thinning
min_spacing: '{{ LAND_WINDOW_LENGTH }}'
seed_time: '{{ current_cycle | to_isotime }}'
category_variable:
name: MetaData/stationIdentification
- filter: Background Check # gross error check
filter variables:
- name: totalSnowDepth
threshold: 6.25
action:
name: reject
- filter: Met Office Buddy Check
filter variables:
- name: totalSnowDepth
rejection_threshold: 0.5
traced_boxes: # trace all observations
min_latitude: -90
max_latitude: 90
min_longitude: -180
max_longitude: 180
search_radius: 150 # km
station_id_variable:
name: MetaData/stationIdentification
num_zonal_bands: 24
sort_by_pressure: false
max_total_num_buddies: 15
max_num_buddies_from_single_band: 10
max_num_buddies_with_same_station_id: 5
use_legacy_buddy_collector: false
horizontal_correlation_scale: { "-90": 150, "90": 150 }
temporal_correlation_scale: PT6H
damping_factor_1: 1.0
damping_factor_2: 1.0
background_error_group: BkgError
- filter: Perform Action
filter variables:
- name: totalSnowDepth
action:
name: assign error
error parameter: 40.0
- filter: Variable Assignment
assignments:
- name: GrossErrorProbability/totalSnowDepth
type: float
value: 0.02
- name: BkgError/totalSnowDepth_background_error
type: float
value: 30.0

3 changes: 3 additions & 0 deletions parm/land/obs/lists/gdas_land_gts_only.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
observers:
- !INC ${OBS_YAML_DIR}/adpsfc_snow.yaml
- !INC ${OBS_YAML_DIR}/snocvr_snow.yaml
1 change: 1 addition & 0 deletions parm/land/obs/lists/gdas_land_prototype.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
observers:
- !INC ${OBS_YAML_DIR}/adpsfc_snow.yaml
- !INC ${OBS_YAML_DIR}/snocvr_snow.yaml
- !INC ${OBS_YAML_DIR}/ims_snow.yaml
10 changes: 7 additions & 3 deletions parm/land/prep/prep_gts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,12 @@ gtsbufr:
mkdir:
- $(DATA)/obs
copy:
- [$(COM_OBS)/$(OPREFIX)adpsfc.tm00.bufr_d, $(DATA)/obs/]
- [{{ COM_OBS }}/{{ OPREFIX }}adpsfc.tm00.bufr_d, {{ DATA }}/obs/]
- [{{ COM_OBS }}/{{ OPREFIX }}snocvr.tm00.bufr_d, {{ DATA }}/obs/]
gtsioda:
copy:
- [$(DATA)/$(OPREFIX)adpsfc_snow.nc4, $(COM_OBS)/$(OPREFIX)adpsfc_snow.nc4]

- [{{ DATA }}/{{ OPREFIX }}adpsfc_snow.nc4, {{ COM_OBS }}/{{ OPREFIX }}adpsfc_snow.nc4]
- [{{ DATA }}/{{ OPREFIX }}snocvr_snow.nc4, {{ COM_OBS }}/{{ OPREFIX }}snocvr_snow.nc4]
bufr2ioda:
adpsfc: {{ HOMEgfs }}/sorc/gdas.cd/test/testinput/bufr_adpsfc_snow.yaml
snocvr: {{ HOMEgfs }}/sorc/gdas.cd/test/testinput/bufr_snocvr_snow.yaml
6 changes: 2 additions & 4 deletions test/testinput/bufr_adpsfc_snow.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# (C) Copyright 2021-2022 NOAA/NWS/NCEP/EMC
#
# This software is licensed under the terms of the Apache Licence Version 2.0
# which can be obtained at http://www.apache.org/licenses/LICENSE-2.0.

observations:
- obs space:
name: bufr

obsdatain: $(DATA)/obs/$(OPREFIX)adpsfc.tm00.bufr_d
obsdatain: '{{ DATA }}/obs/{{ OPREFIX }}adpsfc.tm00.bufr_d'

exports:
variables:
Expand Down Expand Up @@ -41,7 +39,7 @@ observations:

ioda:
backend: netcdf
obsdataout: $(DATA)/$(OPREFIX)adpsfc_snow.nc4
obsdataout: '{{ DATA }}/{{ OPREFIX }}adpsfc_snow.nc4'

variables:

Expand Down
83 changes: 83 additions & 0 deletions test/testinput/bufr_snocvr_snow.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# (C) Copyright 2021-2022 NOAA/NWS/NCEP/EMC
#

observations:
- obs space:
name: bufr

obsdatain: '{{ DATA }}/obs/{{ OPREFIX }}snocvr.tm00.bufr_d'

exports:
variables:
# MetaData
timestamp:
datetime:
year: "*/YEAR[1]"
month: "*/MNTH[1]"
day: "*/DAYS[1]"
hour: "*/HOUR[1]"
minute: "*/MINU[1]"
latitude:
query: "[*/CLAT, */CLATH]"
longitude:
query: "[*/CLON, */CLONH]"
stationIdentification:
query: "*/WGOSLID"

height:
query: "[*/SELV, */HSMSL]"

# ObsValue
totalSnowDepth:
query: "*/TOSD"
transforms:
- scale: 1000.0
filters:
- bounding:
variable: totalSnowDepth
upperBound: 10000000

ioda:
backend: netcdf
obsdataout: '{{ DATA }}/{{ OPREFIX }}snocvr_snow.nc4'

variables:

# MetaData
- name: "MetaData/dateTime"
coordinates: "longitude latitude"
source: variables/timestamp
longName: "Datetime"
units: "seconds since 1970-01-01T00:00:00Z"

- name: "MetaData/latitude"
coordinates: "longitude latitude"
source: variables/latitude
longName: "Latitude"
units: "degree_north"
range: [-90, 90]

- name: "MetaData/longitude"
coordinates: "longitude latitude"
source: variables/longitude
longName: "Longitude"
units: "degree_east"
range: [-180, 180]

- name: "MetaData/height"
coordinates: "longitude latitude"
source: variables/height
longName: "Height of Station"

- name: "MetaData/stationIdentification"
coordinates: "longitude latitude"
source: variables/stationIdentification
longName: "Identification of Observing Location"
units: "m"

# ObsValue
- name: "ObsValue/totalSnowDepth"
coordinates: "longitude latitude"
source: variables/totalSnowDepth
longName: "Total Snow Depth"
units: "mm"
Loading