Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/nicke acft profiles prepbufr #11

Draft
wants to merge 85 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
48b4a3f
added dump diur
rmclaren Oct 8, 2024
21c38b4
AIRCFT is ready
Oct 24, 2024
990f093
Final-ish updates
Oct 29, 2024
fd8109d
update
Oct 31, 2024
df49114
ADPSFC mapping and encoder file
Oct 31, 2024
94f233e
SFCSHP mapping and encoder file
Oct 31, 2024
ba62d79
remove path appends
Oct 31, 2024
f13cc3d
remove path appends
Oct 31, 2024
51928b5
CAT mnemonic
Oct 31, 2024
d46c503
CAT mnemonic
Oct 31, 2024
d3478fd
Add mapping file, python scripts, and shell scripts for satwind goes
Nov 13, 2024
7277cc5
remove test
Nov 18, 2024
a4efbb2
Update components (mapping, Python script, configuration) for satwind
Nov 18, 2024
5689332
remove old yaml
Nov 19, 2024
16db06f
Add configuration files and test shell script
Nov 19, 2024
927bae2
rename configuration yaml
emilyhcliu Nov 19, 2024
9dfd16d
Add temporary testing script
emilyhcliu Nov 19, 2024
54305c9
Add README file
emilyhcliu Nov 19, 2024
89f429a
rename README to README.md
emilyhcliu Nov 19, 2024
b115130
Update README.md
emilyhcliu Nov 19, 2024
f0ba20f
Update readme file
emilyhcliu Nov 19, 2024
8d324d0
Update README.md
emilyhcliu Nov 19, 2024
d70867f
Update README
emilyhcliu Nov 19, 2024
e5a95e7
rename process_bufr2ioda to bufr2ioda.sh
emilyhcliu Nov 19, 2024
7dbc826
Update README.md
emilyhcliu Nov 19, 2024
6346844
Update README.md
emilyhcliu Nov 19, 2024
f91d1d1
Update README.md
emilyhcliu Nov 19, 2024
8ba4d7b
Update README.md
emilyhcliu Nov 19, 2024
87096eb
Update readme
emilyhcliu Nov 19, 2024
3e7789c
Add comments
emilyhcliu Nov 19, 2024
a645370
update comments
emilyhcliu Nov 19, 2024
c0fdb95
update bufr2ioda.sh
emilyhcliu Nov 19, 2024
c262e3d
update usage
emilyhcliu Nov 19, 2024
dde4d27
Add comments
emilyhcliu Nov 19, 2024
d080c23
add cycle in input path
emilyhcliu Nov 20, 2024
09d0061
Modify global attribute in the mapping file
emilyhcliu Nov 20, 2024
8f23b5b
Merge branch 'develop' into feature/dump_satwind_goes
emilyhcliu Nov 21, 2024
cd74eee
remove . before bufr2ioda.sh
emilyhcliu Nov 21, 2024
6351d34
Update README.md
emilyhcliu Nov 21, 2024
b2ea3ae
Update README.md
emilyhcliu Nov 22, 2024
d678190
Update README.md
emilyhcliu Nov 22, 2024
67311d6
remove wxflow from the test script
emilyhcliu Nov 22, 2024
428e600
remove wxflow from input
emilyhcliu Nov 22, 2024
6564a18
Update README.md
emilyhcliu Nov 22, 2024
f757393
Add comment block for logger
emilyhcliu Nov 24, 2024
77f2e92
add bufrtype (this is bufr dump list)
emilyhcliu Nov 24, 2024
00cd504
Update documentation
emilyhcliu Nov 24, 2024
9d9e821
Add bufrtype and update README
emilyhcliu Nov 24, 2024
e9b8833
Add split_by_category input
emilyhcliu Nov 25, 2024
aec8e56
Update README
emilyhcliu Nov 25, 2024
5679a32
Update README.md
emilyhcliu Nov 25, 2024
70f90d9
Rename bufr2ioda to encodeBufr and update readme
emilyhcliu Nov 25, 2024
1365772
Update README.md
emilyhcliu Nov 26, 2024
9a8487a
Merge branch 'feature/dump_satwind_goes' into feature/NICKE_adpsfc_pr…
nicholasesposito Nov 26, 2024
0f5dce2
adpsfc changes. works
nicholasesposito Dec 5, 2024
e523fdb
update 1
nicholasesposito Dec 5, 2024
07a7239
Merge branch 'feature/NICKE_adpsfc_prepbufr' into feature/NICKE_sfcsh…
nicholasesposito Dec 5, 2024
c178022
initial
nicholasesposito Dec 5, 2024
eba41c2
name changes
nicholasesposito Dec 5, 2024
a9fec90
Merge branch 'feature/NICKE_adpsfc_prepbufr' into feature/NICKE_sfcsh…
nicholasesposito Dec 5, 2024
72f5622
changes
nicholasesposito Dec 10, 2024
414b489
encodeBufr_Nick.sh
nicholasesposito Dec 10, 2024
98d73fc
Merge branch 'feature/NICKE_adpsfc_prepbufr' into feature/NICKE_sfcsh…
nicholasesposito Dec 10, 2024
4f92d5c
all works now
nicholasesposito Dec 10, 2024
282e177
small changes for script
nicholasesposito Dec 10, 2024
de85533
Merge branch 'feature/NICKE_adpsfc_prepbufr' into feature/NICKE_sfcsh…
nicholasesposito Dec 10, 2024
61cdae5
all work
nicholasesposito Dec 10, 2024
af895ca
Merge branch 'feature/NICKE_sfcshp_prepbufr' into feature/NICKE_acft_…
nicholasesposito Dec 10, 2024
96b3fd3
initial changes
nicholasesposito Dec 10, 2024
61427a9
rm bufr2ioda* for bufr
nicholasesposito Dec 10, 2024
2410744
Merge branch 'feature/NICKE_sfcshp_prepbufr' into feature/NICKE_acft_…
nicholasesposito Dec 10, 2024
91e3900
initial changes
nicholasesposito Dec 10, 2024
0d98aa0
update mapping - rm obstyp
nicholasesposito Dec 10, 2024
0761d4c
these work
nicholasesposito Dec 10, 2024
38ad9e3
rm adpsfc sfcshp
nicholasesposito Dec 10, 2024
a104092
testing
nicholasesposito Dec 12, 2024
d15f6f8
encodeBufr_Nick.sh
nicholasesposito Dec 12, 2024
1852d03
encodeBufr_Nick.sh from sfcshp
nicholasesposito Dec 12, 2024
4a2ca6e
dateTime, cycle_time
nicholasesposito Dec 12, 2024
fa00d4f
remove longitude range for now, python 3.7/3.10w
Dec 20, 2024
9f6c1ca
4backend, mpi, name changes
Dec 26, 2024
b603c74
cp file to config/mapping
Dec 26, 2024
92511a5
py to mapping
Dec 26, 2024
01e6670
reference time
Dec 27, 2024
b3ac089
dateTime min calc, attribute name update, sh script update
Jan 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions dump/config/bufr2ioda_bufr_backend_satwnd_amv_goes.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"

observations:
- obs space:
name: "satwind_goes-16"
simulated variables: [windDirection, windSpeed]
obsdatain:
engine:
type: bufr
obsfile: "./testinput/2021080100/gdas.t00z.satwnd.tm00.bufr_d"
mapping file: "./bufr2ioda_satwnd_amv_goes_mapping.yaml"
category: ["goes-16"]
cache categories: # optional
- ["goes-16"]
- ["goes-17"]
- ["goes-18"]
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/bufr_backend/gdas.t00z.satwnd.abi_goes-16.tm00.nc"

- obs space:
name: "satwind_goes-17"
simulated variables: [windDirection, windSpeed]
obsdatain:
engine:
type: bufr
obsfile: "./testinput/2021080100/gdas.t00z.satwnd.tm00.bufr_d"
mapping file: "./bufr2ioda_satwnd_amv_goes_mapping.yaml"
category: ["goes-17"]
cache categories: # optional
- ["goes-16"]
- ["goes-17"]
- ["goes-18"]
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/bufr_backend/gdas.t00z.satwnd.abi_goes-17.tm00.nc"

- obs space:
name: "satwind_goes-18"
simulated variables: [windDirection, windSpeed]
obsdatain:
engine:
type: bufr
obsfile: "./testinput/2021080100/gdas.t00z.satwnd.tm00.bufr_d"
mapping file: "./bufr2ioda_satwnd_amv_goes_mapping.yaml"
category: ["goes-18"]
cache categories: # optional
- ["goes-16"]
- ["goes-17"]
- ["goes-18"]
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/bufr_backend/gdas.t00z.satwnd.abi_goes-18.tm00.nc"
18 changes: 18 additions & 0 deletions dump/config/bufr_bufr4backend_acft_profiles_prepbufr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
time window:
begin: "2021-07-31T21:00:00Z"
end: "2021-08-01T03:00:00Z"
bound to include: begin

observations:
- obs space:
name: "AIRCFT"
simulated variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
obsdatain:
engine:
type: bufr
obsfile: "./testinput/2021080100/gdas.t00z.acft_profiles.tm00.prepbufr"
mapping file: "./bufr_acft_profiles_prepbufr_mapping.yaml"
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/bufr4backend/gdas.t00z.acft_profiles_prepbufr.tm00.nc"
18 changes: 18 additions & 0 deletions dump/config/bufr_bufr4backend_acft_profiles_prepbufr_mpi0.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
time window:
begin: "2021-07-31T21:00:00Z"
end: "2021-08-01T03:00:00Z"
bound to include: begin

observations:
- obs space:
name: "AIRCFT"
simulated variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
obsdatain:
engine:
type: bufr
obsfile: "./testinput/2021080100/gdas.t00z.acft_profiles.tm00.prepbufr"
mapping file: "./bufr_acft_profiles_prepbufr_mapping.yaml"
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/bufr4backend/gdas.t00z.acft_profiles_prepbufr.tm00_mpi0.nc"
18 changes: 18 additions & 0 deletions dump/config/bufr_bufr4backend_acft_profiles_prepbufr_mpi4.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
time window:
begin: "2021-07-31T21:00:00Z"
end: "2021-08-01T03:00:00Z"
bound to include: begin

observations:
- obs space:
name: "AIRCFT"
simulated variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
obsdatain:
engine:
type: bufr
obsfile: "./testinput/2021080100/gdas.t00z.acft_profiles.tm00.prepbufr"
mapping file: "./bufr_acft_profiles_prepbufr_mapping.yaml"
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/bufr4backend/gdas.t00z.acft_profiles_prepbufr.tm00_mpi4.nc"
23 changes: 23 additions & 0 deletions dump/config/bufr_script4backend_acft_profiles_prepbufr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
time window:
begin: "2021-07-31T21:00:00Z"
end: "2021-08-01T03:00:00Z"
bound to include: begin

observations:
- obs space:
name: "AIRCFT"
observed variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
derived variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
simulated variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
obsdatain:
engine:
type: script
script file: "bufr_acft_profiles_prepbufr.py"
args:
input_path: "testinput/2021080100/gdas.t00z.acft_profiles.tm00.prepbufr"
mapping_path: "./bufr_acft_profiles_prepbufr_mapping.yaml"
cycle_time: "2021080100"
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/script4backend/gdas.t00z.acft_profiles_prepbufr.tm00.nc"
23 changes: 23 additions & 0 deletions dump/config/bufr_script4backend_acft_profiles_prepbufr_mpi0.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
time window:
begin: "2021-07-31T21:00:00Z"
end: "2021-08-01T03:00:00Z"
bound to include: begin

observations:
- obs space:
name: "AIRCFT"
observed variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
derived variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
simulated variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
obsdatain:
engine:
type: script
script file: "bufr_acft_profiles_prepbufr.py"
args:
input_path: "testinput/2021080100/gdas.t00z.acft_profiles.tm00.prepbufr"
mapping_path: "./bufr_acft_profiles_prepbufr_mapping.yaml"
cycle_time: "2021080100"
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/script4backend/gdas.t00z.acft_profiles_prepbufr.tm00_mpi0.nc"
23 changes: 23 additions & 0 deletions dump/config/bufr_script4backend_acft_profiles_prepbufr_mpi4.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
time window:
begin: "2021-07-31T21:00:00Z"
end: "2021-08-01T03:00:00Z"
bound to include: begin

observations:
- obs space:
name: "AIRCFT"
observed variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
derived variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
simulated variables: ['airTemperature','specificHumidity','windNorthward','windEastward']
obsdatain:
engine:
type: script
script file: "bufr_acft_profiles_prepbufr.py"
args:
input_path: "testinput/2021080100/gdas.t00z.acft_profiles.tm00.prepbufr"
mapping_path: "./bufr_acft_profiles_prepbufr_mapping.yaml"
cycle_time: "2021080100"
obsdataout:
engine:
type: H5File
obsfile: "./testoutput/2021080100/script4backend/gdas.t00z.acft_profiles_prepbufr.tm00_mpi4.nc"
Loading