Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OMPS/OMI Validation #57

Open
ADCollard opened this issue Feb 15, 2023 · 21 comments
Open

OMPS/OMI Validation #57

ADCollard opened this issue Feb 15, 2023 · 21 comments
Assignees
@azadeh-gh

Comments

@ADCollard
Copy link

Review current use of OMPS and add a test YAML to parm/atm/obs/testing in GDASApp to reproduce the GSI (HofX, QC and final obs error)
@azadeh-gh
Copy link

image

@azadeh-gh
Copy link

azadeh-gh commented Mar 3, 2023 •
edited
Loading

ompsnp_npp test YAML added to parm/atm/obs/testing in GDASApp to reproduce the GSI HofX, QC and final obs error.
JEDI h(x) vs. GSI h(x) was identical.

gsi_hofx_vs_jedi_hofx_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

@azadeh-gh
Copy link

azadeh-gh commented Mar 3, 2023 •
edited
Loading

JEDI h(x) - GSI h(x) were almost zero

hofxdiff_histogram_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

@azadeh-gh
Copy link

azadeh-gh commented Mar 3, 2023 •
edited
Loading

Difference of JEDI and GSI final obs error was zero

errordiff_histogram_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

@azadeh-gh
Copy link

azadeh-gh commented Mar 3, 2023 •
edited
Loading

gsi_hofx_vs_obs_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer
jedi_hofx_vs_obs_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

@azadeh-gh
Copy link

azadeh-gh commented Mar 3, 2023 •
edited
Loading

GSI omb vs JEDI omb for both all obs and obs passed QC in JEDI were identical.

gsi_omb_vs_jedi_omb_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

@azadeh-gh
Copy link

azadeh-gh commented Mar 3, 2023 •
edited
Loading

omi_aura test YAML added to parm/atm/obs/testing in GDASApp to reproduce the GSI HofX, QC and final obs error.

@azadeh-gh
Copy link

JEDI h(x) vs. GSI h(x) was identical.
gsi_hofx_vs_jedi_hofx_diag_omi_aura_ozoneTotal

@azadeh-gh
Copy link

errordiff_histogram_diag_omi_aura_ozoneTotal
hofxdiff_histogram_diag_omi_aura_ozoneTotal

@azadeh-gh
Copy link

azadeh-gh commented Mar 9, 2023 •
edited
Loading

Scatter plots from omi_aura with Gross error check with (O - B)

gsi_omb_vs_jedi_omb_omi_aura_diag_2021080100_omi_aura_ozoneTotal

jedi_hofx_vs_obs_omi_aura_diag_2021080100_omi_aura_ozoneTotal

gsi_hofx_vs_obs_omi_aura_diag_2021080100_omi_aura_ozoneTotal

@azadeh-gh
Copy link

azadeh-gh commented Mar 9, 2023 •
edited
Loading

Scatter plots from omi_aura without Gross error check

gsi_omb_vs_jedi_omb_omi_aura_diag_2021080100_omi_aura_ozoneTotal

jedi_hofx_vs_obs_omi_aura_diag_2021080100_omi_aura_ozoneTotal

gsi_hofx_vs_obs_omi_aura_diag_2021080100_omi_aura_ozoneTotal

@azadeh-gh
Copy link

azadeh-gh commented Mar 9, 2023 •
edited
Loading

ompstc8_npp test YAML to parm/atm/obs/testing in GDASApp that reproduces the GSI (HofX, QC and final obs error).

gsi_hofx_vs_jedi_hofx_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

hofxdiff_histogram_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

errordiff_histogram_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

gsi_omb_vs_jedi_omb_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

gsi_hofx_vs_obs_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal
jedi_hofx_vs_obs_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

@azadeh-gh
Copy link

azadeh-gh commented Apr 6, 2023 •
edited
Loading

Sent pull request (PR438)

@azadeh-gh azadeh-gh mentioned this issue Apr 14, 2023
Merged
@emilyhcliu
Copy link

emilyhcliu commented Jul 15, 2023 •
edited
Loading

Re-evaluate OMPSNP NPP

(1) Create input data (obs and geoval) - need to include obs filtered out in the read routine
(2) Update QC filters - there are some minor updates ---> Will issue PR in GDASApp to update the filters in the YAML
(3) Update variable names so they follow the IODA naming convention

Here are the updated checks

HofX JEDI vs. GSI

gsi_hofx_vs_jedi_hofx_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

JEDI HofX vs. Obs GSI HofX vs. Obs
jedi_hofx_vs_obs_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer gsi_hofx_vs_obs_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

gsi_omb_vs_jedi_omb_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

HofX Difference vs. Pressure Layers
hofxdiff_vs_pressure_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

hofxdiff_vs_qcdiff_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

ObsError JEDI vs. GSI
errordiff_vs_pressure_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

@emilyhcliu
Copy link

emilyhcliu commented Jul 17, 2023 •
edited
Loading

Here are the re-evaluation results for OMPSTC using the updated obs file and filters

(no thinning in the input obs file)

gsi_hofx_vs_jedi_hofx_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

JEDI HofX vs. Obs GSI HofX vs. Obs
jedi_hofx_vs_obs_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal gsi_hofx_vs_obs_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

gsi_omb_vs_jedi_omb_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

errordiff_vs_pressure_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

hofxdiff_vs_pressure_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

hofxdiff_vs_qcdiff_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

@emilyhcliu emilyhcliu changed the title OMPS Validation OMPS/OMI Validation Jul 17, 2023
@emilyhcliu
Copy link

emilyhcliu commented Jul 17, 2023 •
edited
Loading

Re-evaluation for OMI AURA

(No data thinning for input data)

gsi_hofx_vs_jedi_hofx_omi_aura_diag_2021080100_omi_aura_ozoneTotal

JEDI HofX vs. Obs GSI HofX vs. Obs
jedi_hofx_vs_obs_omi_aura_diag_2021080100_omi_aura_ozoneTotal gsi_hofx_vs_obs_omi_aura_diag_2021080100_omi_aura_ozoneTotal

gsi_omb_vs_jedi_omb_omi_aura_diag_2021080100_omi_aura_ozoneTotal

errordiff_vs_pressure_omi_aura_diag_2021080100_omi_aura_ozoneTotal
hofxdiff_vs_pressure_omi_aura_diag_2021080100_omi_aura_ozoneTotal

hofxdiff_vs_qcdiff_omi_aura_diag_2021080100_omi_aura_ozoneTotal

The last plot does not look right. Some observations have QC difference = 14, meaning GsiEffectiveQC= 0, but JEDI EffectiveQC =14.
This is not because the filters are wrong. This is related to how we prepared the obs file from the GSI diagnostic.
I will check and redo the obs file from GSI and run the GDASApp test again.

@emilyhcliu
Copy link

See more information in the following T2O issues related to end-to-end tests for ozone
OMI Issue #77
OMPSTC Issue #75
OMPSNP Issue #76

@emilyhcliu
Copy link

emilyhcliu commented Jul 20, 2023 •
edited
Loading

The GDASApp test results based on the updated GSI obs file for OMI AURA

gsi_hofx_vs_jedi_hofx_omi_aura_diag_2021080100_omi_aura_ozoneTotal

JEDI HofX vs Obs GSI HofX vs Obs
jedi_hofx_vs_obs_omi_aura_diag_2021080100_omi_aura_ozoneTotal gsi_hofx_vs_obs_omi_aura_diag_2021080100_omi_aura_ozoneTotal

gsi_omb_vs_jedi_omb_omi_aura_diag_2021080100_omi_aura_ozoneTotal

errordiff_vs_pressure_omi_aura_diag_2021080100_omi_aura_ozoneTotal

hofxdiff_vs_pressure_omi_aura_diag_2021080100_omi_aura_ozoneTotal
hofxdiff_vs_qcdiff_omi_aura_diag_2021080100_omi_aura_ozoneTotal

Here is how to interpret the plot:
QC diff = 9 --- GsiEffectiveQC = 1, and JEDI EffectiveQC = 10 (Missing Value) --- observations are screened from both GSI and UFO.

QC diff = 13 --- GsiEffectiveQC = 1, and JEDI EffectiveQC = 14 (RejectList)--- observation are screened from both GSI and UFO

QC diff = 0 --- GsiEffectiveQC (GSI) and EffectiveQC (JEDI)) are zeros

The plot tells us that the QC results between GSI and JEDI are identical.

@emilyhcliu
Copy link

The GDASApp test results based on the updated GSI obs file for OMPSTC8 NPP

gsi_hofx_vs_jedi_hofx_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

JEDI HofX vs. Obs GSI HofX vs. Obs
jedi_hofx_vs_obs_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal gsi_hofx_vs_obs_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

gsi_omb_vs_jedi_omb_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal
errordiff_vs_pressure_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal
hofxdiff_vs_pressure_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal
hofxdiff_vs_qcdiff_ompstc8_npp_diag_2021080100_ompstc8_npp_ozoneTotal

@emilyhcliu
Copy link

emilyhcliu commented Jul 20, 2023 •
edited
Loading

The GDASApp test results based on the updated GSI obs file for OMPSNP NPP

gsi_hofx_vs_jedi_hofx_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

JEDI HofX vs. Obs GSI HofX vs. Obs
jedi_hofx_vs_obs_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer gsi_hofx_vs_obs_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

gsi_omb_vs_jedi_omb_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer
errordiff_vs_pressure_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer
hofxdiff_vs_pressure_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

hofxdiff_vs_qcdiff_ompsnp_npp_diag_2021080100_ompsnp_npp_ozoneLayer

@emilyhcliu
Copy link

@ADCollard @azadeh-gh Please check the last three new comments. They are updated results for omi, ompstc, and ompsnp.
These results show that the QC flag differences (GsiEffectiveQC vs. EffectiveQC) between GSI and UFO are identical.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@azadeh-gh azadeh-gh

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@emilyhcliu @ADCollard @azadeh-gh