Filtering of the offline ensemble members, and more ... #539

guillaumevernieres · 2023-07-18T17:33:12Z

Temporarily fixes issue SOCA hybrid is broken #534
fixes issue Switch the cost function from 4dvar to fgat #503
sort of fixes Diagonal of B #316
...

emcbot · 2023-07-18T20:40:04Z

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Tue Jul 18 12:47:37 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Jul 18 14:03:41 CDT 2023
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Tue Jul 18 15:40:04 CDT 2023
Tests: 91% tests passed, 5 tests failed out of 53
	1487 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
	1489 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
	1491 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY (Failed)
	1493 - test_gdasapp_soca_socaincr2mom6 (Failed)
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/539/global-workflow/sorc/gdas.cd/build/log.ctest

guillaumevernieres · 2023-07-18T21:40:44Z

Looks like the infamous ufo branch is broken again ... A problem for tomorrow!

guillaumevernieres · 2023-07-21T13:31:15Z

Looks like the infamous ufo branch is broken again ... A problem for tomorrow!

~~@emilyhcliu , the error that I get in the CI is typical of your branch being behind develop ... Could you sync it again? I'm going to push a commit pointing to ufo develop in the meantime.~~
Hummm ... never mind, that's not the issue this time around.

emcbot · 2023-07-21T15:13:15Z

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Fri Jul 21 08:46:23 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Jul 21 09:46:00 CDT 2023
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Fri Jul 21 10:13:15 CDT 2023
Tests: 83% tests passed, 9 tests failed out of 53
	1454 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
	1456 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
	1457 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)
	1458 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY (Failed)
	1460 - test_gdasapp_soca_socaincr2mom6 (Failed)
	1475 - test_gdasapp_atm_jjob_var_run (Failed)
	1476 - test_gdasapp_atm_jjob_var_final (Failed)
	1478 - test_gdasapp_atm_jjob_ens_run (Failed)
	1479 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/539/global-workflow/sorc/gdas.cd/build/log.ctest

emcbot · 2023-07-21T15:18:24Z

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Fri Jul 21 13:54:28 UTC 2023 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Jul 21 14:59:03 UTC 2023
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Fri Jul 21 15:18:23 UTC 2023
Tests: 83% tests passed, 9 tests failed out of 53
	1442 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
	1444 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
	1445 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)
	1446 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY (Failed)
	1448 - test_gdasapp_soca_socaincr2mom6 (Failed)
	1463 - test_gdasapp_atm_jjob_var_run (Failed)
	1464 - test_gdasapp_atm_jjob_var_final (Failed)
	1466 - test_gdasapp_atm_jjob_ens_run (Failed)
	1467 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/539/global-workflow/sorc/gdas.cd/build/log.ctest

emcbot · 2023-07-21T19:54:01Z

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Fri Jul 21 13:31:24 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Jul 21 14:30:40 CDT 2023
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Fri Jul 21 14:54:00 CDT 2023
Tests: 91% tests passed, 5 tests failed out of 53
	1499 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
	1501 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
	1502 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)
	1503 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY (Failed)
	1505 - test_gdasapp_soca_socaincr2mom6 (Failed)
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/539/global-workflow/sorc/gdas.cd/build/log.ctest

emcbot · 2023-07-21T19:58:35Z

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Fri Jul 21 18:31:32 UTC 2023 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Jul 21 19:25:30 UTC 2023
---------------------------------------------------
Tests:                                 *SUCCESS*
Tests: Completed at Fri Jul 21 19:58:35 UTC 2023
Tests: 100% tests passed, 0 tests failed out of 53

emcbot · 2023-07-21T21:08:38Z

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Fri Jul 21 15:16:00 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Jul 21 15:48:41 CDT 2023
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Fri Jul 21 16:08:38 CDT 2023
Tests: 91% tests passed, 5 tests failed out of 53
	1499 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
	1501 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
	1502 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)
	1503 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY (Failed)
	1505 - test_gdasapp_soca_socaincr2mom6 (Failed)
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/539/global-workflow/sorc/gdas.cd/build/log.ctest

guillaumevernieres · 2023-07-21T21:41:22Z

looks like oops develop is behind. Orion CI is failing because of this, if I submit a fix for orion it will break the hera ci ... A problem for Monday.

emcbot · 2023-07-24T14:52:05Z

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Mon Jul 24 08:17:34 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon Jul 24 09:31:44 CDT 2023
---------------------------------------------------
Tests:                                 *SUCCESS*
Tests: Completed at Mon Jul 24 09:52:04 CDT 2023
Tests: 100% tests passed, 0 tests failed out of 53

AndrewEichmann-NOAA

Nothing looks violently wrong

ShastriPaturi

Did not test but can't see any issues either.
Approved.

guillaumevernieres · 2023-07-24T15:30:43Z

Did not test but can't see any issues either. Approved.

No need to test @ShastriPaturi ! That's what the CI is for.

* filter offline ensemble * fixed py norm * point to public oops * update for oops 2258, point ufo to dev * reverted ufo branch and soca obs list * wrong date in yaml

* Prepare GTS snow depth observations for JEDI-based Land DA (#541) * Initiate to process the gts bufr data into the ioda format. * Change the io paths for matching the global-workflow. * Modify the ctest for the bufr2ioda yaml files. * Filtering of the offline ensemble members, and more ... (#539) * filter offline ensemble * fixed py norm * point to public oops * update for oops 2258, point ufo to dev * reverted ufo branch and soca obs list * wrong date in yaml --------- Co-authored-by: Jiarui Dong <[email protected]> Co-authored-by: Guillaume Vernieres <[email protected]>

guillaumevernieres added 3 commits July 18, 2023 13:18

filter offline ensemble

86e879e

fixed py norm

6cbbc4e

point to public oops

49ad981

guillaumevernieres added the orion-GW-RT Queue for automated testing with global-workflow on Orion label Jul 18, 2023

emcbot added orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed orion-GW-RT Queue for automated testing with global-workflow on Orion labels Jul 18, 2023

emcbot added orion-GW-RT-Failed Automated testing with global-workflow failed on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Jul 18, 2023

guillaumevernieres requested review from AndrewEichmann-NOAA and ShastriPaturi July 18, 2023 21:40

update for oops 2258, point ufo to dev

21163e7

guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion and removed orion-GW-RT-Failed Automated testing with global-workflow failed on Orion labels Jul 21, 2023

emcbot added orion-GW-RT-Failed Automated testing with global-workflow failed on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Jul 21, 2023

emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Jul 21, 2023

reverted ufo branch and soca obs list

4cb9f04

guillaumevernieres removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera orion-GW-RT-Failed Automated testing with global-workflow failed on Orion labels Jul 21, 2023

emcbot removed the orion-GW-RT Queue for automated testing with global-workflow on Orion label Jul 21, 2023

emcbot added orion-GW-RT-Failed Automated testing with global-workflow failed on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Jul 21, 2023

emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Jul 21, 2023

guillaumevernieres added orion-GW-RT Queue for automated testing with global-workflow on Orion and removed orion-GW-RT-Failed Automated testing with global-workflow failed on Orion labels Jul 21, 2023

guillaumevernieres marked this pull request as ready for review July 21, 2023 20:03

emcbot added orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed orion-GW-RT Queue for automated testing with global-workflow on Orion labels Jul 21, 2023

emcbot added orion-GW-RT-Failed Automated testing with global-workflow failed on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Jul 21, 2023

guillaumevernieres added orion-GW-RT Queue for automated testing with global-workflow on Orion and removed orion-GW-RT-Failed Automated testing with global-workflow failed on Orion labels Jul 24, 2023

emcbot added orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed orion-GW-RT Queue for automated testing with global-workflow on Orion labels Jul 24, 2023

emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Jul 24, 2023

AndrewEichmann-NOAA approved these changes Jul 24, 2023

View reviewed changes

ShastriPaturi approved these changes Jul 24, 2023

View reviewed changes

guillaumevernieres merged commit 3b2eb40 into develop Jul 24, 2023
25 checks passed

guillaumevernieres deleted the feature/filter_ens_B branch August 3, 2023 20:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Filtering of the offline ensemble members, and more ... #539

Filtering of the offline ensemble members, and more ... #539

guillaumevernieres commented Jul 18, 2023

emcbot commented Jul 18, 2023

guillaumevernieres commented Jul 18, 2023

guillaumevernieres commented Jul 21, 2023 •

edited

Loading

emcbot commented Jul 21, 2023

emcbot commented Jul 21, 2023

emcbot commented Jul 21, 2023

emcbot commented Jul 21, 2023

emcbot commented Jul 21, 2023

guillaumevernieres commented Jul 21, 2023

emcbot commented Jul 24, 2023

AndrewEichmann-NOAA left a comment

ShastriPaturi left a comment

guillaumevernieres commented Jul 24, 2023

Filtering of the offline ensemble members, and more ... #539

Filtering of the offline ensemble members, and more ... #539

Conversation

guillaumevernieres commented Jul 18, 2023

emcbot commented Jul 18, 2023

guillaumevernieres commented Jul 18, 2023

guillaumevernieres commented Jul 21, 2023 • edited Loading

emcbot commented Jul 21, 2023

emcbot commented Jul 21, 2023

emcbot commented Jul 21, 2023

emcbot commented Jul 21, 2023

emcbot commented Jul 21, 2023

guillaumevernieres commented Jul 21, 2023

emcbot commented Jul 24, 2023

AndrewEichmann-NOAA left a comment

Choose a reason for hiding this comment

ShastriPaturi left a comment

Choose a reason for hiding this comment

guillaumevernieres commented Jul 24, 2023

guillaumevernieres commented Jul 21, 2023 •

edited

Loading