Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/b2i #1275

Merged
merged 58 commits into from
Sep 27, 2024
Merged

Feature/b2i #1275

merged 58 commits into from
Sep 27, 2024

Conversation

givelberg
Copy link
Collaborator

6 bufr to ioda converters
this is second refactoring of the code.
the variables are grouped into metadata, obs and additional ioda vars
metadata is read from a yaml file (could also be a json file)
additional variables now include ocean basin
converters generate log files and have a command line option to test
the log against a reference file
Log files contain some human readable data, as well as cryptographic
hash strings computed from the variables.
Simple utilities to plot data are included.

givelberg and others added 30 commits August 5, 2024 15:19
@givelberg
working b2i 6 scripts with tests; initial refactoring of bufr2ioda_in…
8e9cc52 
…situ
@givelberg
Update ARGO.py
56c0cef
@givelberg
Update ALTKOB.py
e1b3d74
@givelberg
Update BATHY.py
a15bc59
@givelberg
Update GLIDER.py
3f03aa9
@givelberg
Update MARINEMAMMAL.py
4d15eb9
@givelberg
Update TESAC.py
4bc9a6b
@givelberg
Update TRKOB.py
a995f15
@givelberg
Update XBTCTD.py
15c05ab
@givelberg
Update altkob_ioda_variables.py
72d787b
@givelberg
Update argo_ioda_variables.py
54c11c9
@givelberg
Update argo_ioda_variables.py
441419d
@givelberg
Update bathy_ioda_variables.py
9ec439f
@givelberg
Update glider_ioda_variables.py
455e9a3
@givelberg
Update marinemammal_ioda_variables.py
c8d6c5e
@givelberg
Update tesac_ioda_variables.py
68366f7
@givelberg
Update trkob_ioda_variables.py
bbdae7c
@givelberg
Update xbtctd_ioda_variables.py
8487acc
@givelberg
Update bufr2ioda_config.py
a37155c
@givelberg
Update bufr2ioda_converter.py
e158b43
@givelberg
Update util.py
f57bcee
@givelberg
Update altkob_ioda_variables.py
a0f8110
@givelberg
Update util.py
f7ca21d
@givelberg
Update ioda_variables.py
0ad2baa
@givelberg
Update bufr2ioda_config.py
927fd52
@givelberg
Update util.py
7f13f69
@givelberg
Update util.py
24c029f
@givelberg
Update ioda_variables.py
62f7145
@givelberg
Update trkob_ioda_variables.py
0ff9e9b
@givelberg
Update ioda_variables.py
8956f7a
@guillaumevernieres guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Sep 13, 2024
@guillaumevernieres
Copy link
Contributor

@givelberg , your branch is behind develop and the code norm is failing ...

@guillaumevernieres
Copy link
Contributor

I'll run the test manually @givelberg , since I broke everything earlier today ...

@guillaumevernieres
Copy link
Contributor

I'll run the test manually @givelberg , since I broke everything earlier today ...

great, the build failed. Probably an issue on my side. I'll run the ci and cherry pick the ctests relevant to your changes @givelberg .

@guillaumevernieres guillaumevernieres added the hera-GW-RT Queue for automated testing with global-workflow on Hera label Sep 18, 2024
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Sep 18, 2024
guillaumevernieres
guillaumevernieres requested changes Sep 18, 2024
View reviewed changes
Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments.

test/soca/CMakeLists.txt Outdated Show resolved Hide resolved
test/soca/CMakeLists.txt Outdated Show resolved Hide resolved
test/soca/CMakeLists.txt Outdated Show resolved Hide resolved
@emcbot
Copy link

emcbot commented Sep 19, 2024

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Wed Sep 18 20:21:03 UTC 2024 on hfe02
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Wed Sep 18 21:05:53 UTC 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Thu Sep 19 00:55:44 UTC 2024
Tests: 65% tests passed, 16 tests failed out of 46
	1892 - test_gdasapp_WCDA-3DVAR-C48mx500_gdasfcst_202103241200 (Failed)
	1893 - test_gdasapp_WCDA-3DVAR-C48mx500_gdasprepoceanobs_202103241800 (Failed)
	1902 - test_gdasapp_atm_jjob_var_init (Failed)
	1903 - test_gdasapp_atm_jjob_var_run (Failed)
	1904 - test_gdasapp_atm_jjob_var_inc (Failed)
	1905 - test_gdasapp_atm_jjob_var_final (Failed)
	1906 - test_gdasapp_atm_jjob_ens_init (Failed)
	1912 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/role.jedipara/CI/GDASApp/workflow/PR/1275/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Sep 19, 2024
@guillaumevernieres guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera labels Sep 26, 2024
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Sep 26, 2024
@emcbot
Copy link

emcbot commented Sep 26, 2024

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Thu Sep 26 22:27:42 UTC 2024 on hfe02
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Thu Sep 26 23:16:18 UTC 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Thu Sep 26 23:56:04 UTC 2024
Tests: 87% tests passed, 6 tests failed out of 46
	1903 - test_gdasapp_atm_jjob_var_init (Failed)
	1904 - test_gdasapp_atm_jjob_var_run (Failed)
	1905 - test_gdasapp_atm_jjob_var_inc (Failed)
	1906 - test_gdasapp_atm_jjob_var_final (Failed)
	1907 - test_gdasapp_atm_jjob_ens_init (Failed)
	1908 - test_gdasapp_atm_jjob_ens_letkf (Failed)
Tests: see output at /scratch1/NCEPDEV/da/role.jedipara/CI/GDASApp/workflow/PR/1275/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Sep 26, 2024
guillaumevernieres
guillaumevernieres approved these changes Sep 27, 2024
View reviewed changes
Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this @givelberg , but your tests were not triggered in the hera ci.

@guillaumevernieres guillaumevernieres merged commit 2be1fcd into develop Sep 27, 2024
11 checks passed
@guillaumevernieres guillaumevernieres deleted the feature/b2i branch September 27, 2024 11:42
@guillaumevernieres guillaumevernieres mentioned this pull request Sep 27, 2024
Closed
jiaruidong2017 pushed a commit that referenced this pull request Oct 3, 2024
@givelberg @guillaumevernieres @jiaruidong2017
Feature/b2i (#1275)
1853218 
6 bufr to ioda converters
this is second refactoring of the code.
the variables are grouped into metadata, obs and additional ioda vars
metadata is read from a yaml file (could also be a json file)
additional variables now include ocean basin
converters generate log files and have a command line option to test
the log against a reference file
Log files contain some human readable data, as well as cryptographic 
hash strings computed from the variables.
Simple utilities to plot data are included.

---------

Co-authored-by: Guillaume Vernieres <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AndrewEichmann-NOAA AndrewEichmann-NOAA AndrewEichmann-NOAA left review comments

@guillaumevernieres guillaumevernieres guillaumevernieres approved these changes

@apchoiCMD apchoiCMD Awaiting requested review from apchoiCMD

@JohnSteffen-NOAA JohnSteffen-NOAA Awaiting requested review from JohnSteffen-NOAA

@KatherineLukens-NOAA KatherineLukens-NOAA Awaiting requested review from KatherineLukens-NOAA

Assignees
No one assigned
Labels
hera-GW-RT-Failed Automated testing with global-workflow failed on Hera
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@givelberg @emcbot @guillaumevernieres @AndrewEichmann-NOAA