Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

updating pythonpath in JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP to match changes to ioda #1724

Merged
merged 1 commit into from
Jul 7, 2023
Merged

updating pythonpath in JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP to match changes to ioda #1724

merged 1 commit into from
Jul 7, 2023

Conversation

AndrewEichmann-NOAA
Copy link
Contributor

@AndrewEichmann-NOAA AndrewEichmann-NOAA commented Jul 5, 2023 •
edited
Loading

Description

Updates PYTHONPATH in jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP to reflect changes in ioda as described here:
JCSDA-internal/ioda-converters#1238

Also addressed by NOAA-EMC/GDASApp#519 and NOAA-EMC/GDASApp#518

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Tested with WCDA cycling by @ShastriPaturi

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

@AndrewEichmann-NOAA
Copy link
Contributor Author

AndrewEichmann-NOAA commented Jul 5, 2023 •
edited
Loading

FYI @CoryMartin-NOAA @guillaumevernieres @ShastriPaturi

CoryMartin-NOAA
CoryMartin-NOAA reviewed Jul 6, 2023
View reviewed changes
jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
@@ -35,7 +35,8 @@ RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} generate_com -rx \

# Add UFSDA to PYTHONPATH
ufsdaPATH="${HOMEgfs}/sorc/gdas.cd/ush/"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${ufsdaPATH}"
pyiodaPATH="${HOMEgfs}/sorc/gdas.cd/build/lib/python3.7/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to be careful about prescribing a python version here I think... but this is probably the best solution for now, sadly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's why I included you. I wasn't sure if we could do the same thing here as in GDASApp.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had the same concerns. @aerorahul?

guillaumevernieres
guillaumevernieres approved these changes Jul 6, 2023
View reviewed changes
Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AndrewEichmann-NOAA

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion labels Jul 6, 2023
@emcbot emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Jul 6, 2023
@emcbot
Copy link

emcbot commented Jul 6, 2023

Automated global-workflow Testing Results:
Machine: Orion
Start: Thu Jul 6 16:44:19 CDT 2023 on Orion-login-1.HPC.MsState.Edu

Checkout: SUCCESS
Checkout: Completed at Thu Jul 6 16:46:06 CDT 2023
Build: SUCCESS
Build: Completed at Thu Jul 6 18:03:54 CDT 2023
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:00 CDT 2023 for experiment C48_S2S
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:05 CDT 2023 for experiment C96_atm3DVar
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:10 CDT 2023 for experiment C96C48_hybatmDA

@emcbot
Copy link

emcbot commented Jul 6, 2023

Automated global-workflow Testing Results:
Machine: Orion
Start: Thu Jul 6 16:44:19 CDT 2023 on Orion-login-1.HPC.MsState.Edu

Checkout: SUCCESS
Checkout: Completed at Thu Jul 6 16:46:06 CDT 2023
Build: SUCCESS
Build: Completed at Thu Jul 6 18:03:54 CDT 2023
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:00 CDT 2023 for experiment C48_S2S
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:05 CDT 2023 for experiment C96_atm3DVar
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:10 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: SUCCESS
Experiment C48_S2S Completed at Thu Jul 6 18:45:16 CDT 2023
with 18 successfully completed jobs

@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Jul 7, 2023
@emcbot
Copy link

emcbot commented Jul 7, 2023

Automated global-workflow Testing Results:
Machine: Orion
Start: Thu Jul 6 16:44:19 CDT 2023 on Orion-login-1.HPC.MsState.Edu

Checkout: SUCCESS
Checkout: Completed at Thu Jul 6 16:46:06 CDT 2023
Build: SUCCESS
Build: Completed at Thu Jul 6 18:03:54 CDT 2023
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:00 CDT 2023 for experiment C48_S2S
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:05 CDT 2023 for experiment C96_atm3DVar
Created experiment: SUCCESS
Case setup: Completed at Thu Jul 6 18:04:10 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: SUCCESS
Experiment C48_S2S Completed at Thu Jul 6 18:45:16 CDT 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA Terminated: FAILED
Experiment C96C48_hybatmDA Terminated with 1 tasks failed at Thu Jul 6 20:06:09 CDT 2023
Error logs:
/work2/noaa/stmp/GFS_CI_ROOT/PR/1724/RUNTESTS/C96C48_hybatmDA/COMROT/C96C48_hybatmDA/logs/2021122100/gdasvrfy.log

@emcbot emcbot added CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Jul 7, 2023
@emcbot
Copy link

emcbot commented Jul 7, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul  7 00:49:07 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul  7 00:53:49 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul  7 01:59:28 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:33 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:36 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:38 UTC 2023 for experiment C96_atm3DVar

@emcbot
Copy link

emcbot commented Jul 7, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul  7 00:49:07 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul  7 00:53:49 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul  7 01:59:28 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:33 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:36 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:38 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Fri Jul  7 02:42:09 UTC 2023
with 18 successfully completed jobs

@emcbot
Copy link

emcbot commented Jul 7, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul  7 00:49:07 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul  7 00:53:49 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul  7 01:59:28 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:33 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:36 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:38 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Fri Jul  7 02:42:09 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Fri Jul  7 04:21:07 UTC 2023
with 148 successfully completed jobs

@emcbot
Copy link

emcbot commented Jul 7, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul  7 00:49:07 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul  7 00:53:49 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul  7 01:59:28 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:33 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:36 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:38 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Fri Jul  7 02:42:09 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Fri Jul  7 04:21:07 UTC 2023
with 148 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Fri Jul  7 05:03:16 UTC 2023
with 86 successfully completed jobs

@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Jul 7, 2023
@emcbot
Copy link

emcbot commented Jul 7, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul  7 00:49:07 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul  7 00:53:49 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul  7 01:59:28 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:33 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:36 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul  7 01:59:38 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Fri Jul  7 02:42:09 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Fri Jul  7 04:21:07 UTC 2023
with 148 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Fri Jul  7 05:03:16 UTC 2023
with 86 successfully completed jobs

@WalterKolczynski-NOAA
Copy link
Contributor

Orion issues seems to be the same gdasvrfy one we've been seeing a lot in CI tests recently.

@AndrewEichmann-NOAA
Copy link
Contributor Author

@WalterKolczynski-NOAA That appears to be an issue with cartopy trying to download a file from a compute node

@guillaumevernieres
Copy link
Contributor

@WalterKolczynski-NOAA That appears to be an issue with cartopy trying to download a file from a compute node

Different "vrfy" @AndrewEichmann-NOAA .

@AndrewEichmann-NOAA
Copy link
Contributor Author

@guillaumevernieres I have GDASApp-brain

WalterKolczynski-NOAA
WalterKolczynski-NOAA approved these changes Jul 7, 2023
View reviewed changes
Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for now unless @aerorahul objects. We can deal with having a python version in the py path later.

aerorahul
aerorahul approved these changes Jul 7, 2023
View reviewed changes
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not paying attention to any ocean DA jobs or scripts.
Its the wild west here.

@guillaumevernieres
Copy link
Contributor

I am not paying attention to any ocean DA jobs or scripts. Its the wild west here.

I'm not sure what that comment is for. Your jungle vs my wild west @aerorahul .

@aerorahul
Copy link
Contributor

I am not paying attention to any ocean DA jobs or scripts. Its the wild west here.

I'm not sure what that comment is for. Your jungle vs my wild west @aerorahul .

exactly. I meant, if it works, then good. I don't have to pay attention since we will revisit this when we bring this in.

@aerorahul aerorahul merged commit 0a17478 into NOAA-EMC:develop Jul 7, 2023
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CoryMartin-NOAA CoryMartin-NOAA CoryMartin-NOAA left review comments

@aerorahul aerorahul aerorahul approved these changes

@guillaumevernieres guillaumevernieres guillaumevernieres approved these changes

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA approved these changes

Assignees
No one assigned
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@AndrewEichmann-NOAA @emcbot @WalterKolczynski-NOAA @guillaumevernieres @aerorahul @CoryMartin-NOAA