Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GFSv16.3.22 - Remove WAFS from GFS #2591

Open
KateFriedman-NOAA opened this issue May 10, 2024 · 14 comments
Open

GFSv16.3.22 - Remove WAFS from GFS #2591

KateFriedman-NOAA opened this issue May 10, 2024 · 14 comments
Assignees
@aerorahul @KateFriedman-NOAA
Labels
production update Processing update in production

Comments

@KateFriedman-NOAA
Copy link
Member

KateFriedman-NOAA commented May 10, 2024 •
edited
Loading

Description

Remove the WAFS (formerly EMC_gfs_wafs) from the GFS, it will now be a standalone system in operations.

WAFS issue: NOAA-EMC/WAFS#23

Target version

v16.3.22

Expected workflow changes

Remove WAFS pieces:

  • jobs in ecf def
  • ecf scripts
  • wafs job, ex-scripts, and ush scripts
  • WAFS jobs from rocoto mesh
  • configs
  • set WAFSF="NO"
  • discontinue GTG checkout and -o flag for checkout (?)
@KateFriedman-NOAA KateFriedman-NOAA added the production update Processing update in production label May 10, 2024
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.4.1 - Remove WAFS from GFS GFSv16.4.0 - Remove WAFS from GFS May 10, 2024
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.4.0 - Remove WAFS from GFS GFSv16.?.? - Remove WAFS from GFS Jun 10, 2024
@KateFriedman-NOAA
Copy link
Member Author

Timeline update from @aerorahul :

WAFS has a code delivery in Q1FY25 with an implementation in Q2FY25.
The GFS will be delivered and implemented at the same time removing WAFS from the GFS.

@KateFriedman-NOAA
Copy link
Member Author

KateFriedman-NOAA commented Jul 8, 2024 •
edited
Loading

This will be v16.4.0

@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.?.? - Remove WAFS from GFS GFSv16.4.0 - Remove WAFS from GFS Jul 8, 2024
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.4.0 - Remove WAFS from GFS GFSv16.3.18 - Remove WAFS from GFS Sep 4, 2024
@KateFriedman-NOAA
Copy link
Member Author

Created branch: https://github.com/NOAA-EMC/global-workflow/tree/release/gfs.v16.3.18

@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.18 - Remove WAFS from GFS GFSv16.3.19 - Remove WAFS from GFS Sep 16, 2024
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Sep 16, 2024
Merged
7 tasks
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 16, 2024
@KateFriedman-NOAA
Update release notes for WAFS removal from GFS
563d542 
Also update gfs_ver=v16.3.19

Refs NOAA-EMC#2591
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Sep 16, 2024
Merged
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 18, 2024
@KateFriedman-NOAA
Update release notes to mention new file name
fc47df6 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 19, 2024
@KateFriedman-NOAA
Update release notes based on feedback from WAFS
355a6af 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 20, 2024
@KateFriedman-NOAA
Updates to file list in release notes
98b902b 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit that referenced this issue Sep 20, 2024
@KateFriedman-NOAA
Update release notes for WAFS removal from GFS (#2927)
d661f1f 
Update the release notes for the WAFS removal from the GFS.
Files and settings that are removed are noted.

Also update `gfs_ver=v16.3.19`.

Refs #2591
@KateFriedman-NOAA
Copy link
Member Author

KateFriedman-NOAA commented Sep 24, 2024 •
edited
Loading

Just renamed the release branch to release/gfs.v16.3.21 to match the new version in ops. The ARFC for GFSv16.3.19 took the version number and the obsproc/v1.3 update will take GFSv16.3.20. Will update release notes and internal version shortly, need to prep release branch for ARFC GFSv16.3.19 first.

@RussTreadon-NOAA
Copy link
Contributor

Thank you @KateFriedman-NOAA for your quick action.

DA team members are meeting this afternoon to make a final decision as to what course of action to take. Submitting an ARFC to place PlanetiQ data in monitor mode is one option. Another option is to let the PlanetiQ data flow into and be assimilated by the GFS. This later option requires no EIB action.

This issue is opened as a "heads up" to EIB in case the DA team goes with the first option.

@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Sep 24, 2024
Closed
10 tasks
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.19 - Remove WAFS from GFS GFSv16.3.21 - Remove WAFS from GFS Sep 24, 2024
KateFriedman-NOAA added a commit that referenced this issue Oct 1, 2024
@KateFriedman-NOAA
Update WAFS branch to v16.3.21
c019ab5 
Refs #2591
@KateFriedman-NOAA
Copy link
Member Author

Synced the release/gfs.v16.3.21 branch with dev/gfs.v16 after the GFSv16.3.19 branch was merged into dev/gfs.v16. Also updated the release branch for the new version v16.3.21.

KateFriedman-NOAA added a commit that referenced this issue Oct 9, 2024
@KateFriedman-NOAA
Sync merge GFSv16.3.20 changes into GFSv16.3.21 release branch
56a2243 
Merge remote-tracking branch 'origin/release/gfs.v16.3.20' into release/gfs.v16.3.21

Refs #2591

* origin/release/gfs.v16.3.20:
  Update GSI tag to gfsda.v16.3.20 for saildrone data (#2986)
  Update WAFS tag in release notes
  Set WAFS tag back to new gfs_wafs.v6.3.3
  Revert "Remove WAFS from ops (#2889)"
  Update for obsproc/v1.3 and new AFWA global snow filename (#2957)
@KateFriedman-NOAA
Copy link
Member Author

Sync merged the recently-handed-off changes for GFSv16.3.20 into the release/gfs.v16.3.21 branch and cut hand-off tag EMC-v16.3.21.

@KateFriedman-NOAA
Copy link
Member Author

CDF submitted to NCO.

@KateFriedman-NOAA
Copy link
Member Author

WAFS CDF was also just submitted to NCO. EVS will also be submitting a companion CDF soon.

@KateFriedman-NOAA
Copy link
Member Author

@YaliMao-NOAA NCO has requested that we increase the walltime of the wafs 0p25 blending job to 40mins in the new WAFSv7.0.0 package. NCO made the change locally in the GFSv16.3.20 install:

I increased the walltime of 
/lfs/h1/ops/para/packages/gfs.v16.3.20/ecf/scripts/gfs/atmos/post_processing/grib2_wafs/jgfs_atmos_wafs_blending_0p25.ecf
from 30 to 40 minutes to match a change Wei made in gfs.v16.3.18. - Diane Stokes

Please make the same increase in the WAFSv7.0.0 package and recut the WAFS hand-off tag. Thanks!

@YaliMao-NOAA
Copy link
Contributor

YaliMao-NOAA commented Oct 11, 2024 •
edited
Loading

@KateFriedman-NOAA Thank you for the notification.
I am going to check whether it's necessary to increase the walltime in wafsv7.0.0 since blending process is parallel in wafsv7.0.0

@KateFriedman-NOAA
Copy link
Member Author

@YaliMao-NOAA Good point, sounds good, thanks!

KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Oct 30, 2024
@KateFriedman-NOAA
Update enkfgdas_update job resource/env settings
529ce82 
- Add OMP_PROC_BIND=true and update APRUN_ENKF to use "--cpu-bind core"
instead of "--cpu-bind depth" in WCOSS2.env.
- Adjust npe (280) and thread (16) values.

Updates come from testing by NCO/GDIT and produce a more stable runtime
and bitwise identical outputs to ops.

NCO requests that we include these updates in the GFSv16.3.21 package.

Refs NOAA-EMC#2591
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Oct 30, 2024
Merged
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Oct 30, 2024
@KateFriedman-NOAA
Add enkfgdas_update resource/env change to release notes
3e23ead 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit that referenced this issue Oct 30, 2024
@KateFriedman-NOAA
Update enkfgdas_update job resource/env settings (#3047)
ab8e453 
Updates for the `enkfgdas_update` job from testing by NCO/GDIT and EMC.
They produce a more stable runtime and bitwise identical outputs to ops.

Add `export OMP_PROC_BIND=true` and update `APRUN_ENKF` to use
`--cpu-bind core` instead of `--cpu-bind depth` in WCOSS2.env. 
Also, adjust npe (280) and thread (16) values.

NCO requests that we include these updates in the GFSv16.3.21 package.

Refs #2591
KateFriedman-NOAA added a commit that referenced this issue Oct 30, 2024
@KateFriedman-NOAA
Merge remote-tracking branch 'origin/release/gfs.v16.3.20' into relea…
bc1dd7d 
…se/gfs.v16.3.21

Refs #2591

* origin/release/gfs.v16.3.20:
  Add sfcprep to config.resources.nco.static (#3031)
@KateFriedman-NOAA
Copy link
Member Author

Merge synced the release/gfs.v16.3.20 changes into the release/gfs.v16.3.21 branch and recut the EMC-v16.3.21 tag.

@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.21 - Remove WAFS from GFS GFSv16.3.22 - Remove WAFS from GFS Nov 8, 2024
@KateFriedman-NOAA
Copy link
Member Author

This is now v16.3.22. The new ops change documented in issue #3082 will go in before this and pushes this to v16.3.22.

KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Nov 13, 2024
@KateFriedman-NOAA
Update GFS version to v16.3.22
0de673d 
Refs NOAA-EMC#2591
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Nov 13, 2024
Merged
KateFriedman-NOAA added a commit that referenced this issue Nov 13, 2024
@KateFriedman-NOAA
Update GFS version to v16.3.22 (#3095)
9861ce0 
Update the version for the "WAFS removal from GFS" release branch to v16.3.22.

Refs #2591
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aerorahul aerorahul

@KateFriedman-NOAA KateFriedman-NOAA

Labels
production update Processing update in production
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@aerorahul @RussTreadon-NOAA @KateFriedman-NOAA @YaliMao-NOAA