Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and optimize ngen-related image builds #430

Merged
merged 6 commits into from
Oct 24, 2023

Conversation

robertbartel
Copy link
Contributor

Address a few problems and add some optimizations to the ngen worker, calibration worker, partitioner Docker images:

  • Move to RockyLinux 9.1 to get default support for GCC 11
    • Needed in particular to work around some current issues in the framework
  • Update version of mpich to play nicely with Fortran from GCC 11
  • Correct Boost dependency version to be recent enough for current framework requirements (now 1.82.0)
  • Modify build stages that build ngen to (typically) do a git pull just before to ensure working on most recent commit for a branch
  • Ensuring Docker stack config, which controls the overall build process, establishes the same set of ARG values for related images (ngen, ngen-calibration, etc.) so that things behave as expected/consistently and the build cache is used effectively
  • Ensure use of pip's --no-cache-dir option to optimize storage space a bit

Moving Boost dependency to match new ngen requirements, and updating
mpich version to 3.3.2 to cooperate with gcc fortran 11.
Ensuring images with similar bases pass ARGs the same.
Fixing in stage for ngen-partitioner image.
@robertbartel robertbartel added bug Something isn't working enhancement New feature or request maas MaaS Workstream labels Sep 18, 2023
Copy link
Member

@aaraney aaraney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have one question that should be addressed, but I dont think it necessarily needs to hold this up. Thanks, @robertbartel!

@@ -59,7 +59,17 @@ services:
context: ./ngen
target: rocky_ngen_build_testing
args:
REPO_URL: ${NGEN_REPO_URL?No NGen repo url configured}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are your thoughts on being more explicit here and naming these NGEN_REPO_URL, NGEN_BRANCH, and NGEN_COMMIT instead? If I were looking at a .env file that contained this information and saw REPO_URL I would be a little confused about which repo to specify.

@robertbartel robertbartel merged commit 47fbbb7 into NOAA-OWP:master Oct 24, 2023
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request maas MaaS Workstream
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants