Progress Report 6–10 June

[esheehan-gsl]

🚴🏻 Progress

• Began discussion about contents of the diag files (Getting Started with the NetCDF Diag Files #25)

😖 Regressions

• Out sick most of the week

💬 To Discuss

• Getting Started with the NetCDF Diag Files #25

📆 Next Week

• API for producing distributions of observations - forecast values from NetCDF diag files

[ian-noaa]

🚴🏻 Progress

• We can now deploy images to VLab's AWS container registry with GitHub Actions (Configure CI to deploy to ECR #29)
• CI will not deploy to ECR if Trivy detects security issues (Block the deploy job if Trivy finds vulnerabilties #30)

😖 Regressions

• None

💬 To Discuss

• Where we want to commit our Kubernetes manifests (pending discussion with VLab on 6/14)

📆 This Week

• Work with VLab on deploying images into the Kubernetes cluster
• Help with API for NetCDF diag files

[esheehan-gsl]

How are things looking for getting the images deployed to k8s? I’d love to be able to demo from the actual AWS infrastructure on 27 June.

[ian-noaa]

Just got off a meeting with Bryan and it sounds like we'll be able to get started on it. (They had to chat with their security folks to figure out how they would keep vulnerable images out of the cluster.) I'd say it's 50/50 that we'll have things up and running by 27 June.

Currently they're looking at using an event from their scanning tool to trigger a lambda function for the deploy. They're focused on the main-tagged images for now and will look into preview environments for us once that's up and going.

It sounds like we're going to set up a second git repo (they're debating between GitHub & VLab) for the Kubernetes manifests so we have a log of what gets changed in the cluster. Hopefully we'll be able to also use the manifests in there to deploy locally if we want.

[esheehan-gsl]

How much do we care about this second repo for the manifests? Will we be working with it a lot—i.e. is there a reason for us to prefer GitHub over VLab? Is it something we could fold into this monorepo?

[ian-noaa]

Good question.

We might be working with it when adding new services but I don't think we'll be working directly with it day-by-day. There will most likely be automated commits from the AWS infrastructure to create & destroy preview environments for our PRs or to bump image tags when our CI pushes new images.

These cluster config repos can be used to deploy multiple applications into a cluster (e.g. - the unified-graphics apps and services like Prometheus for monitoring, Jaeger for tracing, etc...) so most recommendations are to separate it from the application code. I think keeping it separate from the app code will also cut down on noise in the app repo history from infrastructure-related commits so I'm okay with it at the moment.

Ideally these repos are owned by the infra teams and the application teams don't have to fiddle with them too much.