Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Floating tag support #1295

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+16 −3
Open

Floating tag support #1295

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3009db4
Floating tag support
mthalman Dec 11, 2024
3562aea
break up lines to satisfy linter
mthalman Dec 11, 2024
e434503
Updates from feedback
mthalman Dec 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions lifecycle.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,24 @@ Note: this content should be moved to another location as it is not lifecycle re

Build and test images are referenced in repo infra files, across a variety of `main` and `release/*` branches. Updating these references is a multi-step detail-oriented task. It is a pain, but necessary.

We use version specific references in infra to ensure that our CI builds are reliable. One can imagine using floating OS tags (such as `debian-oldest` and `debian-latest`), however such an approach would be guaranteed to break our build. We know that since we often see build and test breaks that need addressing in PRs where we update build and test images.
Several types of tag styles are provided:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would these tags be created regardless if they are being used or would they be created on demand? I question if they are going to be used for every distro/image variant.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also lean towards on-demand. In fact, I'm not sure we want all distros being tested with floating tags.

Copy link
Member

@richlander richlander Dec 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several types

Several or two?

* version-specific: The tag name includes the distro's version. Example: `alpine-3.21-helix-amd64`
* floating: The tag name has no version. Instead it is routinely updated to reference a new version as the distro's lifecycle progresses. There are two sub-types of floating tags:
* latest: References the latest validated version of the distro. Example: `alpine-latest-helix-amd64`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about this tags being unbounded. At some point in a product release, it is time to turn up the dial on stability and to not move OS versions forward. In a year, we may want to stop updating .NET tags for that reasons. Similarly, "oldest" for .NET 8 and .NET 10 do not match, now. I've already moved .NET 10 to use Alpine 3.21 in dotnet/runtime as "oldest".

I'm a big fan of the model we used for our build images. They are version-specific. I think we should use the same model for build and test images. That approach would be significantly simplifying for teams. Just one model to understand.

https://github.com/dotnet/runtime/blob/main/docs/workflow/using-docker.md#the-official-runtime-docker-images

I'm also not convinced that we need oldest and latest. I think we should just offer one image variant to teams, just like we do for build images. If they need more versions, they can go with DIY.

In dotnet/runtime, the CoreCLR build targets the trailing edge of OS versions we support. Libraries biases to newest.

I think the best model is to:

  • Continue to use runtime to hold the baton on "oldest"
  • Use libraries to validate that "newest" is OK.
  • Push other repos (that use the floating tag) to newest after that, with a single tag (that doesn't say "newest").

Today, these tags are (effectively) exclusively referenced in our own infra. As we move to VMR/UB, there may be more users of these images. We need to be cognizant of that and ensure we don't break them when they attempt to build and test old .NET versions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a big fan of the model we used for our build images. They are version-specific. I think we should use the same model for build and test images.

That gets outside the scope of what is being addressed here. That's going to move us into producing new images, not just tagging changes. I would encourage you to create a separate issue for this.

I think the best model is to:

  • Continue to use runtime to hold the baton on "oldest"
  • Use libraries to validate that "newest" is OK.
  • Push other repos (that use the floating tag) to newest after that, with a single tag (that doesn't say "newest").

Let me check with you that I'm interpreting this correctly. This means we would have a floating tag that represents the latest version. If repos want an older version, they should use a version-specific tag. And libraries would still use a version-specific tag, targeting the latest, that we can use for vetting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why we need new images. I'm only talking about tagging.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this concrete. We should model this for .NET 8 and 10 for Alpine. As I said, I'd like to retain the ability to stop taking Alpine changes for .NET 8 late in the .NET 8 lifecycle (last 6 months).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why we need new images. I'm only talking about tagging.

I thought you were referring to using net8.0 and net9.0 tagging like we do for the build images.

Copy link
Member

@richlander richlander Dec 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. For our build images, it's completely different content. For this, I'm suggesting just different tags, all handled in manifest.json.

Imagine just before .NET 9 EOL:

  • alpine-net9.0 -- same as alpine-3.23-helix-amd64, referenced in release/9.0 branches
  • alpine-net10.0 -- same as alpine-3.24-helix-amd64, referenced in release/10.0 branches

The .NET 9 image is eventually pinned to Alpine 3.23 while the .NET 10 image keeps on marching forward. The reason is that .NET 9 is in maintenance and we value high stability and low cost at that point.

We don't need any special tools for this. We could do this now.

Copy link
Member

@richlander richlander Dec 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some thoughts on users of these tags:

  • Should only update tags (from Alpine 3.23 to 3.24, for example) in the middle weeks of the month. If Patch Tuesday is the B week, then we should update change these tags on a C or D week, ideally also on a Tuesday. We don't want to disrupt servicing activities.
  • If a change breaks one or a few teams, it is their responsibility to fix their code. That's part of the terms of use of these tags. If a change breaks most users, then the tag change should be reverted.
  • The tags should only be updated after they test cleanly in runtime, otherwise, we're in YOLO mode.
  • Not appropriate for repos that have -staging branches, like https://github.com/dotnet/runtime/tree/release/8.0-staging. That pattern places a high premium on a commit encoding/representing all aspects of code and config. Floating tags are antithetical to that approach.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Patch Tuesday is the B week, then we should update change these tags on a C or D week, ideally also on a Tuesday. We don't want to disrupt servicing activities.

Timing is a good point. Wouldn't we want to make these prior to code complete?

Copy link
Member

@richlander richlander Dec 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Perhaps that's just the C week. I don't know what that timing is so really just bringing up the topic. I am certain the B week is a bad choice. There is always the chance that we'll regret something we shipped and need to make a quick update in the B week.

* oldest: References the oldest supported version of the distro. Example: `alpine-oldest-helix-amd64`

At times, it may be necessary to use a [fixed image reference](https://github.com/dotnet/runtime/pull/110199#discussion_r1859075989) for build reliability. If this is ever done, a tracking issue should be created (before the PR is merged) so that we remember to resolve the underlying issue and update the image reference.
FLoating tags are beneficial for repos that are not susceptible to breaking changes that occur from new distro versions because the source that references the tag doesn't need to be updated in order to make use of the new version.
Conversely, some repos may be susceptible to distro breaking changes in which case the version-specific tags should be used ([dotnet/runtime](https://github.com/dotnet/runtime) is an example of such a repo).

There is a vetting process before a `latest` floating tag gets moved to a newer distro version:

1. Version-specific tags for the new distro version are provided.
1. The [dotnet/runtime](https://github.com/dotnet/runtime) repo is updated to reference the new version-specific tags.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't there going to be cases where images aren't referenced by the runtime?

I don't like this vetting process in a way. If vetting is required you shouldn't be using the version specific tags. I tend to learn towards the .NET Docker Alpine floating tag policy. The new distro is made available one month, the next month the floating tag is updated. This is a more scalable model.

Copy link
Member

@richlander richlander Dec 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works as a policy. If we found an issue in runtime, we'd pause roll-out in dotnet-docker. We'd do the same with this system. It does mean that we'll have to be intentional w/ensuring we get coverage in runtime pretty quickly.

As long as the new system is pause-able, it doesn't need vetting.

1. Once runtime has successfully taken a dependency on the new version, the `latest` floating tag is updated to reference that version.

At times, it may be necessary to use a [fixed image reference](https://github.com/dotnet/runtime/pull/110199#discussion_r1859075989) for build reliability.
This is done by referencing the digest of the specific image that is needed (e.g. `mcr.microsoft.com/dotnet-buildtools/prereqs@sha256:56feee03d202e008a98f3c92784f79f3f0b3a512074f7f8ee2b1ba4ca4c08c6e`).
If this is ever done, a tracking issue should be created (before the PR is merged) so that we remember to resolve the underlying issue and update the image reference.

The following locations are examples of infra that gets updated when new images are available.

Expand Down
Loading