I cannot use kind build node-image, complains about invalid directory and it true its not there

[jamiemacdonald04]

What happened:

I tried to build and it failed. See command below, no such file or directory

What you expected to happen:
A different version of k8s node to be created

How to reproduce it (as minimally and precisely as possible):
kind build node-image --type url https://dl.k8s.io/v1.30.0/kubernetes-server-linux-arm64.tar.gz

Anything else we need to know?:
This is the output from the ocmmand

Building using URL: "https://dl.k8s.io/v1.30.0/kubernetes-server-linux-arm64.tar.gz"
Starting to build Kubernetes
Downloading "https://dl.k8s.io/v1.30.0/kubernetes-server-linux-arm64.tar.gz"
Failed to build Kubernetes: open /var/folders/zr/tc5l708x4_s_dm1f4qtmpv4m0000gn/T/k8s-tar-extract-140925075/kubernetes/version: no such file or directory

Environment:

• kind version: (kind v0.25.0 go1.23.3 darwin/arm64
• Runtime info: Version: 25.0.3
• OS (e.g. from /etc/os-release): mac
• Kubernetes version: (use kubectl version): Client Version: v1.30.2
• Any proxies or other special environment settings?: No

[stmcginnis]

It looks like the format of the published tarball has changed:

$ tree /tmp/k8s-tar-extract-2532324674/kubernetes/ -L 3
/tmp/k8s-tar-extract-2532324674/kubernetes/
├── kubernetes-src.tar.gz
├── LICENSES
│   ├── LICENSE
│   ├── OWNERS
│   ├── third_party
│   │   ├── forked
│   │   ├── gimme
│   │   └── multiarch
│   └── vendor
│       ├── bitbucket.org
│       ├── cloud.google.com
│       ├── github.com
│       ├── go.etcd.io
│       ├── golang.org
│       ├── google.golang.org
│       ├── go.opencensus.io
│       ├── go.opentelemetry.io
│       ├── gopkg.in
│       ├── go.starlark.net
│       ├── go.uber.org
│       ├── k8s.io
│       └── sigs.k8s.io
└── server
    └── bin
        ├── apiextensions-apiserver
        ├── kubeadm
        ├── kube-aggregator
        ├── kube-apiserver
        ├── kube-apiserver.docker_tag
        ├── kube-apiserver.tar
        ├── kube-controller-manager
        ├── kube-controller-manager.docker_tag
        ├── kube-controller-manager.tar
        ├── kubectl
        ├── kubectl-convert
        ├── kubectl.docker_tag
        ├── kubectl.tar
        ├── kubelet
        ├── kube-log-runner
        ├── kube-proxy
        ├── kube-proxy.docker_tag
        ├── kube-proxy.tar
        ├── kube-scheduler
        ├── kube-scheduler.docker_tag
        ├── kube-scheduler.tar
        └── mounter

$ find /tmp/k8s-tar-extract-2532324674/kubernetes/ -name version
$

[BenTheElder]

It looks like the format of the published tarball has changed:

We need to file a bug with SIG Release then, the release format should not be changing.

[BenTheElder]

First we should confirm that it did change between releases, and then file an issue in kubernetes/kubernetes to track.

I.E. let's root-cause before proceeding.

[stmcginnis]

Turns out this was a release bug, fixed in kubernetes/kubernetes#125076

That fix affects v1.31.0 and later. So if you really need v1.30.0 you will need to use a local source build for kind build node-image.

[jamiemacdonald04]

Thanks Sent from my iPhoneOn 25 Nov 2024, at 17:42, Sean McGinnis ***@***.***> wrote: Turns out this was a release bug, fixed in kubernetes/kubernetes#125076 That fix affects v1.31.0 and later. So if you really need v1.30.0 you will need to use a local source build for kind build node-image. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[jamiemacdonald04]

Oh the kind example documentation points to this imageKind regards Jamie Sent from my iPhoneOn 25 Nov 2024, at 17:42, Sean McGinnis ***@***.***> wrote: Turns out this was a release bug, fixed in kubernetes/kubernetes#125076 That fix affects v1.31.0 and later. So if you really need v1.30.0 you will need to use a local source build for kind build node-image. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[BenTheElder]

That fix affects v1.31.0 and later.

1.30.0 and earlier right? In 1.31.0+ Kubernetes releases should have kubernetes/kubernetes#125076 which includes the version file in this tarball.

And yes, the kind docs have a 1.30.0 example build, so we'll want to ship a workaround.

FWIW, building from source also has the ability to produces smaller images in <1.31.0 (see push-node.sh in this repo, there's a toggle that allows disabling built-in-cloud providers, the actual code removals landed in 1.31.0), this feature will work best for 1.31.0+ even after fixing this bug.

This is a bug on our end, and the docs currently have a non-working example, so we'll have to fix that. Thanks for reporting it.

/lifecycle active

I think the question is how we can best do that, right now we allow URLs directly to the tarball, which doesn't have this for older releases.

Earlier in slack I suggested maybe we could reverse-mangle the docker tag.
I think it might be that we need to do something like:

1. If version file exists, prefer that
2. If version file does not exist, exec one of the core binaries with --version and parse the output (EDIT: which we can't do on the host because of cross-arch, but we can within the build container). This will be slower and more fragile but unblock support for older releases.

When that lands we should ship a new release with the bugfix. Maybe we can time that to align with 1.32 which releases in the coming weeks upstream.

In the meantime, unfortunately the best option will be Kubernetes source builds, which we also have docs for. (Or 1.31+ builds)

Builds from binaries are pretty new and we clearly need to iron them out better, we should've caught this regression while trying to fix using pre-release builds, that's on me.

[stmcginnis]

That fix affects v1.31.0 and later.

1.30.0 and earlier right?

The bug impacts 1.30.0 and earlier. The fix affects (fixes) v1.31.0 and later. ;)