Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: multiarch build by using crosscompilation #336

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ykulazhenkov
Copy link
Contributor

@ykulazhenkov ykulazhenkov commented Nov 25, 2024

What this PR does / why we need it:

Use quay.io/centos/centos:stream9 with the –platform=$BUILDPLATFORM flag,
in this case the image is pulled for the builder host's current architecture.
The BUILDOS and BUILDARCH args are used to download the right go binary for
the build platform.
Cross-compilation occurs in the builder image using TARGETOS and TARGETARCH build arguments to determine the target OS/arch.
These args are set automatically by the multiarch-build process (with docker buildx).
The final container image (registry.access.redhat.com/ubi9/ubi-minimal) is pulled for the correct target architecture, such as amd64 or arm64.

This update fixes support for multiarch builds and speeds up image building, as cross-compilation is faster than compiling on a non-native platform.

Special notes for your reviewer:

Fixes support for multiarch builds added in #307

Multiarch build was broken by #319

Release note:

NONE

@kubevirt-bot kubevirt-bot added dco-signoff: no Indicates the PR's author has not DCO signed all their commits. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 25, 2024
@kubevirt-bot
Copy link
Collaborator

Hi @ykulazhenkov. Thanks for your PR.

I'm waiting for a k8snetworkplumbingwg member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ykulazhenkov ykulazhenkov force-pushed the pr-fix-multiarch-build-golang-img branch from bb3536d to ce4b397 Compare November 25, 2024 10:47
@kubevirt-bot kubevirt-bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. release-note-none Denotes a PR that doesn't merit a release note. and removed dco-signoff: no Indicates the PR's author has not DCO signed all their commits. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Nov 25, 2024
@ykulazhenkov
Copy link
Contributor Author

@phoracek @SchSeba please take a look on this one

@ykulazhenkov ykulazhenkov mentioned this pull request Nov 25, 2024
@ykulazhenkov
Copy link
Contributor Author

@@ -27,9 +27,5 @@ jobs:
with:
context: .
push: false
# no need to explicitly set goarch,
# correct arch will be selected for each build platform
build-args: |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to set this anymore. Default for the goarch= build-arg is empty string.

Copy link
Member

@oshoval oshoval left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks

cmd/Dockerfile Outdated
@@ -1,35 +1,37 @@
FROM quay.io/centos/centos:stream9 as builder
FROM --platform=$BUILDPLATFORM docker.io/golang:1.23 AS builder
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go ver is hardcoded here instead taking from go.mod
i believe we can live with it, just raising for awareness in case you think otherwsie

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is totally acceptable (and even beneficial) to use a newer version of Golang to build the binary. The 1.23 tag will provide us with the latest patch release of 1.23, including all fixes to known CVEs. When Go 1.24 is released, we can update the builder image to the latest version but keep 1.22 in the go.mod file. This approach allows us to be flexible about the compiler version (for downstream consumers who rebuild the image), while the “official” image build always uses a “fresh” version.

Copy link
Member

@oshoval oshoval Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It requires changing on more than one place, which isnt what we strive to do usually,
only rarely we need a different compiler than a language level imho

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's use go.mod if this is standard practice in kubevirt-related projects. I updated PR to rollback to explicit go download.

@ykulazhenkov ykulazhenkov force-pushed the pr-fix-multiarch-build-golang-img branch from ce4b397 to 4222e29 Compare November 25, 2024 14:11
@ykulazhenkov ykulazhenkov changed the title Fix: multiarch build by moving the binaries build stage to the golang image Fix: multiarch build by using crosscompilation Nov 25, 2024
Copy link
Member

@oshoval oshoval left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome thank you

@kubevirt-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: oshoval, ykulazhenkov
Once this PR has been reviewed and has the lgtm label, please assign alonakaplan for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


COPY . .
RUN go mod download
Copy link
Member

@oshoval oshoval Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious, why we need this suddenly ?

Copy link
Contributor Author

@ykulazhenkov ykulazhenkov Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First, we copy go.mod and go.sum to download dependencies explicitly (go mod download), then we copy other code files. This optimizes image rebuilds when only source code changes, allowing Docker to use cached dependencies.
This construction is employed to prevent re-downloading the dependencies whenever possible.
This is pretty common practice. For example, kubebuilder will generate Dockerfile that use such optimization https://github.com/kubernetes-sigs/kubebuilder/blob/cbc6e383c342f1337ab37ee4aa0755957a01f9c7/pkg/plugins/golang/v4/scaffolds/internal/templates/dockerfile.go#L46

@ashokpariya0
Copy link

@ykulazhenkov We should update the GOARCH ?= amd64 line (as seen in this Makefile) to:
GOARCH ?= $(shell uname -m | sed 's/x86_64/amd64/')
This change will dynamically set GOARCH based on the host architecture, converting x86_64 to amd64 as needed.

@oshoval changes look good to me, assuming the idea is to provide multiplatform support only through GitHub Actions, exclusively for the Docker container runtime, and not through make commands or for the Podman container runtime.

@oshoval
Copy link
Member

oshoval commented Nov 28, 2024

@oshoval changes look good to me, assuming the idea is to provide multiplatform support only through GitHub Actions, exclusively for the Docker container runtime, and not through make commands or for the Podman container runtime.

thanks

at the end we will do need also images that are multi platform though isnt it? (as the issue / PRs you work on)
it is fine to do so in steps, as long as those steps can co exists

@ashokpariya0
Copy link

@oshoval changes look good to me, assuming the idea is to provide multiplatform support only through GitHub Actions, exclusively for the Docker container runtime, and not through make commands or for the Podman container runtime.

thanks

at the end we will do need also images that are multi platform though isnt it? (as the issue / PRs you work on) it is fine to do so in steps, as long as those steps can co exists

Yes, we can consider adding multiplatform support using a Makefile(using make commands) for both Docker and Podman in the future. This would allow us to build images on our local machine, in a specific environment, or within any CI system, giving us more control over the build process and environment.
For Podman, there are no additional dependencies required to build multiplatform images, while for Docker, we would need the docker buildx tool.

Use quay.io/centos/centos:stream9 with the –platform=$BUILDPLATFORM flag,
in this case the image is pulled for the builder host's current architecture.
The BUILDOS and BUILDARCH args are used to download the right go binary for
the build platform.
Cross-compilation occurs in the builder image using TARGETOS and TARGETARCH build arguments to determine the target OS/arch.
These args are set automatically by the multiarch-build process (with docker buildx).
The final container image (registry.access.redhat.com/ubi9/ubi-minimal) is pulled for the correct target architecture, such as amd64 or arm64.

This update fixes support for multiarch builds and speeds up image building, as cross-compilation is faster than compiling on a non-native platform.

Signed-off-by: Yury Kulazhenkov <[email protected]>
@ykulazhenkov ykulazhenkov force-pushed the pr-fix-multiarch-build-golang-img branch from 4222e29 to d6a30e9 Compare November 28, 2024 15:50
@ykulazhenkov
Copy link
Contributor Author

We should update the GOARCH ?= amd64 line (as seen in this Makefile) to:
GOARCH ?= $(shell uname -m | sed 's/x86_64/amd64/')
This change will dynamically set GOARCH based on the host architecture, converting x86_64 to amd64 as needed.

done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dco-signoff: yes Indicates the PR's author has DCO signed all their commits. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. release-note-none Denotes a PR that doesn't merit a release note.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants