Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache module dependencies between container builds #8400

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

kaovilai
Copy link
Member

@kaovilai kaovilai commented Nov 12, 2024

  • Cache module dependencies between container builds

Thank you for contributing to Velero!

Please add a summary of your change

Makes subsequent builds faster if only code change and no go.mod changes.

Does your change fix a particular issue?

Fixes #(issue)

Please indicate you've done the following:

@kaovilai
Copy link
Member Author

/kind changelog-not-required

@github-actions github-actions bot added kind/changelog-not-required PR does not require a user changelog. Often for docs, website, or build changes Website non-docs changes for the website labels Nov 12, 2024
@kaovilai kaovilai closed this Nov 12, 2024
@kaovilai kaovilai reopened this Nov 12, 2024
@kaovilai kaovilai marked this pull request as ready for review November 12, 2024 21:36
Copy link

codecov bot commented Nov 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.97%. Comparing base (7a51e0d) to head (1051dee).
Report is 16 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8400      +/-   ##
==========================================
+ Coverage   58.95%   58.97%   +0.02%     
==========================================
  Files         367      368       +1     
  Lines       38902    38973      +71     
==========================================
+ Hits        22933    22986      +53     
- Misses      14507    14524      +17     
- Partials     1462     1463       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

@kaovilai kaovilai force-pushed the cache-mod-deps branch 2 times, most recently from 0b3f013 to 4edd799 Compare November 14, 2024 18:35
Dockerfile Outdated Show resolved Hide resolved
@kaovilai kaovilai force-pushed the cache-mod-deps branch 2 times, most recently from 2789858 to 8781e95 Compare November 15, 2024 04:18
hack/build-restic.sh Outdated Show resolved Hide resolved
@kaovilai kaovilai force-pushed the cache-mod-deps branch 3 times, most recently from fdf8ba0 to cd4423e Compare November 15, 2024 07:29
@kaovilai

This comment was marked as outdated.

@kaovilai
Copy link
Member Author

as of e465159
Making one .go change and rebuild saves about 15s vs main branch 🤷

export BUILDX_PLATFORMS=linux/arm64; make all-containers && echo "// test cache" >> pkg/constant/constant.go && time make all-containers
...
make all-containers  0.87s user 0.90s system 2% cpu 1:15.95 total
Details

export BUILDX_PLATFORMS=linux/arm64; make all-containers && echo "// test cache" >> pkg/constant/constant.go && time make all-containers
[+] Building 0.7s (26/26) FINISHED                                                                                             kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 4.05kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.3s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.3s
 => [auth] paketobuildpacks/run-jammy-tiny:pull token for registry-1.docker.io                                                           0.0s
 => [auth] library/golang:pull token for registry-1.docker.io                                                                            0.0s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => [stage-4 1/4] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [restic-builder 1/7] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.3s
 => => transferring context: 561.56kB                                                                                                    0.3s
 => CACHED [velero-builder-base 2/6] RUN mkdir -p /output/usr/bin                                                                        0.0s
 => CACHED [velero-builder-base 3/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                      0.0s
 => CACHED [velero-builder-base 4/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                          0.0s
 => CACHED [velero-builder-base 5/6] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb go mod download                                    0.0s
 => CACHED [velero-builder-base 6/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                       0.0s
 => CACHED [velero-builder 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/veler  0.0s
 => CACHED [stage-4 2/4] COPY --from=velero-builder /output /                                                                            0.0s
 => CACHED [velero-builder-helper 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /outpu  0.0s
 => CACHED [stage-4 3/4] COPY --from=velero-builder-helper /output /                                                                     0.0s
 => CACHED [restic-builder 2/7] RUN mkdir -p /output/usr/bin && mkdir -p /build/restic                                                   0.0s
 => CACHED [restic-builder 3/7] WORKDIR /build/restic                                                                                    0.0s
 => CACHED [restic-builder 4/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         git clone   0.0s
 => CACHED [restic-builder 5/7] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                0.0s
 => CACHED [restic-builder 6/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         git apply   0.0s
 => CACHED [restic-builder 7/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         GOARM=$(ec  0.0s
 => CACHED [stage-4 4/4] COPY --from=restic-builder /output /                                                                            0.0s
 => exporting to image                                                                                                                   0.1s
 => => exporting layers                                                                                                                  0.0s
 => => exporting manifest sha256:dcfbf179655194c22da2659e05a68ea402f1d1549ff0a6b1d3ca32a73ea49333                                        0.0s
 => => exporting config sha256:c5dc14d0311b58770a32d62e5206deec2a416c8b7a70bcc7ceebd7f82217c711                                          0.0s
 => => exporting attestation manifest sha256:1647713ba73b9d510d193c64a33df1f0a1d6c66462ed12a3e86adfbd005dc436                            0.0s
 => => exporting manifest list sha256:7037efc2a8a65ed211f09b845f3dbb45008e31e7932c09c49791396521595689                                   0.0s
container: velero/velero:main
[+] Building 0.4s (24/24) FINISHED                                                                                             kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 4.05kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.1s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.1s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => [restic-builder 1/7] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 543.23kB                                                                                                    0.2s
 => [stage-4 1/4] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => CACHED [velero-builder-base 2/6] RUN mkdir -p /output/usr/bin                                                                        0.0s
 => CACHED [velero-builder-base 3/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                      0.0s
 => CACHED [velero-builder-base 4/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                          0.0s
 => CACHED [velero-builder-base 5/6] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb go mod download                                    0.0s
 => CACHED [velero-builder-base 6/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                       0.0s
 => CACHED [velero-builder 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/veler  0.0s
 => CACHED [stage-4 2/4] COPY --from=velero-builder /output /                                                                            0.0s
 => CACHED [velero-builder-helper 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /outpu  0.0s
 => CACHED [stage-4 3/4] COPY --from=velero-builder-helper /output /                                                                     0.0s
 => CACHED [restic-builder 2/7] RUN mkdir -p /output/usr/bin && mkdir -p /build/restic                                                   0.0s
 => CACHED [restic-builder 3/7] WORKDIR /build/restic                                                                                    0.0s
 => CACHED [restic-builder 4/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [restic-builder 5/7] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                0.0s
 => CACHED [restic-builder 6/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [restic-builder 7/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [stage-4 4/4] COPY --from=restic-builder /output /                                                                            0.0s
 => exporting to image                                                                                                                   0.1s
 => => exporting layers                                                                                                                  0.0s
 => => exporting manifest sha256:255255475f956eab91df8fcc77f4bb5227b16c8107f694efa287eb955879296a                                        0.0s
 => => exporting config sha256:b963c6d13b12412bd87fba4fedbf7f88a9dfdf10c37eaf5b7e1d05177e6ccefb                                          0.0s
 => => exporting attestation manifest sha256:ff8d442eda7b9fab5480296f8f654beb4b9316c77d9575ab2839ebb236f90d0a                            0.0s
 => => exporting manifest list sha256:bfb0cc9ad1cacf09e849c960eb218f9ce03b8646b841b62cb48c2a1592a13730                                   0.0s
container: velero/velero-restore-helper:main
[+] Building 52.4s (24/24) FINISHED                                                                                            kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 4.05kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.1s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.1s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [restic-builder 1/7] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => CACHED [stage-4 1/4] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 544.21kB                                                                                                    0.2s
 => [velero-builder-base 2/6] RUN mkdir -p /output/usr/bin                                                                               2.9s
 => [velero-builder-base 3/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                             2.0s
 => [velero-builder-base 4/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                                 1.8s
 => [velero-builder-base 5/6] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb go mod download                                           1.8s
 => [velero-builder-base 6/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                              2.0s
 => [velero-builder 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/velero       37.7s
 => [velero-builder-helper 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/veler  4.9s
 => CACHED [restic-builder 2/7] RUN mkdir -p /output/usr/bin && mkdir -p /build/restic                                                   0.0s
 => CACHED [restic-builder 3/7] WORKDIR /build/restic                                                                                    0.0s
 => CACHED [restic-builder 4/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         git clone   0.0s
 => CACHED [restic-builder 5/7] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                0.0s
 => CACHED [restic-builder 6/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         git apply   0.0s
 => CACHED [restic-builder 7/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         GOARM=$(ec  0.0s
 => [stage-4 2/4] COPY --from=velero-builder /output /                                                                                   0.3s
 => [stage-4 3/4] COPY --from=velero-builder-helper /output /                                                                            0.2s
 => [stage-4 4/4] COPY --from=restic-builder /output /                                                                                   0.1s
 => exporting to image                                                                                                                   3.1s
 => => exporting layers                                                                                                                  3.0s
 => => exporting manifest sha256:c823dfc0ba2540e89ab6594ef91c18aec4c1633cd777e21d06591a740e10022a                                        0.0s
 => => exporting config sha256:ce5ecab4a7989ca4e673977cad3a1fcfec4fb9b43631d04f8a6fffe188a709c2                                          0.0s
 => => exporting attestation manifest sha256:b474847335e75a25362468baaadfa89ee505825aa3f68735e4c0275d1f482840                            0.0s
 => => exporting manifest list sha256:772be783cef2ecd2579fcc51200b7d463f663afd21a7f041e95069326ce8eb26                                   0.0s
container: velero/velero:main
[+] Building 22.3s (24/24) FINISHED                                                                                            kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 4.05kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.2s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.2s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [restic-builder 1/7] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 561.56kB                                                                                                    0.2s
 => CACHED [stage-4 1/4] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [velero-builder-base 2/6] RUN mkdir -p /output/usr/bin                                                                               2.1s
 => [velero-builder-base 3/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                             1.7s
 => [velero-builder-base 4/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                                 1.6s
 => [velero-builder-base 5/6] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb go mod download                                           2.1s
 => [velero-builder-base 6/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                              6.6s
 => [velero-builder 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/velero-resto  4.7s
 => [velero-builder-helper 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/veler  6.8s
 => [stage-4 2/4] COPY --from=velero-builder /output /                                                                                   0.1s
 => CACHED [restic-builder 2/7] RUN mkdir -p /output/usr/bin && mkdir -p /build/restic                                                   0.0s
 => CACHED [restic-builder 3/7] WORKDIR /build/restic                                                                                    0.0s
 => CACHED [restic-builder 4/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [restic-builder 5/7] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                0.0s
 => CACHED [restic-builder 6/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [restic-builder 7/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => [stage-4 3/4] COPY --from=velero-builder-helper /output /                                                                            0.1s
 => [stage-4 4/4] COPY --from=restic-builder /output /                                                                                   0.1s
 => exporting to image                                                                                                                   0.7s
 => => exporting layers                                                                                                                  0.6s
 => => exporting manifest sha256:90d5e28e7f7af8a3f3f7d7644dd966d158decb4c8234ae2add50bcfd5c78be42                                        0.0s
 => => exporting config sha256:874f8a23969c0475ab6935408fea9df2fb66ff5a1e2464ec20255c59239caef2                                          0.0s
 => => exporting attestation manifest sha256:1e80048ae5da47811786af550b7f51f41ac177b089053f161ee3a75f963fa080                            0.0s
 => => exporting manifest list sha256:a417ed6e4df5a1443c637f51e3e90eff5d0041566437a69a48cd14d91a975a10                                   0.0s
container: velero/velero-restore-helper:main
make all-containers  0.87s user 0.90s system 2% cpu 1:15.95 total

vs main branch

git reset --hard && git checkout upstream/main && export BUILDX_PLATFORMS=linux/arm64; make all-containers && echo "// test cache" >> pkg/constant/constant.go && time make all-containers
...
make all-containers  0.93s user 1.01s system 2% cpu 1:28.55 total
Details

❯ git reset --hard && git checkout upstream/main && export BUILDX_PLATFORMS=linux/arm64; make all-containers && echo "// test cache" >> pkg/constant/constant.go && time make all-containers
HEAD is now at e46515963 Cache module dependencies between container builds
HEAD is now at dacd5eff9 Merge pull request #8380 from sseago/worker-count
[+] Building 70.9s (15/15) FINISHED                                                                                            kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 2.38kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.2s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.2s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [internal] load build context                                                                                                        0.3s
 => => transferring context: 1.44MB                                                                                                      0.3s
 => CACHED [restic-builder 1/3] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [velero-builder 2/4] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                                  2.7s
 => [restic-builder 2/3] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   5.1s
 => [velero-builder 3/4] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   4.5s
 => [restic-builder 3/3] RUN mkdir -p /output/usr/bin &&     export GOARM=$(echo "" | cut -c2-) &&     /go/src/github.com/vmware-tanzu  38.6s
 => [velero-builder 4/4] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero      59.7s
 => [stage-2 2/3] COPY --from=velero-builder /output /                                                                                   0.2s 
 => [stage-2 3/3] COPY --from=restic-builder /output /                                                                                   0.1s 
 => exporting to image                                                                                                                   3.0s 
 => => exporting layers                                                                                                                  2.9s 
 => => exporting manifest sha256:cc0bcc448812dc8cda723d577dbc4b0a7f1b0b2c959d11bf6138132d0aca781b                                        0.0s 
 => => exporting config sha256:af7e3af8c54a3e2f29090bf0e18f50595cdd6c085e314f81bd0e207a97784bf5                                          0.0s 
 => => exporting attestation manifest sha256:4dabab59e9a03035e65aec199abaf41388bafb8bf1e91492f488f9febb2caa76                            0.0s 
 => => exporting manifest list sha256:69c0f6643eb59d576739e3a27b09a972c4743020b8b4b2bc273d88a432e6a719                                   0.0s 
container: velero/velero:main                                                                                                                 
[+] Building 8.5s (15/15) FINISHED                                                                                             kubernetes:crc 
 => [internal] load build definition from Dockerfile                                                                                     0.0s 
 => => transferring dockerfile: 2.38kB                                                                                                   0.0s 
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.2s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.2s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [internal] load build context                                                                                                        0.3s
 => => transferring context: 543.35kB                                                                                                    0.3s
 => [restic-builder 1/3] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => CACHED [restic-builder 2/3] COPY . /go/src/github.com/vmware-tanzu/velero                                                            0.0s
 => CACHED [velero-builder 2/4] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                           0.0s
 => CACHED [velero-builder 3/4] COPY . /go/src/github.com/vmware-tanzu/velero                                                            0.0s
 => [restic-builder 3/3] RUN mkdir -p /output/usr/bin &&     export GOARM=$(echo "" | cut -c2-) &&     /go/src/github.com/vmware-tanzu/  2.2s
 => [velero-builder 4/4] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero-rest  6.8s
 => [stage-2 2/3] COPY --from=velero-builder /output /                                                                                   0.1s 
 => [stage-2 3/3] COPY --from=restic-builder /output /                                                                                   0.1s
 => exporting to image                                                                                                                   0.7s
 => => exporting layers                                                                                                                  0.6s
 => => exporting manifest sha256:d4a247b9a394bbc7a9a811a0c09c979cb626df084b9a9f412d27af87c390ef3a                                        0.0s
 => => exporting config sha256:a78a5ad0417fd9393119e25fc7d0fadc9072a9ecfb670652c6b112a36c6eb459                                          0.0s
 => => exporting attestation manifest sha256:808c4106e969f75e9defc3e9e758539ea15273d65ae35f7c5823a1e531530916                            0.0s
 => => exporting manifest list sha256:11501d2996a4fb0726d80a88943ee80fbffafeeb4ffcfbc4d4e1b21ba0d68485                                   0.0s
container: velero/velero-restore-helper:main
[+] Building 71.5s (15/15) FINISHED                                                                                            kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 2.38kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.1s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.1s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => CACHED [restic-builder 1/3] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 544.33kB                                                                                                    0.2s
 => CACHED [velero-builder 2/4] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                           0.0s
 => [restic-builder 2/3] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   3.5s
 => [velero-builder 3/4] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   5.5s
 => [restic-builder 3/3] RUN mkdir -p /output/usr/bin &&     export GOARM=$(echo "" | cut -c2-) &&     /go/src/github.com/vmware-tanzu  45.2s
 => [velero-builder 4/4] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero      61.8s
 => [stage-2 2/3] COPY --from=velero-builder /output /                                                                                   0.2s 
 => [stage-2 3/3] COPY --from=restic-builder /output /                                                                                   0.1s 
 => exporting to image                                                                                                                   3.0s 
 => => exporting layers                                                                                                                  2.9s 
 => => exporting manifest sha256:e42bc864d6742dc5823df75c79d488f2fbe93ddac7425bf718e6876567b83024                                        0.0s 
 => => exporting config sha256:a5a2d96b157fcdb1ad02d4011796e489428609560b1206b0e76fbb9792032547                                          0.0s 
 => => exporting attestation manifest sha256:8dfc981ebcc11a3acc60251044f35757b5797870d9154656153de9d4d20869d4                            0.0s 
 => => exporting manifest list sha256:1be37e0ed679001fcbc553ba112865fba9cb42a6a523202f1066c3b2f7edd09f                                   0.0s 
container: velero/velero:main                                                                                                                 
[+] Building 15.8s (17/17) FINISHED                                                                                            kubernetes:crc 
 => [internal] load build definition from Dockerfile                                                                                     0.0s 
 => => transferring dockerfile: 2.38kB                                                                                                   0.0s 
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.3s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.3s
 => [auth] library/golang:pull token for registry-1.docker.io                                                                            0.0s
 => [auth] paketobuildpacks/run-jammy-tiny:pull token for registry-1.docker.io                                                           0.0s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => CACHED [restic-builder 1/3] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 561.69kB                                                                                                    0.2s
 => CACHED [velero-builder 2/4] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                           0.0s
 => [velero-builder 3/4] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   2.2s
 => [restic-builder 2/3] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   3.8s
 => [velero-builder 4/4] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero-res  11.7s
 => [restic-builder 3/3] RUN mkdir -p /output/usr/bin &&     export GOARM=$(echo "" | cut -c2-) &&     /go/src/github.com/vmware-tanzu/  9.4s
 => [stage-2 2/3] COPY --from=velero-builder /output /                                                                                   0.1s 
 => [stage-2 3/3] COPY --from=restic-builder /output /                                                                                   0.1s
 => exporting to image                                                                                                                   0.7s
 => => exporting layers                                                                                                                  0.6s
 => => exporting manifest sha256:added338a5b3f516ca59ff06a053c987abd11d5dfe44565385d6be94737cb591                                        0.0s
 => => exporting config sha256:f26c3643831e7df6379b746ee2429e6a8dcbdf3ec220e830e83999bfc329138c                                          0.0s
 => => exporting attestation manifest sha256:aaa90d2e1055a79d71d33c756e25f2bdddc493ab2ab7d56e901a8df1503ab326                            0.0s
 => => exporting manifest list sha256:4f3bb657a22d2e90e497ec1ed917542b7c045d0d1fdde5f2bc577a247393f092                                   0.0s
container: velero/velero-restore-helper:main
make all-containers  0.93s user 1.01s system 2% cpu 1:28.55 total

@reasonerjt reasonerjt removed their request for review November 18, 2024 13:01
Dockerfile Outdated Show resolved Hide resolved
Signed-off-by: Tiger Kaovilai <[email protected]>

break up velero-builder stage to allow more concurrent layer builds.

Signed-off-by: Tiger Kaovilai <[email protected]>

Add restic caching

if fix_restic_cve.txt changes, only the dep added during patch will require redownload.
```
❯ make container
[+] Building 64.5s (20/20) FINISHED                                                                                                                                                                                                                      docker-container:colima-multiplat
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                  0.0s
 => => transferring dockerfile: 3.24kB                                                                                                                                                                                                                                                0.0s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                                                                                                                                                                     0.1s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                                                                                                                                                               0.1s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                     0.0s
 => => transferring context: 60B                                                                                                                                                                                                                                                      0.0s
 => [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae                                                                                                                                       0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae                                                                                                                                               0.0s
 => [restic-builder 1/5] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4                                                                                                                                          0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4                                                                                                                                                         0.0s
 => [internal] load build context                                                                                                                                                                                                                                                     0.3s
 => => transferring context: 546.21kB                                                                                                                                                                                                                                                 0.3s
 => CACHED [restic-builder 2/5] RUN if [ "velero" = "velero" ]; then         mkdir -p /build/restic &&         cd /build/restic &&         git clone --single-branch -b v0.15.0 https://github.com/restic/restic.git . &&         go mod download;     fi                             0.0s
 => [restic-builder 3/5] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                                                                                                                                                                    0.0s
 => CACHED [velero-builder 2/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                                                                                                                                                                        0.0s
 => CACHED [velero-builder 3/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                                                                                                                                                                            0.0s
 => CACHED [velero-builder 4/6] RUN go mod download                                                                                                                                                                                                                                   0.0s
 => [velero-builder 5/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                                                                                                                                                                                6.5s
 => [restic-builder 4/5] RUN if [ "velero" = "velero" ]; then         mkdir -p /output/usr/bin &&         cd /build/restic &&         git apply /go/src/github.com/vmware-tanzu/velero/hack/fix_restic_cve.txt &&         go mod download;     fi                                     7.2s
 => [velero-builder 6/6] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero     -ldflags "-X github.com/vmware-tanzu/velero/pkg/buildinfo.Version=main -X github.com/vmware-tanzu/velero/pkg/buildinfo.GitSHA=590f6df48d3fdc  55.5s
 => [restic-builder 5/5] RUN if [ "velero" = "velero" ]; then         cd /build/restic &&         GOARM=$(echo "" | cut -c2-) go run build.go --goos "linux" --goarch "amd64" --goarm "" -o /output/usr/bin/restic &&         chmod +x /output/usr/bin/restic;     fi &&     go cle  30.0s
 => CACHED [stage-2 2/3] COPY --from=velero-builder /output /                                                                                                                                                                                                                         0.0s
 => CACHED [stage-2 3/3] COPY --from=restic-builder /output /                                                                                                                                                                                                                         0.0s
 => exporting to docker image format                                                                                                                                                                                                                                                  0.7s
 => => exporting layers                                                                                                                                                                                                                                                               0.0s
 => => exporting manifest sha256:d275186bbf7b7fb08da8fe957037d8600c7b96693fe59bdfb119d977058daaa0                                                                                                                                                                                     0.0s
 => => exporting config sha256:5cb88b0eac0b14df3efd549e0ca4df1980501a5f5cea677f0de1f5dd97050141                                                                                                                                                                                       0.0s
 => => sending tarball                                                                                                                                                                                                                                                                0.7s
 => importing to docker                                                                                                                                                                                                                                                               0.0s
container: velero/velero:main
```

Signed-off-by: Tiger Kaovilai <[email protected]>
@kaovilai kaovilai requested a review from blackpiglet November 20, 2024 02:18
@kaovilai kaovilai removed the Website non-docs changes for the website label Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/changelog-not-required PR does not require a user changelog. Often for docs, website, or build changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants