Use BuildKit syntax in Dockerfile

[MisterDA]

• Enable BuildKit on ocaml-ci.org deployments ocurrent/ocurrent-deployer#246

Should speed up builds a bit.

[sabine]

https://deploy.ci.ocaml.org/job/2024-11-25/151612-ocluster-build-0edd49 complains:

Error response from daemon: dockerfile parse error line 21: Unknown flag: link

[MisterDA]

Yes, BuildKit needs to be enabled in ocurrent-deployer for ocaml.org. Regardless, are you interested in these changes?

[sabine]

With the outcome of a faster build, they seem useful in the sense of taking less resources of the build cluster.

[cuihtlauac]

Thanks, @MisterDA. We're also considering replacing the opam-repo pin with a docker image pin. See the discussion #2760. Am I correct to assume using buildkit is an independent matter? We should be able to have both, in separate stages.

In this PR, it seems three things are done together

1. Using buildkit features
2. Moving the opam repo pin
3. Improve the syntax, clarity and style

As much as possible, I'd like to have those addressed in separate PRs.

3. -> Should be addressed first, let's merge that and have it out of sight.
4. -> The pin needs to be changed in several places at the same time (yes, this is clunky). It's better to have a PR that only does that
5. -> BuildKit seems very cool. I'm excited to take advantage of it.

P.S.

Sorry for sounding bureaucratic. With @mtelvers we recently had to make an urgent fix. A subtle bug was introduced when moving to OCaml 5. It didn't break until 5.2.1 was released.

Our builds (Docker + local + GitHub) are brittle. Let's be extra cautious when tweaking them. We all love debugging. But the holiday season is close. We'd better make sure build issues don't gatecrash into the parties.

[cuihtlauac]

@MisterDA, @mtelvers, @shonfeder: I have computed MTBC (mean time between commits) for ocamlorg and opam-repo

ocamlorg: ~14h14m
opam-repo: ~1h45m

In ocamlorg, commits are pushed one by one (we've turned off push to main and all merges are stashed). Can we make the same assumption for opam-repo?

Those numbers should be compared with the change frequencies of the ocamlorg pin and the docker image

ocamlorg build pin: ~ 2 per year
docker image: 1 per week

That's why I believe we shouldn’t fetch from opam-repo early. It should take place right before init-cache

[shonfeder]

Hi! :) I'm afraid I'll need a bit of clarification.

1. I don't really get the intended significance of the mean time between commits. Could you flesh this out a bit for me?

2. I'm not sure what "all merges are stashed" means.

3. In general, given that you've asked that this to be factored into separate PRs (a great call, IMO!) I'm not exactly sure the relevance of this point. Is it related to the discussion at Recreate PR #2861: Dockerfile cleanup #2866 (review) ?

Thanks in advance for the clarification!

[cuihtlauac]

@MisterDA: I've merged the cleanup PR. Can you rebase this one?