Support for "docker buildx bake" command with 'jf rt build-docker-create'

[enaess]

While you can use docker buildx build and specify a tag on the command line and push the image, then use the jf rt build-docker-create command to push the resulting images using the build-metadata file generated by the docker buildx bake command.

Simply create a docker-bake.hcl file like the following:

group "linux" {
  targets = [
    "java-builder-ubuntu-jammy-java-11",
    "java-builder-ubuntu-jammy-java-17",
  ]
}

variable "REGISTRY" {
  default = "INSERT YOUR ARTIFACTORY URL HERE"
}

variable "REPOSITORY" {
  default = "INSERT YOUR REPOSITORY HERE"
}

target "ubuntu-jammy-java-11" {
  dockerfile = "Dockerfile"
  args = {
     JAVA_VERSION="11"
     VERSION="jammy"
  }
}

target "ubuntu-jammy-java-17" {
  dockerfile = "Dockerfile"
  args = {
     JAVA_VERSION="17"
     VERSION="jammy"
  }
}

target "java-builder-ubuntu-jammy-java-11" {
  dockerfile = "Dockerfile.java"
  contexts = {
    base-image = "target:ubuntu-jammy-java-11"
  }
  tags = [
    "${REGISTRY}/${REPOSITORY}/java-builder:java-11-latest",
    "${REGISTRY}/${REPOSITORY}/java-builder:linux-11-latest",
    "${REGISTRY}/${REPOSITORY}/java-builder:ubuntu-java-11-latest",
  ]
  platforms = ["linux/amd64"]
}

target "java-builder-ubuntu-jammy-java-17" {
  dockerfile = "Dockerfile.java"
  contexts = {
    base-image = "target:ubuntu-jammy-java-17"
  }
  tags = [
    "${REGISTRY}/${REPOSITORY}/java-builder:java-17-latest",
    "${REGISTRY}/${REPOSITORY}/java-builder:linux-latest",
    "${REGISTRY}/${REPOSITORY}/java-builder:linux-17-latest",
    "${REGISTRY}/${REPOSITORY}/java-builder:ubuntu-java-latest",
    "${REGISTRY}/${REPOSITORY}/java-builder:ubuntu-java-17-latest",
    "${REGISTRY}/${REPOSITORY}/java-builder:latest"
  ]
  platforms = ["linux/amd64"]
}

Then create two Dockerfile called Dockerfile and Dockerfile.java

Dockerfile

ARG VERSION=jammy
FROM ubuntu:${VERSION}

# Get basic tools to create image
ARG DEBIAN_FRONTEND=noninteractive
RUN    apt-get update \
	&& apt-get install -y gnupg curl java-common procps tzdata locales apt-utils lsb-release \
	&& apt-get -y clean \
	&& rm -rf /var/lib/apt/lists/*

# Setup the timezone and reconfigure locale
ENV TZ=America/Los_Angeles
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
RUN locale-gen en_US en_US.UTF-8
RUN dpkg-reconfigure locales 

Dockerfile.java

FROM base-image

ARG JAVA_VERSION=17
RUN curl -fsSL https://apt.corretto.aws/corretto.key | gpg --dearmor | tee /usr/share/keyrings/amazon-keyring.gpg > /dev/null \
    && echo "deb [signed-by=/usr/share/keyrings/amazon-keyring.gpg] https://apt.corretto.aws stable main" | tee /etc/apt/sources.list.d/corretto.list > /dev/null \
	&& apt-get update \
	&& apt-get install -y java-$JAVA_VERSION-amazon-corretto-jdk \
	&& apt-get -y clean \
	&& rm -rf /var/lib/apt/lists/* /usr/share/keyrings/amazon-keyring.gpg /etc/apt/sources.list.d/corretto.list

ENV JAVA_HOME="/usr/lib/jvm/java-${JAVA_VERSION}-amazon-corretto"
ENV PATH="${JAVA_HOME}/bin:${PATH}"

Now run the docker bake command:
docker buildx bake --file docker-bake.hcl --metadata-file=build-metadata --push linux

Trying to run the jf rt build-docker-create <repository> --server-id=<server-id> --image-file build-metadata --build-name MyBuild --build-number 1 fails complaining about the presence of a image 256 sha sum.

Looking a bit further, the error message
unexpected file format "build-metadata". The file should include one line in the following format: image-tag@sha256

Appears in the jfrog-cli-core project in artifactory/utils/container/buildinfo.go line 165. My guess is that this command doesn't take into account the build-metadata can be yet another sub-level deep e.g. wrapped with the "target" { ... } of the image. And even if it did, the list of tags inside the image.name property is a comma separated list.

When we do build of multiple containers, the docker buildx bake offers a contextual "inheritance" so to speak where you can use FROM base-image inside a secondary Dockerfile to model a "extends" without requiring this image to be tagged first. Also, being able to create multiple containers in one go is extremely useful to avoid the overhead of building each container by itself.

Maybe in the short term some magic using jq could help create multiple files on the fly by massaging the content of the meta-data file and then adding it, but it isn't apparent that multiple sequential executions of build-docker-create will append to each-other, so who knows... Re-writing my entire Jenkinsfile to support this seems silly at this point

Please consider this feature request soon.

[Lykathia]

It's more frustrating, because there is jfrog documentation (blog posts) that suggests this should work - but as shown above, does not.
https://jfrog.com/help/r/title-artifactory-how-to-use-jfrog-cli-to-add-build-info-to-images-built-with-docker-buildx/artifactory-how-to-use-jfrog-cli-to-add-build-info-to-images-built-with-docker-buildx
https://jfrog.com/help/r/jfrog-cli/pushing-docker-images-using-buildx

The underlying check (which I believe comes from the buildinfo repository?) needs to be updated to support the new provenance format, or something to that effect I believe.

Current workaround is to do something akin to (in a github action)

          cat > build-metadata <<EOF
            ${{ steps.bake.outputs.metadata }}
          EOF

          image_details=`cat build-metadata | jq '.default | ."image.name" + "@" + ."containerimage.digest"'`
          jfrog rt build-docker-create docker-local --image-file <( echo "$image_details")