build: create parent directories for tar output in lazy writer

[crazy-max]

This change updates the tar output logic for exporters to ensure parent directories are created automatically when writing to a file destination.

Previously, file creation would fail if the target directory did not exist, which would fail builds that rely on dynamic directory creation:

$ docker buildx build --output type=tar,dest=./does/not/exists/result.tar -<<< $'FROM busybox'
ERROR: failed to build: failed to open open ./does/not/exists/result.tar: no such file or directory

I have created a lazy file writer that calls os.MkdirAll before creating the file so we are consistent with the local output that already creates subdirectories if they don't exist.

$ docker buildx build --output type=tar,dest=./does/not/exists/result.tar -<<< $'FROM busybox'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 50B done
#1 DONE 0.0s

#2 [auth] library/busybox:pull token for registry-1.docker.io
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/busybox:latest
#3 DONE 0.7s

#4 [internal] load .dockerignore
#4 transferring context: 2B done
#4 DONE 0.0s

#5 [1/1] FROM docker.io/library/busybox:latest@sha256:2f590fc602ce325cbff2ccfc39499014d039546dc400ef8bbf5c6ffb860632e7
#5 resolve docker.io/library/busybox:latest@sha256:2f590fc602ce325cbff2ccfc39499014d039546dc400ef8bbf5c6ffb860632e7 0.0s done
#5 CACHED

#6 exporting to client tarball
#6 sending tarball 0.1s done
#6 DONE 0.1s

$ tree does/
does/
└── not
    └── exists
        └── result.tar

2 directories, 1 file

With bake:

# docker-bake.hcl
target "default" {
  output = ["type=tar,dest=./does/not/exists/result.tar"]
}

Before:

$ docker buildx bake --print
#1 [internal] load local bake definitions
#1 reading docker-bake.hcl 79B / 79B done
#1 DONE 0.0s
ERROR: failed to open open ./does/not/exists/result.tar: no such file or directory

After:

$ docker buildx bake --print
#1 [internal] load local bake definitions
#1 reading docker-bake.hcl 79B / 79B done
#1 DONE 0.0s
{
  "group": {
    "default": {
      "targets": [
        "default"
      ]
    }
  },
  "target": {
    "default": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        {
          "dest": "./does/not/exists/result.tar",
          "type": "tar"
        }
      ]
    }
  }
}

This also fixes another issue where there is left over build result on client side before the build even starts like:

target "default" {
  output = ["type=docker,dest=./dist/result.tar"]
}

$ docker buildx bake --print
#1 [internal] load local bake definitions
#1 reading docker-bake.hcl 72B / 72B done
#1 DONE 0.0s
{
  "group": {
    "default": {
      "targets": [
        "default"
      ]
    }
  },
  "target": {
    "default": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        {
          "dest": "../dist/result.tar",
          "type": "docker"
        }
      ]
    }
  }
}

$ ls ./dist
result.tar

cc @maxcleme

[crazy-max]

Needs to make changes in this test to disable fs entitlements otherwise it fails: https://github.com/docker/buildx/actions/runs/18711570130/job/53361047820?pr=3478#step:7:481

=== RUN   TestIntegration/TestBuildMetadataProvenance/worker=remote/default
    bake.go:1375: 
        	Error Trace:	/src/tests/bake.go:1375
        	            				/src/tests/bake.go:1334
        	Error:      	Received unexpected error:
        	            	exit status 1
        	Test:       	TestIntegration/TestBakeMetadataProvenance/worker=remote/max
        	Messages:   	#0 building with "integration-remote-44bdm3ecqo4ofzttczc4bg3yy" instance using remote driver
        	            	
        	            	#1 [internal] load local bake definitions
        	            	#1 reading docker-bake.hcl 22B / 22B done
        	            	#1 DONE 0.0s
        	            	Your build is requesting privileges for following possibly insecure capabilities:
        	            	
        	            	 - Write access to path ../002
        	            	
        	            	Pass "--allow=fs.write=../002" to grant requested privileges.
        	            	
        	            	Your full command with requested privileges:
        	            	
        	            	buildx bake --allow=fs.write=../002 --metadata-file /tmp/TestIntegrationTestBakeMetadataProvenanceworker=remotemax18445594/002/md.json --set default.output=type=docker,dest=/tmp/TestIntegrationTestBakeMetadataProvenanceworker=remotemax18445594/002/image.tar
        	            	
        	            	To disable filesystem entitlements checks, you can set BUILDX_BAKE_ENTITLEMENTS_FS=0 .
        	            	
        	            	ERROR: additional privileges requested

Seems fs entitlements were not correctly applied for tar output before.

[crazy-max]

Seems fs entitlements were not correctly applied for tar output before.

As we lazily create directories now for tar output then it goes through

buildx/bake/entitlements.go

Lines 595 to 603 in abf6ab4

	// If the component doesn't exist, return the last valid path
	if os.IsNotExist(err) {
	for r := len(dest) - 1; r >= volLen; r-- {
	if os.IsPathSeparator(dest[r]) {
	return dest[:r], in[start:], nil
	}
	}
	return vol, in[start:], nil
	}

and returns the last valid path which triggers the privileges request.

Before the tar output file was already created before entitlement check:

buildx/build/opt.go

Line 879 in f478741

f, err := os.Create(entry.Destination)

buildx/bake/bake.go

Line 1542 in f478741

bo.Exports, bo.ExportsLocalPathsTemporary, err = build.CreateExports(t.Outputs)

so didn't go through

buildx/bake/entitlements.go

Lines 595 to 603 in abf6ab4

	// If the component doesn't exist, return the last valid path
	if os.IsNotExist(err) {
	for r := len(dest) - 1; r >= volLen; r-- {
	if os.IsPathSeparator(dest[r]) {
	return dest[:r], in[start:], nil
	}
	}
	return vol, in[start:], nil
	}

anymore.

[crazy-max]

Hum actually this is not correct, this test should have failed before. I'm taking another look.

[crazy-max]

Oh I think I get it now, it seems it does not collect overrides for checking entitlements:

target "default" {}

$ docker buildx bake --set *.output=type=docker,dest=../dist/result.tar
#0 building with "default" instance using docker driver

#1 [internal] load local bake definitions
#1 reading docker-bake.hcl 21B / 21B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 2B done
#2 DONE 0.1s

Which makes sense as override is a user input but don't think we support this atm. cc @tonistiigi

So before my change the tar file was already created before the entitlement check and didn't request privilege which is wrong.

Now with my change we create the tar file lazily and therefore entitlement checks the parent dir and requests privilege which is good but if I create the file before:

$ touch ../dist/result.tar
$ docker buildx bake --set *.output=type=docker,dest=../dist/result.tar
#0 building with "default" instance using docker driver

#1 [internal] load local bake definitions
#1 reading docker-bake.hcl 21B / 21B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 DONE 0.0s

Then it builds and does not request privilege which is wrong. So we need to fix this.