Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When pulling a signed image that is part of a image-index the image can't be pulled. #66

Open
jmarrero opened this issue Aug 7, 2024 · 1 comment

Comments

@jmarrero
Copy link
Member

jmarrero commented Aug 7, 2024

The error is seen by using rpm-ostree or bootc to do a deployment of a signed image:

ERROR Switching: Pulling: Creating importer: Failed to invoke skopeo proxy method OpenImage: remote error: A signature was required, but no signature exists

By adding:

        c.arg("--debug");

and

        println!("stdout:\n{:?}", c.stdout(Stdio::piped()).output());

before:
https://github.com/containers/containers-image-proxy-rs/blob/main/src/imageproxy.rs#L227

I was able to print
Printing the skopeo debug output which shows:

DEBU[0000] GET https://registry.redhat.io/v2/rhel9/rhel-bootc/manifests/9.4
DEBU[0000] Content-Type from manifest GET is "application/vnd.oci.image.index.v1+json"
DEBU[0000] IsRunningImageAllowed for image docker:registry.redhat.io/rhel9/rhel-bootc:9.4

...


DEBU[0000]  Using transport "docker" specific policy section registry.redhat.io
DEBU[0000] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=b73c1d3280d6e7b3a6ed9d52138e8e4dde7384dd0227deda67c97f428f97551a/signature-1
DEBU[0001] ... got status 404, as expected = end of signatures
DEBU[0001] Not looking for sigstore attachments: disabled by configuration
DEBU[0001] Requirement 0: denied, done

This does not pull any signatures.

However when we try skopeo copy we see:

DEBU[0000] GET https://registry.redhat.io/v2/rhel9/rhel-bootc/manifests/9.4
DEBU[0000] Content-Type from manifest GET is "application/vnd.oci.image.index.v1+json"
DEBU[0000] Using SQLite blob info cache at /var/lib/containers/cache/blob-info-cache-v1.sqlite
DEBU[0000] Source is a manifest list; copying (only) instance sha256:68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf for current system
DEBU[0000] GET https://registry.redhat.io/v2/rhel9/rhel-bootc/manifests/sha256:68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf

...

DEBU[0000]  Using transport "docker" specific policy section registry.redhat.io
DEBU[0000] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-1
DEBU[0000] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-2
DEBU[0000] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-3
DEBU[0000] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-4
DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-5
DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-6
DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-7
DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-8
DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel9/rhel-bootc@sha256=68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf/signature-9
DEBU[0004] ... got status 404, as expected = end of signatures
DEBU[0004] Not looking for sigstore attachments: disabled by configuration
DEBU[0004]  Requirement 0: allowed
DEBU[0004] Overall: allowed

Which correctly pulls the signatures, however this code goes thru this path:
https://github.com/containers/image/blob/8c7c58c5aacd70fe8bc25da54f966a59baf175b0/copy/copy.go#L318

This path looks at the image-index and checks signatures only for the system image.

This path as I understand we avoid, and implement our own way of pulling using the skopeo/proxy
https://github.com/containers/skopeo/blob/main/cmd/skopeo/proxy.go#L412

I am still not sure if the fix should be here or in skopeo/proxy, but my undestanding right now is that we have to implement something similar to what containers/image/copy does.

Ultimately this blocks any deployment when a more strict /etc/containers/policy.json is defined such as when using RHEL.

Currently a workaround is to use the Manifest List Digest: such as: registry.redhat.io/rhel9/rhel-bootc@sha256:68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf

instead of the tag.

@jmarrero
Copy link
Member Author

jmarrero commented Aug 7, 2024

If I change the policy.json I see:

DEBU[0000] Overall: allowed                             
TRACE open_image:impl_request: completed request self=ImageProxy imgref="docker://registry.redhat.io/rhel9/rhel-bootc:9.4" self=ImageProxy method="OpenImage"
TRACE impl_request: sending request GetManifest self=ImageProxy method="GetManifest"
DEBU[0000] GET https://registry.redhat.io/v2/rhel9/rhel-bootc/manifests/sha256:68100781edd6317a9c9f419374568a641b8402b1901740cfd38cdd20db902caf 
DEBU[0001] Content-Type from manifest GET is "application/vnd.oci.image.manifest.v1+json" 
TRACE impl_request: completed request self=ImageProxy method="GetManifest"
DEBUG finish_pipe: closing pipe self=ImageProxy pipeid=8
TRACE finish_pipe:impl_request: sending request FinishPipe self=ImageProxy pipeid=8 self=ImageProxy method="FinishPipe"
TRACE finish_pipe:impl_request: completed request self=ImageProxy pipeid=8 self=ImageProxy method="FinishPipe"
TRACE impl_request: sending request GetFullConfig self=ImageProxy method="GetFullConfig"

Maybe we can change the validation to happen during or after getManifest since it looks like at that point we would have the SHA that has the signature. We fail right now on OpenImage.

Also another way to avoid this is to sign the Digest. To get the digest you must sign you can run:

skopeo inspect docker://registry.redhat.io/rhel9/rhel-bootc:9.4 | jq .Digest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant