Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLID-280: WIP: Copy signatures during M2M #1065

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sherine-k
Copy link
Contributor

@sherine-k sherine-k commented Feb 4, 2025

This PR is NOT TO BE MERGED.

This is just a POC of what would be needed to start copying / verifying signatures.

The goal of the PR is to initiate discussions with the rest of the OCP community around how signature verification will look like on a disconnected cluster.

One of the priorities is to confirm the system configuration for signature lookup and verification.

This PR demonstrates how to perform a mirror to mirror workflow with signature mirroring :

Command used:

./bin/oc-mirror --v2 -c v2config.yaml --workspace file:///home/skhoury/45580/ docker://sherinefedora:5000

ImageSetConfig - v2config.yaml

mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.17
      packages:
      - name: devworkspace-operator


kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1

/etc/containers/registries.d/registry.redhat.io.yaml

docker:
  registry.redhat.io:
    use-sigstore-attachments: true

⚠️ Note that we are not using lookaside here.
❓ Is this going to be the OCP node default configuration as well? OCPSTRAT-1245

/etc/containers/registries.d/registry.redhat.io.yaml

docker:
  registry.redhat.io:
    use-sigstore-attachments: true

/etc/containers/registries.d/sherinefedora:5000.yaml

docker:
  "sherinefedora:5000":
    use-sigstore-attachments: true

⚠️ This is the destination registry. Red Hat default configuration might not provide such a file. oc-mirror documentation needs to cover this?

/etc/containers/registries.d/localhost:55000.yaml

docker:
     "localhost:55000":
         use-sigstore-attachments: true

⚠️ This is the oc-mirror internal cache. By default it uses port 55000 , but this can be overridden by the user. We need to think about how best to set this configuration file, here or in $HOME/.config/containers/registries.d (no root permissions required)

/etc/containers/policy.json

{
    "default": [
        {
            "type": "insecureAcceptAnything"
        }
    ],
    "transports": {
        "docker": {
          "registry.redhat.io/redhat/certified-operator-index": [
            {
              "type": "signedBy",
              "keyType": "GPGKeys",
              "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-isv"
            }
          ],
          "registry.redhat.io/redhat/community-operator-index": [
            {
              "type": "signedBy",
              "keyType": "GPGKeys",
              "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-isv"
            }
          ],
          "registry.redhat.io/redhat/redhat-marketplace-index": [
            {
              "type": "signedBy",
              "keyType": "GPGKeys",
              "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-isv"
            }
          ],
	    "registry.access.redhat.com": [
		{
		    "type": "signedBy",
		    "keyType": "GPGKeys",
		    "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release"
		}
	    ],
	    "registry.redhat.io": [
		{
		    "type": "signedBy",
		    "keyType": "GPGKeys",
		    "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release"
		}
	    ]
	},
        "docker-daemon": {
	    "": [
		{
		    "type": "insecureAcceptAnything"
		}
	    ]
	}
    }
}

Console output

2025/02/04 10:58:53  [INFO]   : 👋 Hello, welcome to oc-mirror
2025/02/04 10:58:53  [INFO]   : ⚙️  setting up the environment for you...
2025/02/04 10:58:53  [INFO]   : 🔀 workflow mode: mirrorToMirror 
2025/02/04 10:58:53  [INFO]   : 🕵  going to discover the necessary images...
2025/02/04 10:58:53  [INFO]   : 🔍 collecting release images...
2025/02/04 10:58:53  [INFO]   : 🔍 collecting operator images...
 ✓   (0s) Collecting catalog registry.redhat.io/redhat/redhat-operator-index:v4.17 
2025/02/04 10:58:54  [INFO]   : 🔍 collecting additional images...
2025/02/04 10:58:54  [INFO]   : 🔍 collecting helm images...
2025/02/04 10:58:54  [INFO]   : 🔂 rebuilding catalogs
2025/02/04 10:58:54  [INFO]   : 🚀 Start copying the images...
2025/02/04 10:58:54  [INFO]   : 📌 images to copy 7 
 ✓   (0s) redhat-operator-index:v4.17 ➡️  sherinefedora:5000/redhat/ 
 ✓ (3s) devworkspace-operator-bundle@sha256:085fe1841e55a06190f79f7d9c17f19b20d6b6155e28de1d567d3f68ca95ff81 ➡️  sherinefedora:500…
 ✓   (8s) redhat-operator-index:v4.17 ➡️  cache 
 ✓ (19s) devworkspace-rhel9-operator@sha256:73472149035bc7f0c4ecaf911f68bd391a36d45a21238fd2f46a839ab9e77dcd ➡️  sherinefedora:500…
 ✓ (22s) ose-kube-rbac-proxy@sha256:1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2 ➡️  sherinefedora:5000/opensh…
 ✓ (36s) devworkspace-project-clone-rhel9@sha256:afa28466d9b8c9d6421bb7789bccb68ecde9ea2c38617825e75c53ba357d4b76 ➡️  sherinefedor…
7 / 7 (37s) [=======================================================================================================================] 100 %
 ✓   (37s) ubi-minimal@sha256:b87097994ed62fbf1de70bc75debe8dacf3ea6e00dd577d74503ef66452c59d6 ➡️  sherinefedora:5000/ubi9/ 
2025/02/04 10:59:32  [INFO]   : === Results ===
2025/02/04 10:59:32  [INFO]   :  ✓  7 / 7 operator images mirrored successfully
2025/02/04 10:59:32  [INFO]   : 📄 Generating IDMS file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/idms-oc-mirror.yaml file created
2025/02/04 10:59:32  [INFO]   : 📄 No images by tag were mirrored. Skipping ITMS generation.
2025/02/04 10:59:32  [INFO]   : 📄 Generating CatalogSource file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/cs-redhat-operator-index-v4-17.yaml file created
2025/02/04 10:59:32  [INFO]   : 📄 Generating ClusterCatalog file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/cc-redhat-operator-index-v4-17.yaml file created
2025/02/04 10:59:32  [INFO]   : mirror time     : 38.550461664s
2025/02/04 10:59:32  [INFO]   : 👋 Goodbye, thank you for using oc-mirror

Verification of signature mirroring

$ curl https://sherinefedora:5000/v2/openshift4/ose-kube-rbac-proxy/tags/list | jq
{
  "name": "openshift4/ose-kube-rbac-proxy",
  "tags": [
    "a366ef7",
    "sha256-1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2",
    "sha256-1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2.sig",
    "sha256-4488f5fdf6f10a551fe046e62cd4f3d856530cfb15831f7e1e2de531e7f2caeb",
    "sha256-4840b301e261cfb291697d907d98b177ba57a122e5563e42c8d831d3c42e5ef6.sig",
    "sha256-863efdda55c3da31afbf622c67b7336d61e73ee96ca762676c4b7d98f1e8f98c.sig",
    "sha256-c9c6c66b59e1775217ae048446b123817db2a8bc6ee729914101916dc29bcd3c.sig",
    "sha256-e4fd184a8691e7b8d2d8da561cec09e87f2825b66774fffeb50da6a9e0b515ff.sig"
  ]
}

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 4, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 4, 2025

@sherine-k: This pull request references CLID-280 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "4.19.0" version, but no target version was set.

In response to this:

This PR is NOT TO BE MERGED.

This is just a POC of what would be needed to start copying / verifying signatures.

The goal of the PR is to initiate discussions with the rest of the OCP community around how signature verification will look like on a disconnected cluster.

One of the priorities is to confirm the system configuration for signature lookup and verification.

This PR demonstrates how to perform a mirror to mirror workflow with signature mirroring :

Command used:

./bin/oc-mirror --v2 -c v2config.yaml --workspace file:///home/skhoury/45580/ docker://sherinefedora:5000

ImageSetConfig - v2config.yaml

mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.17
     packages:
     - name: devworkspace-operator


kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1

/etc/containers/registries.d/registry.redhat.io.yaml

docker:
 registry.redhat.io:
   use-sigstore-attachments: true

⚠️ Note that we are not using lookaside here.
❓ Is this going to be the OCP node default configuration as well? OCPSTRAT-1245

/etc/containers/registries.d/registry.redhat.io.yaml

docker:
 registry.redhat.io:
   use-sigstore-attachments: true

/etc/containers/registries.d/sherinefedora:5000.yaml

docker:
 "sherinefedora:5000":
   use-sigstore-attachments: true

⚠️ This is the destination registry. Red Hat default configuration might not provide such a file. oc-mirror documentation needs to cover this?

/etc/containers/registries.d/localhost:55000.yaml

docker:
    "localhost:55000":
        use-sigstore-attachments: true

⚠️ This is the oc-mirror internal cache. By default it uses port 55000 , but this can be overridden by the user. We need to think about how best to set this configuration file, here or in $HOME/.config/containers/registries.d (no root permissions required)

Console output

2025/02/04 10:58:53  [INFO]   : 👋 Hello, welcome to oc-mirror
2025/02/04 10:58:53  [INFO]   : ⚙️  setting up the environment for you...
2025/02/04 10:58:53  [INFO]   : 🔀 workflow mode: mirrorToMirror 
2025/02/04 10:58:53  [INFO]   : 🕵  going to discover the necessary images...
2025/02/04 10:58:53  [INFO]   : 🔍 collecting release images...
2025/02/04 10:58:53  [INFO]   : 🔍 collecting operator images...
✓   (0s) Collecting catalog registry.redhat.io/redhat/redhat-operator-index:v4.17 
2025/02/04 10:58:54  [INFO]   : 🔍 collecting additional images...
2025/02/04 10:58:54  [INFO]   : 🔍 collecting helm images...
2025/02/04 10:58:54  [INFO]   : 🔂 rebuilding catalogs
2025/02/04 10:58:54  [INFO]   : 🚀 Start copying the images...
2025/02/04 10:58:54  [INFO]   : 📌 images to copy 7 
✓   (0s) redhat-operator-index:v4.17 ➡️  sherinefedora:5000/redhat/ 
✓ (3s) devworkspace-operator-bundle@sha256:085fe1841e55a06190f79f7d9c17f19b20d6b6155e28de1d567d3f68ca95ff81 ➡️  sherinefedora:500…
✓   (8s) redhat-operator-index:v4.17 ➡️  cache 
✓ (19s) devworkspace-rhel9-operator@sha256:73472149035bc7f0c4ecaf911f68bd391a36d45a21238fd2f46a839ab9e77dcd ➡️  sherinefedora:500…
✓ (22s) ose-kube-rbac-proxy@sha256:1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2 ➡️  sherinefedora:5000/opensh…
✓ (36s) devworkspace-project-clone-rhel9@sha256:afa28466d9b8c9d6421bb7789bccb68ecde9ea2c38617825e75c53ba357d4b76 ➡️  sherinefedor…
7 / 7 (37s) [=======================================================================================================================] 100 %
✓   (37s) ubi-minimal@sha256:b87097994ed62fbf1de70bc75debe8dacf3ea6e00dd577d74503ef66452c59d6 ➡️  sherinefedora:5000/ubi9/ 
2025/02/04 10:59:32  [INFO]   : === Results ===
2025/02/04 10:59:32  [INFO]   :  ✓  7 / 7 operator images mirrored successfully
2025/02/04 10:59:32  [INFO]   : 📄 Generating IDMS file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/idms-oc-mirror.yaml file created
2025/02/04 10:59:32  [INFO]   : 📄 No images by tag were mirrored. Skipping ITMS generation.
2025/02/04 10:59:32  [INFO]   : 📄 Generating CatalogSource file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/cs-redhat-operator-index-v4-17.yaml file created
2025/02/04 10:59:32  [INFO]   : 📄 Generating ClusterCatalog file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/cc-redhat-operator-index-v4-17.yaml file created
2025/02/04 10:59:32  [INFO]   : mirror time     : 38.550461664s
2025/02/04 10:59:32  [INFO]   : 👋 Goodbye, thank you for using oc-mirror

Verification of signature mirroring

$ curl https://sherinefedora:5000/v2/openshift4/ose-kube-rbac-proxy/tags/list | jq
{
 "name": "openshift4/ose-kube-rbac-proxy",
 "tags": [
   "a366ef7",
   "sha256-1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2",
   "sha256-1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2.sig",
   "sha256-4488f5fdf6f10a551fe046e62cd4f3d856530cfb15831f7e1e2de531e7f2caeb",
   "sha256-4840b301e261cfb291697d907d98b177ba57a122e5563e42c8d831d3c42e5ef6.sig",
   "sha256-863efdda55c3da31afbf622c67b7336d61e73ee96ca762676c4b7d98f1e8f98c.sig",
   "sha256-c9c6c66b59e1775217ae048446b123817db2a8bc6ee729914101916dc29bcd3c.sig",
   "sha256-e4fd184a8691e7b8d2d8da561cec09e87f2825b66774fffeb50da6a9e0b515ff.sig"
 ]
}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from lmzuccarelli and r4f4 February 4, 2025 10:46
Copy link

openshift-ci bot commented Feb 4, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sherine-k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 4, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 4, 2025

@sherine-k: This pull request references CLID-280 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "4.19.0" version, but no target version was set.

In response to this:

This PR is NOT TO BE MERGED.

This is just a POC of what would be needed to start copying / verifying signatures.

The goal of the PR is to initiate discussions with the rest of the OCP community around how signature verification will look like on a disconnected cluster.

One of the priorities is to confirm the system configuration for signature lookup and verification.

This PR demonstrates how to perform a mirror to mirror workflow with signature mirroring :

Command used:

./bin/oc-mirror --v2 -c v2config.yaml --workspace file:///home/skhoury/45580/ docker://sherinefedora:5000

ImageSetConfig - v2config.yaml

mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.17
     packages:
     - name: devworkspace-operator


kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1

/etc/containers/registries.d/registry.redhat.io.yaml

docker:
 registry.redhat.io:
   use-sigstore-attachments: true

⚠️ Note that we are not using lookaside here.
❓ Is this going to be the OCP node default configuration as well? OCPSTRAT-1245

/etc/containers/registries.d/registry.redhat.io.yaml

docker:
 registry.redhat.io:
   use-sigstore-attachments: true

/etc/containers/registries.d/sherinefedora:5000.yaml

docker:
 "sherinefedora:5000":
   use-sigstore-attachments: true

⚠️ This is the destination registry. Red Hat default configuration might not provide such a file. oc-mirror documentation needs to cover this?

/etc/containers/registries.d/localhost:55000.yaml

docker:
    "localhost:55000":
        use-sigstore-attachments: true

⚠️ This is the oc-mirror internal cache. By default it uses port 55000 , but this can be overridden by the user. We need to think about how best to set this configuration file, here or in $HOME/.config/containers/registries.d (no root permissions required)

/etc/containers/policy.json

{
   "default": [
       {
           "type": "insecureAcceptAnything"
       }
   ],
   "transports": {
       "docker": {
         "registry.redhat.io/redhat/certified-operator-index": [
           {
             "type": "signedBy",
             "keyType": "GPGKeys",
             "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-isv"
           }
         ],
         "registry.redhat.io/redhat/community-operator-index": [
           {
             "type": "signedBy",
             "keyType": "GPGKeys",
             "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-isv"
           }
         ],
         "registry.redhat.io/redhat/redhat-marketplace-index": [
           {
             "type": "signedBy",
             "keyType": "GPGKeys",
             "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-isv"
           }
         ],
      "registry.access.redhat.com": [
  	{
  	    "type": "signedBy",
  	    "keyType": "GPGKeys",
  	    "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release"
  	}
      ],
      "registry.redhat.io": [
  	{
  	    "type": "signedBy",
  	    "keyType": "GPGKeys",
  	    "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release"
  	}
      ]
  },
       "docker-daemon": {
      "": [
  	{
  	    "type": "insecureAcceptAnything"
  	}
      ]
  }
   }
}

Console output

2025/02/04 10:58:53  [INFO]   : 👋 Hello, welcome to oc-mirror
2025/02/04 10:58:53  [INFO]   : ⚙️  setting up the environment for you...
2025/02/04 10:58:53  [INFO]   : 🔀 workflow mode: mirrorToMirror 
2025/02/04 10:58:53  [INFO]   : 🕵  going to discover the necessary images...
2025/02/04 10:58:53  [INFO]   : 🔍 collecting release images...
2025/02/04 10:58:53  [INFO]   : 🔍 collecting operator images...
✓   (0s) Collecting catalog registry.redhat.io/redhat/redhat-operator-index:v4.17 
2025/02/04 10:58:54  [INFO]   : 🔍 collecting additional images...
2025/02/04 10:58:54  [INFO]   : 🔍 collecting helm images...
2025/02/04 10:58:54  [INFO]   : 🔂 rebuilding catalogs
2025/02/04 10:58:54  [INFO]   : 🚀 Start copying the images...
2025/02/04 10:58:54  [INFO]   : 📌 images to copy 7 
✓   (0s) redhat-operator-index:v4.17 ➡️  sherinefedora:5000/redhat/ 
✓ (3s) devworkspace-operator-bundle@sha256:085fe1841e55a06190f79f7d9c17f19b20d6b6155e28de1d567d3f68ca95ff81 ➡️  sherinefedora:500…
✓   (8s) redhat-operator-index:v4.17 ➡️  cache 
✓ (19s) devworkspace-rhel9-operator@sha256:73472149035bc7f0c4ecaf911f68bd391a36d45a21238fd2f46a839ab9e77dcd ➡️  sherinefedora:500…
✓ (22s) ose-kube-rbac-proxy@sha256:1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2 ➡️  sherinefedora:5000/opensh…
✓ (36s) devworkspace-project-clone-rhel9@sha256:afa28466d9b8c9d6421bb7789bccb68ecde9ea2c38617825e75c53ba357d4b76 ➡️  sherinefedor…
7 / 7 (37s) [=======================================================================================================================] 100 %
✓   (37s) ubi-minimal@sha256:b87097994ed62fbf1de70bc75debe8dacf3ea6e00dd577d74503ef66452c59d6 ➡️  sherinefedora:5000/ubi9/ 
2025/02/04 10:59:32  [INFO]   : === Results ===
2025/02/04 10:59:32  [INFO]   :  ✓  7 / 7 operator images mirrored successfully
2025/02/04 10:59:32  [INFO]   : 📄 Generating IDMS file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/idms-oc-mirror.yaml file created
2025/02/04 10:59:32  [INFO]   : 📄 No images by tag were mirrored. Skipping ITMS generation.
2025/02/04 10:59:32  [INFO]   : 📄 Generating CatalogSource file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/cs-redhat-operator-index-v4-17.yaml file created
2025/02/04 10:59:32  [INFO]   : 📄 Generating ClusterCatalog file...
2025/02/04 10:59:32  [INFO]   : /home/skhoury/45580/working-dir/cluster-resources/cc-redhat-operator-index-v4-17.yaml file created
2025/02/04 10:59:32  [INFO]   : mirror time     : 38.550461664s
2025/02/04 10:59:32  [INFO]   : 👋 Goodbye, thank you for using oc-mirror

Verification of signature mirroring

$ curl https://sherinefedora:5000/v2/openshift4/ose-kube-rbac-proxy/tags/list | jq
{
 "name": "openshift4/ose-kube-rbac-proxy",
 "tags": [
   "a366ef7",
   "sha256-1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2",
   "sha256-1fc9d5afd6c486643f127ef8f4f8ef09eb8d206422b496a4cd365f0b4ab0a2c2.sig",
   "sha256-4488f5fdf6f10a551fe046e62cd4f3d856530cfb15831f7e1e2de531e7f2caeb",
   "sha256-4840b301e261cfb291697d907d98b177ba57a122e5563e42c8d831d3c42e5ef6.sig",
   "sha256-863efdda55c3da31afbf622c67b7336d61e73ee96ca762676c4b7d98f1e8f98c.sig",
   "sha256-c9c6c66b59e1775217ae048446b123817db2a8bc6ee729914101916dc29bcd3c.sig",
   "sha256-e4fd184a8691e7b8d2d8da561cec09e87f2825b66774fffeb50da6a9e0b515ff.sig"
 ]
}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@sherine-k
Copy link
Contributor Author

sherine-k commented Feb 4, 2025

@saschagrunert , @wking , @sub-mod , @mtrmac ,
I'm reaching out based on your involvement with OCPSTRAT-1245.
My first concern is to verify how the policy.json and registries.d files would be like on cluster:

  • lookaside URLs will not be available in a disconnected environment. The system on which oc-mirror will run would need to work with use-sigstore-attachments: true only. Is this what you also expect? Will this be aligned by MCO on the nodes for CRIO?
  • For the moment, the scope is to verify signatures for images delivered by Red Hat. Keys that should be mirrored alongside the images will be the ones referenced in the /etc/containers/policy.json on the operating system running oc-mirror. Is this what you expect should be mirrored as well?

cc @tlwu2013

Copy link

openshift-ci bot commented Feb 4, 2025

@sherine-k: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@lmzuccarelli
Copy link
Contributor

@sherine-k - could you re-title or put a hold on the PR for it not to be merged

@r4f4
Copy link
Contributor

r4f4 commented Feb 4, 2025

/label do-not-merge/work-in-progress

Copy link

openshift-ci bot commented Feb 4, 2025

@r4f4: The label(s) /label do-not-merge/work-in-progress cannot be applied. These labels are supported: acknowledge-critical-fixes-only, platform/aws, platform/azure, platform/baremetal, platform/google, platform/libvirt, platform/openstack, ga, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, px-approved, docs-approved, qe-approved, no-qe, downstream-change-needed, rebase/manual, cluster-config-api-changed, approved, backport-risk-assessed, bugzilla/valid-bug, cherry-pick-approved, jira/valid-bug, staff-eng-approved. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/label do-not-merge/work-in-progress

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@saschagrunert
Copy link
Member

cc @harche @QiWang19 ☝️

@aguidirh
Copy link
Contributor

aguidirh commented Feb 4, 2025

Hi @mtrmac,

Do you know if files like /etc/containers/registries.d/registry.redhat.io.yaml are going to work together with the registries.conf in containers/image?

Example:

/etc/containers/registries.d/registry.redhat.io.yaml

docker:
  registry.redhat.io:
    use-sigstore-attachments: true

registries.conf

[[registry]]
location="registry.redhat.io"
[[registry.mirror]]
location="enterprise-registry.in"

With both files above, is containers/image going to get the signatures from enterprise-registry.in instead of registry.redhat.io ?

@aguidirh
Copy link
Contributor

aguidirh commented Feb 4, 2025

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 4, 2025
@mtrmac
Copy link

mtrmac commented Feb 4, 2025

  • lookaside URLs will not be available in a disconnected environment. The system on which oc-mirror will run would need to work with use-sigstore-attachments: true only. Is this what you also expect?

I think that’s reasonable going forward. OCP’s ClusterImagePolicy/ImagePolicy also expose only the sigstore signatures, not the GPG signatures which require separate lookaside servers.

There is a chance that some customers might want to use GPG signatures (even with RHEL 10! conversations ongoing), so keep that in mind, but I don’t think it’s blocking for the first version. (GPG signatures are not exposed as an OCP custom resource either.)

  • For the moment, the scope is to verify signatures for images delivered by Red Hat. Keys that should be mirrored alongside the images will be the ones referenced in the /etc/containers/policy.json on the operating system running oc-mirror. Is this what you expect should be mirrored as well?

I wouldn’t expect oc-mirror to deal with keys at all (other than, maybe, check that the mirrored images are correctly signed to detect failures early). The way mirroring with ITMS/IDMS is designed, policy.json on an internet-connected node and a disconnected node look exactly the same, they only differ in registries.d/registries.conf. So, users should be able to use the same ClusterImagePolicy/ImagePolicy CRs as they do on connected clusters.

Or maybe oc-mirror handles transferring a much larger set of custom resources into the disconnected site? In that case it should be possible to transfer the ClusterImagePolicy/… CRs unmodified.

@mtrmac
Copy link

mtrmac commented Feb 4, 2025

Do you know if files like /etc/containers/registries.d/registry.redhat.io.yaml are going to work together with the registries.conf in containers/image?

“Work together”, yes. “Work as hypothesized”, no. registries.conf turns a “logical name” (visible in pod specs and policy.json into physical locations of mirrors. Then, registries.d defines how to access an individual physical location.

In this example, registries.d needs an entry configuring use-sigstore-attachments for enterprise-registry.in (and also one for registry.redhat.io if the system is connected and could fall back to direct access.)

@mtrmac
Copy link

mtrmac commented Feb 4, 2025

Then, registries.d defines how to access an individual physical location.

For example, for users that do want to configure lookaside/lookaside-staging, that would typically point at a different URL for every mirror. So registries.d needs to be based on physical locations, not on logical names.

@sherine-k
Copy link
Contributor Author

sherine-k commented Feb 5, 2025

@mtrmac , @tlwu2013

I wouldn’t expect oc-mirror to deal with keys at all (other than, maybe, check that the mirrored images are correctly signed to detect failures early). The way mirroring with ITMS/IDMS is designed, policy.json on an internet-connected node and a disconnected node look exactly the same, they only differ in registries.d/registries.conf. So, users should be able to use the same ClusterImagePolicy/ImagePolicy CRs as they do on connected clusters.

Or maybe oc-mirror handles transferring a much larger set of custom resources into the disconnected site? In that case it should be possible to transfer the ClusterImagePolicy/… CRs unmodified.

This makes me think that instead of copying keys, oc-mirror might want to

  • copy policy.json, registries.d folders (to be used by an oc-mirror instance in the enclave for mirroring from tar to final registry) under its output folder cluster-resources (where ITMS/IDMS is also generated),
    • these files are the ones present on the machine where oc-mirror runs, as configured by user
    • @mtrmac we need probably you to confirm how to choose entries between the /etc ones and $HOME/.containers
  • but also generate a ClusterImagePolicy for the disconnected cluster based on the above files
  • leave out any certificates, keys, lookaside sites for the end user to take care of separately

What are your thoughts?

@mtrmac
Copy link

mtrmac commented Feb 5, 2025

If I understand the above comment correctly, oc-mirror would create, for its own purposes, a copy of policy.json etc. along the data that is being transferred, and use that at the destination to write to the destination (and to check that the images written to the destination correctly follow the policy). That makes sense to me.

@mtrmac we need probably you to confirm how to choose entries between the /etc ones and $HOME/.containers

Right now that kind of information is not exposed — c/image doesn’t want to promise, as an API, that the set of configuration sources in the environment is never going to expand to new locations / new source types. [E.g., there is a proposal floating around, on cloud-hosted computers, to automatically use registry credentials from the cloud provider’s key service. I don’t know how likely that is to happen, but an API to “return the list of files relevant for the current configuration” would then return misleading results.] In some cases the APIs return “a human-consumable description” which is explicitly not designed to be machine-parseable.

I realize that this would need some solution, if the input needs to be the raw RHEL config files.


but also generate a ClusterImagePolicy for the disconnected cluster based on the above files

Reading this, I probably have a rather wrong model about the oc-mirror usage/environment assumptions. I was thinking that in the OCP product context, users primarily configure things via CRs like ClusterImagePolicy, and the configuration files generated from that by MCO (like policy.json and registries.d/*) are not the user-relevant UI. Under that (now dubious) assumption, users would primarily work in terms of CRs (whether deployed on a source cluster, or just provided to tools as configuration).

If the primary input to oc-mirror is actually the raw RHEL configuration like policy.json, registries.d, that changes things a bit. The OCP CRs expose only a subset of the raw RHEL feature set, so some conversions would not be possible.

And, on OCP, policy.json is the outcome of merging various CRs (several individual ClusterImagePolicy resources, allowedRegistries/blockedRegistries options of the cluster …); some of them would be a part of the OCP release and automatically updated over time, some would be user-created; so, hypothetically, starting with a source OCP cluster, having the cluster generate policy.json/registries.conf, and then trying to generate ClusterImagePolicy from those files, the code would probably want to somehow split the parts of the policy that originate from user-created CRs and that need to be explicitly configured at the destination, and parts of the policy that ship with OCP and should not be duplicated at the destination. That seems hard to do. Maybe there should be two paths, one for OCP-to-OCP mirroring, where configuration is maintained directly as CRs, and one for non-OCP-to-OCP mirroring, where configuration is either created from raw RHEL files, or maybe the user would have to manually set that up without the tools helping.

I’m afraid I never read much of the oc-mirror documentation at all, so I don’t have a good mental model.

At a very high level, I 100% agree that users should only need to define “the set of mirroring relationships” in one place, with tools then both guiding through the mirroring operation, and automatically generating related configurations, to the extent that can be automated, to avoid typos/mistakes, and unnecessary busywork.

@mtrmac
Copy link

mtrmac commented Feb 5, 2025

leave out any certificates, keys, lookaside sites for the end user to take care of separately

policy.json can refer to certificates/public keys via paths on the filesystem, or embed them inside the policy file directly. So, in some cases, the copying the policy would automatically include the keys. But if it refers to paths on the filesystem, they don’t all need to be copied separately — e.g. RHEL distributes the RH registry keys as RPMs, and the default RHEL policy refers to these files, does not embed copies into policy.json. (And any updates to the policy should continue to refer to the files, not create copies, because the RPMs can be updated over time.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants