Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: expose image manifest digest of all copied artifacts #1086

Merged
merged 2 commits into from
Jun 21, 2024
Merged

fix: expose image manifest digest of all copied artifacts #1086

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0a3eaf6
fix: expose image manifest digest of all copied artifacts
ralphbean Jun 21, 2024
17c058b
Regenerate oci-copy-oci-ta.yaml
ralphbean Jun 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 6 additions & 45 deletions task/oci-copy-oci-ta/0.1/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,57 +1,18 @@
# oci-copy-oci-ta task

Given an `oci-copy.yaml` file in the user's source directory, the `oci-copy` task will copy content from arbitrary urls into the OCI registry.

It generates a limited SBOM and pushes that into the OCI registry alongside the image.

It is not to be considered safe for general use as it cannot provide a high degree of provenance for artficats and reports them only as "general" type artifacts in the purl spec it reports in the SBOM. Use only in limited situations.
Given a file in the user's source directory, copy content from arbitrary urls into the OCI registry.

## Parameters
|name|description|default value|required|
|---|---|---|---|
|IMAGE|Reference of the image buildah will produce.||true|
|SOURCE_ARTIFACT|The trusted artifact URI containing the application source code.||true|
|IMAGE|Reference of the image we will push||true|
|OCI_COPY_FILE|Path to the oci copy file.|./oci-copy.yaml|false|
|SOURCE_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the application source code.||true|

## Results
|name|description|
|---|---|
|IMAGE_DIGEST|Digest of the image just built|
|IMAGE_URL|Image repository where the built image was pushed|

## oci-copy.yaml schema
JSON schema for the `oci-copy.yaml` file.
|IMAGES|URIs for all image manifests published, for signing|
|IMAGE_DIGEST|Digest of the artifact just pushed|
|IMAGE_URL|Repository where the artifact was pushed|

```json
{
"type": "object",
"required": ["artifacts"],
"properties": {
"artifacts": {
"type": "array",
"items": {
"type": "object",
"required": ["source", "filename", "type", "sha256sum"],
"properties": {
"source": {
"description": "URL of the artifact to copy",
"type": "string"
},
"filename": {
"description": "Filename that should be applied to the artifact in the OCI registry",
"type": "string"
},
"type": {
"description": "Media type that should be applied to the artifact in the OCI registry",
"type": "string"
},
"sha256sum": {
"description": "Digest of the artifact to be checked before copy",
"type": "string"
}
}
}
}
}
}
```
126 changes: 71 additions & 55 deletions task/oci-copy-oci-ta/0.1/oci-copy-oci-ta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,49 +1,63 @@
---
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: oci-copy-oci-ta
annotations:
tekton.dev/pipelines.minVersion: 0.12.1
tekton.dev/tags: image-build, appstudio, hacbs
labels:
app.kubernetes.io/version: "0.1"
build.appstudio.redhat.com/build_type: oci-artifact
name: oci-copy-oci-ta
spec:
description: Given a file in the user's source directory, copy content from arbitrary urls into the OCI registry.
description: Given a file in the user's source directory, copy content from
arbitrary urls into the OCI registry.
params:
- description: Reference of the image we will push
name: IMAGE
type: string
- description: The Trusted Artifact URI pointing to the artifact with the application source code.
name: SOURCE_ARTIFACT
- name: IMAGE
description: Reference of the image we will push
type: string
- default: ./oci-copy.yaml
- name: OCI_COPY_FILE
description: Path to the oci copy file.
name: OCI_COPY_FILE
type: string
default: ./oci-copy.yaml
- name: SOURCE_ARTIFACT
description: The Trusted Artifact URI pointing to the artifact with
the application source code.
type: string
results:
- description: Digest of the artifact just pushed
name: IMAGE_DIGEST
- description: Repository where the artifact was pushed
name: IMAGE_URL
- name: IMAGES
description: URIs for all image manifests published, for signing
- name: IMAGE_DIGEST
description: Digest of the artifact just pushed
- name: IMAGE_URL
description: Repository where the artifact was pushed
volumes:
- name: varlibcontainers
emptyDir: {}
- name: workdir
emptyDir: {}
- name: workdir
emptyDir: {}
stepTemplate:
env:
- name: OCI_COPY_FILE
value: $(params.OCI_COPY_FILE)
- name: IMAGE
value: $(params.IMAGE)
- name: OCI_COPY_FILE
value: $(params.OCI_COPY_FILE)
volumeMounts:
- mountPath: "/var/workdir"
- mountPath: /var/workdir
name: workdir
- mountPath: /var/workdir
name: workdir
steps:
- image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:4e39fb97f4444c2946944482df47b39c5bbc195c54c6560b0647635f553ab23d
name: use-trusted-artifact
- name: use-trusted-artifact
image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:4e39fb97f4444c2946944482df47b39c5bbc195c54c6560b0647635f553ab23d
args:
- use
- $(params.SOURCE_ARTIFACT)=/var/workdir/source
- $(params.CACHI2_ARTIFACT)=/var/workdir/cachi2
- name: prepare
image: quay.io/konflux-ci/yq:latest@sha256:974dea6375ee9df561ffd3baf994db2b61777a71f3bcf0050c5dca91ac9b3430
workingDir: /var/workdir
script: |
set -eu
set -o pipefail
Expand All @@ -60,28 +74,20 @@ spec:
artifact_type=$(echo $entry | yq .type)
artifact_digest=$(echo $entry | yq .sha256sum)

echo "declare OCI_SOURCE=${source}" > /var/workdir/vars/$filename
echo "declare OCI_FILENAME=${filename}" >> /var/workdir/vars/$filename
echo "declare OCI_ARTIFACT_TYPE=${artifact_type}" >> /var/workdir/vars/$filename
echo "declare OCI_ARTIFACT_DIGEST=${artifact_digest}" >> /var/workdir/vars/$filename
echo "declare OCI_SOURCE=${source}" >/var/workdir/vars/$filename
echo "declare OCI_FILENAME=${filename}" >>/var/workdir/vars/$filename
echo "declare OCI_ARTIFACT_TYPE=${artifact_type}" >>/var/workdir/vars/$filename
echo "declare OCI_ARTIFACT_DIGEST=${artifact_digest}" >>/var/workdir/vars/$filename

echo "Wrote /var/workdir/vars/$filename with contents:"
cat /var/workdir/vars/$filename
done
workingDir: $(workspaces.source.path)

- name: oci-copy
image: quay.io/redhat-appstudio/buildah:v1.35.4@sha256:3d3575bb7d0df64abcf1f22f06e82101a945d03317db1f3caac12814f796d01c
computeResources:
limits:
memory: 1Gi
requests:
cpu: 250m
memory: 512Mi
securityContext:
capabilities:
add:
- SETFCAP
workingDir: /var/workdir
volumeMounts:
- mountPath: /var/lib/containers
name: varlibcontainers
script: |
set -eu
set -o pipefail
Expand All @@ -104,19 +110,39 @@ spec:

echo "Pushing conents to $IMAGE"
buildah manifest push \
--digestfile $(workspaces.source.path)/image-digest \
--digestfile /var/workdir/image-digest \
--authfile $HOME/.docker/config.json \
--all \
$IMAGE

cat "$(workspaces.source.path)"/image-digest | tee $(results.IMAGE_DIGEST.path)
echo -n "$IMAGE" | tee $(results.IMAGE_URL.path)
volumeMounts:
- mountPath: /var/lib/containers
name: varlibcontainers
workingDir: $(workspaces.source.path)
- image: quay.io/konflux-ci/yq:latest@sha256:974dea6375ee9df561ffd3baf994db2b61777a71f3bcf0050c5dca91ac9b3430
name: sbom-generate
IMAGE_INDEX_DIGEST=$(cat "/var/workdir"/image-digest)
echo -n "$IMAGE_INDEX_DIGEST" | tee "$(results.IMAGE_DIGEST.path)"
echo -n "$IMAGE" | tee "$(results.IMAGE_URL.path)"
buildah manifest inspect $IMAGE@$IMAGE_INDEX_DIGEST | tee "/var/workdir"/image-manifests
computeResources:
limits:
memory: 1Gi
requests:
cpu: 250m
memory: 512Mi
securityContext:
capabilities:
add:
- SETFCAP
- name: artifact-manifest-generate
image: quay.io/konflux-ci/yq:latest@sha256:974dea6375ee9df561ffd3baf994db2b61777a71f3bcf0050c5dca91ac9b3430
workingDir: /var/workdir
script: |
MANIFEST_DIGESTS=$(cat "/var/workdir"/image-manifests | yq -r '.manifests[].digest')
for MANIFEST_DIGEST in $MANIFEST_DIGESTS; do
reference="$IMAGE@$MANIFEST_DIGEST, "
echo "Writing ${reference} to $(results.IMAGES.path)"
echo -n "${reference}" >>$(results.IMAGES.path)
done
echo -n "$IMAGE@$(cat $(results.IMAGE_DIGEST.path))" >>$(results.IMAGES.path)
- name: sbom-generate
image: quay.io/konflux-ci/yq:latest@sha256:974dea6375ee9df561ffd3baf994db2b61777a71f3bcf0050c5dca91ac9b3430
workingDir: /var/workdir
script: |
cat >sbom-cyclonedx.json <<EOL
{
Expand All @@ -139,7 +165,6 @@ spec:
echo "Recording purl $purl"
yq -oj -i '.components += [ {"purl": "'$purl'", "type": "file", "name": "'$OCI_FILENAME'", "hashes": [{"alg": "SHA-256", "content": "'$OCI_ARTIFACT_DIGEST'"}], "externalReferences": [{"type": "distribution", "url": "'$OCI_SOURCE'"}]} ]' sbom-cyclonedx.json
done
workingDir: $(workspaces.source.path)
- name: upload-sbom
image: quay.io/redhat-appstudio/cosign:v2.1.1@sha256:c883d6f8d39148f2cea71bff4622d196d89df3e510f36c140c097b932f0dd5d5
args:
Expand All @@ -150,13 +175,4 @@ spec:
- --type
- cyclonedx
- $(params.IMAGE)
workingDir: $(workspaces.source.path)

volumes:
- emptyDir: {}
name: varlibcontainers
- emptyDir: {}
name: workdir
workspaces:
- description: Workspace containing the source artifacts to copy
name: source
workingDir: /var/workdir
1 change: 1 addition & 0 deletions task/oci-copy/0.1/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ It is not to be considered safe for general use as it cannot provide a high degr
|---|---|
|IMAGE_DIGEST|Digest of the image just built|
|IMAGE_URL|Image repository where the built image was pushed|
|IMAGES|A comma separate list of all the individual image manifests produced|

## Workspaces
|name|description|optional|
Expand Down
25 changes: 21 additions & 4 deletions task/oci-copy/0.1/oci-copy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ spec:
name: IMAGE_DIGEST
- description: Repository where the artifact was pushed
name: IMAGE_URL
- description: URIs for all image manifests published, for signing
name: IMAGES
stepTemplate:
env:
- name: OCI_COPY_FILE
Expand Down Expand Up @@ -100,14 +102,29 @@ spec:
--all \
$IMAGE

cat "$(workspaces.source.path)"/image-digest | tee $(results.IMAGE_DIGEST.path)
echo -n "$IMAGE" | tee $(results.IMAGE_URL.path)
IMAGE_INDEX_DIGEST=$(cat "$(workspaces.source.path)"/image-digest)
echo -n "$IMAGE_INDEX_DIGEST" | tee "$(results.IMAGE_DIGEST.path)"
echo -n "$IMAGE" | tee "$(results.IMAGE_URL.path)"
buildah manifest inspect $IMAGE@$IMAGE_INDEX_DIGEST | tee "$(workspaces.source.path)"/image-manifests
volumeMounts:
- mountPath: /var/lib/containers
name: varlibcontainers
workingDir: $(workspaces.source.path)
- image: quay.io/konflux-ci/yq:latest@sha256:974dea6375ee9df561ffd3baf994db2b61777a71f3bcf0050c5dca91ac9b3430
name: sbom-generate

- name: artifact-manifest-generate
image: quay.io/konflux-ci/yq:latest@sha256:974dea6375ee9df561ffd3baf994db2b61777a71f3bcf0050c5dca91ac9b3430
script: |
MANIFEST_DIGESTS=$(cat "$(workspaces.source.path)"/image-manifests | yq -r '.manifests[].digest')
for MANIFEST_DIGEST in $MANIFEST_DIGESTS; do
reference="$IMAGE@$MANIFEST_DIGEST, "
echo "Writing ${reference} to $(results.IMAGES.path)"
echo -n "${reference}" >> $(results.IMAGES.path)
done
echo -n "$IMAGE@$(cat $(results.IMAGE_DIGEST.path))" >> $(results.IMAGES.path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This imposes a pretty nasty limit on the number of images that this task can copy (before the result breaks the 4kb size limit). If the general expectation is that this task will be used for copying, say, 10 artifacts, we should be good. But maybe EC could (should it?) support signed index images referencing unsigned manifests?

workingDir: $(workspaces.source.path)

- name: sbom-generate
image: quay.io/konflux-ci/yq:latest@sha256:974dea6375ee9df561ffd3baf994db2b61777a71f3bcf0050c5dca91ac9b3430
script: |
cat >sbom-cyclonedx.json <<EOL
{
Expand Down
Loading