Skip to content

Conversation

@awalford16
Copy link
Collaborator

Adds an API endpoint of /object/move/$FILE which triggers a minio-copy workflow specifying the ingress bucket and file to be copied.

Copies the data to a PVC which is mounted into the workflow pod.

Authentication with minio is currently handled by the existing argo-artifacts-minio secret that exists in the pulumi (currently minio admin).

Test workflow result:

15:35:06 ❯ k logs minio-copy-kqpvx -n argo-workflows
Added `artifacts` successfully.
`artifacts/ingress/parallel-cluster.yaml` -> `/data/parallel-cluster.yaml`
┌──────────┬─────────────┬──────────┬─────────────┐
│ Total    │ Transferred │ Duration │ Speed       │
│ 1.08 KiB │ 1.08 KiB    │ 00m00s   │ 29.08 KiB/s │
└──────────┴─────────────┴──────────┴─────────────┘
time="2025-10-30T15:35:04.947Z" level=info msg="sub-process exited" argo=true error="<nil>"

@craddm craddm self-requested a review November 4, 2025 12:41
Comment on lines +15 to +18
volumes:
- name: workflow-data-ingress
persistentVolumeClaim:
claimName: workflow-data-ingress
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this parameters rather than a pre-set name

Comment on lines +125 to +137
workflow_data_pvc = PersistentVolumeClaim(
"workflow-data-pvc",
metadata=ObjectMetaArgs(
name="workflow-data-ingress",
namespace=argo_workflows_ns.metadata.name,
),
spec=PersistentVolumeClaimSpecArgs(
access_modes=["ReadWriteMany"],
storage_class_name="fridge",
resources=VolumeResourceRequirementsArgs(requests={"storage": "2Gi"}),
),
)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to be doing this here. This should be interfacing with the PVC defined in #140

Copy link
Contributor

@craddm craddm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should generalise the template and avoid creating PVCs as part of this PR - see comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants