Mount volumes defined in a WorkflowTemplate when it's called by another workflow

[cscetbon]

Summary

What change needs making?

Volumes set on a WorkflowTemplate should be used when called by another workflow

Use Cases

When would you use this?

Here is what doesn't work today

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: template1
spec:
  entrypoint: template1
  volumeClaimTemplates:
  - metadata:
      name: my-pvc
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5M
  arguments: &parameters
    parameters: 
    - name: run
  templates:
    - name: template1
      inputs: *parameters
      steps:
        - - name: template10
            template: template10
            arguments:
              parameters:
                - name: run
                  value: "{{inputs.parameters.run}}"
        - - name: template11
            templateRef:
              name: template2
              template: template2

    - name: template10
      inputs: *parameters
      script:
        image: debian:9.4
        command: [bash]
        source: |
          echo "{{inputs.parameters.run}}" > /mnt/pvc/output
        volumeMounts:
          - name: my-pvc
            mountPath: /mnt/pvc

---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: template2
spec:
  entrypoint: template2
  templates:
    - name: template2
      script:
        image: debian:9.4
        command: [bash]
        source: |
          cat /mnt/pvc/output
        volumeMounts:
          - name: my-pvc
            mountPath: /mnt/pvc
---
apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
  name: cronwf-ex1
spec:
  schedule: 0 11 * * *
  concurrencyPolicy: Forbid
  workflowSpec:
    entrypoint: jobs
    # volumeClaimTemplates:
    # - metadata:
    #     name: my-pvc
    #   spec:
    #     accessModes: [ "ReadWriteOnce" ]
    #     resources:
    #       requests:
    #         storage: 5M
    templates:
    - name: jobs
      steps:
      - - name: template1
          templateRef:
            name: template1
            template: template1
          arguments:
            parameters:
            - name: run
              value: "{{item}}"
          withItems:
            - run1
            # -... 
            - run23

works well
argo submit --from wftmpl/template1 --watch

works only if the commented out lines on the cron workflow are uncommented
argo submit --from cronwf/cronwf-ex1 --watch

The issue with adding the volume on the cronworkflow is that it's shared by all the runX however I want to keep it in the template1 to make sure each runX uses a different PVC (gp2 can't use ReadWriteMany for instance).

volumes available on the workflow template should be automatically mounted when called by another template. It could be not mounted only if the calling workflow already has a corresponding volume for instance.

---

Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

[cscetbon]

I don't think it should be closed.

[sarabala1979]

It is not a bug. It is by design. if you are referring templateRef on workflow, it can't access the Spec level elements. PVC definition should be in workflow.

[wobrycki]

Hi @sarabala1979, is there any way to configure argo-workflows, so that volumeClaimTemplates: is always defined, per default, regardless of which workflow I start (like cron without volumeClaimTemplates: defined)?

Our use-case is similar. We have a template which needs a volume and has volumeMounts:.
It's used in may places (cron, other workflows). Now, we need to remember to always put volumeClaimTemplates into every "parent" workflow. Isn't it contradiction to the point of having reusable sub-templates?

[sarabala1979]

@wobrycki you can try workflowDefaults on workflow-controller configmap. WorkflowDefaults will be applied to all workflows that are executed on that controller.

argo-workflows/docs/default-workflow-specs.md

Line 32 in 256ff72

The following would be specified in the Config Map:

[apiwoni]

@wobrycki As a workaround for this limitation, I have used workflow-of-workflows pattern where new instance of workflow invokes re-usable template since I can utilize workflowTemplateRef in this pattern.