Skip to content

Enable developer to customize CLI command options #2576

@HCharlie

Description

@HCharlie

Hey team,

I want to find a way to extend the already existing commands(which is listed below) to create some customized behavor to my flow python file, which is decorated by the customized decorators.

Currently, I've also explored the nflx metaflow extension template, but I feel it's a bit complex at the moment for developers to get onboarded, and really wish there's a simple way to do so like the custom decorator with a simple generator function, I wonder if there's something similar to the CLI part.

My current goal is to build something like this

python myflow.py <command>  <subcommand>

For example, when running python myflow.py feature plan, which takes the @feature decorator and it's step fun as input of the plan function, and process it.

(.venv) ➜  example python myflow.py --help
...
Commands:
  check           Check that the flow is valid (default).
  dump            Get data artifacts of a task or all tasks in a step.
  help            Show all available commands.
  output-dot      Visualize the flow with Graphviz.
  output-raw      Output internal state of the flow graph.
  resume          Resume execution of a previous run of this flow.
  run             Run the workflow locally.
  show            Show structure of the flow.
  version         Print the Metaflow version
  step-functions  Commands related to AWS Step Functions.
  argo-workflows  Commands related to Argo Workflows.
  card            Commands related to @card decorator.
  kubernetes      Commands related to Kubernetes.
  package         Commands related to code packages.
  airflow         Commands related to Airflow.
  tag             Commands related to tagging.
  spot-metadata   Commands related to spot metadata.
  batch           Commands related to AWS Batch.
  logs            Commands related to logs

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions