CLI:options.timeout is not working

[mtuchi]

User story

In some cases i want my workflow to run longer the default timeout of 5minute ~ 300000ms.

Details

Using @openfn/cli v1.0.0 the options.timeout is not working, unless you specify the -t --timeout option when running your workflow.

Implementation notes

See sample workflow below 👇🏽 ,

{
  "options": {
    "start": "getUsers",
    "timeout": "300"
  },
  "workflow": {
    "steps": [
      {
        "id": "getUsers",
        "expression": "each([1,2,3,4,5],get(s=>`https://jsonplaceholder.typicode.com/users/${s.data}`))",
        "adaptor": "http"
      }
    ]
  }
}

If you run the workflow openfn worfklow.json -m -o tmp/output.json, The workflow will run successful but Ideally the workflow above should throw an error saying [R/T] ✘ Error: Timeout expired (300ms). But it doesn't fail unless you specify the timeout when running the workflow. Eg: openfn worfklow.json -m -o tmp/output.json -t 300.

[josephjclark]

Took a quick look at this with some spare time this afternoon.

It's actually quite a hard one.

The CLI uses yargs to default the value of timeout to 5 minutes.

When we load the workflow, we say "If the CLI specified an option as an argument (ie --timeout 100, use that value. Otherwise, use options from the plan". And to know the user arguments, we look at the yargs options.

But if yargs defaulted an option, it'll have a value and so override the plan value. We have no way of knowing right now whether a value is a default, or was explicitly set by the user.

The correct behaviour here is: prefer explicit CLI inputs to options on the plan.

The solutions I see are:

1. Don't use yargs default for timeout, or any other option that happens to be a workflow option, and set the default elsewhere. This feels like its introducing inconsistency.
2. Save a list somewhere of which arguments the user passed explicitly. I like this idea but I don't know where we'll get or save that list...