Scheduled workflows?

[abrunner94]

Is there or will there be a way to schedule workflows using CRON expressions?

[reasv]

I think the idea is you use sleep('date-time') instead
Not entirely sure how this works with code updates though.
Frankly the documentation leaves a lot of questions open at this point

[abrunner94]

I suppose I could use Vercel Cron Jobs to call a route every minute which looks for user-defined cron jobs in a database and then executes those workflows. It would only work with Vercel though

[alestivanello]

Is there or will there be a way to schedule workflows using CRON expressions?

I think so. You can set up a cron route and then run -> await start(yourWorkflow)

[pranaygp]

We do want to have native cron support in workflow. I'll write an RFC for it shortly. In the meantime, you can either

1. Use vercel cron to spin a route that does await start(yourWorkflow)
2. Write your cron inside a workflow and just start the workflow once. See examples below:

function cronWorkflow() {
  "use workflow";
  
  while (true) {
    await sleep("1 day"); // or whatever
    someStep(); // you can await if you want, but don't have to. This can "run in the background"
  }
}

function someStep() {
  "use workflow";

  // do things...
}

Note: I'm calling a workflow inside a workflow. This works but doesn't actually do anything today (it's treated as a normal function in the workflow) - but we are working on "child workflows" soon so this appears better in o11y etc (see #69)

Note 2: A problem with this approach is that the run is tied to whatever vercel deployment you start it on. If you deploy a new version where you changed something in the cron, you will need to cancel and start it again. Hence why the vercel cron approach might be better

If you care about consistently starting at the same time, no matter when you triggered the workflow, you can do this

function tillTomorrowAt6AM() {
  const now = new Date();
  return new Date(
    now.getFullYear(),
    now.getMonth(),
    now.getDate() + 1,
    6, 0, 0, 0
  );
}

function cronWorkflow() {
  "use workflow";
  
  while (true) {
    await sleep(tillTomorrowAt6AM());

    someStep();
  }
}

function someStep() {
  "use workflow";

  // do things...
}

[reasv]

If you implement native integration with cron, it would be nice if it was possible to define the behaviour whenever the previous workflow run is still ongoing. For batch jobs, this avoids accidentally spawning a second overlapping run if the previous one has still not completed.

Of course, it's always possible to use a database to do this, by keeping track of state (ie, whether the previous run is complete), but given that the purpose of this framework is to simplify running tasks reliably, I would think that this is definitely in-scope given the potential pitfalls of the database approach

[reasv]

I think what would be sufficient is an option to specify how many concurrent instances of the cron job can be running at the same time, and a boolean to choose whether to queue up or reject/cancel runs that would be triggered if not for the concurrency limit.

[pranaygp]

100% and that's a great callout @reasv. The Tags Spec might come in handy for supporting various kinds of idempotency like that so you can customize the behavior.

I'll spend more time thinking about this DX and share an RFC at some point for native scheduled syntax. There are a few important pieces that still need to be in place before that :)

[reasv]

Another idea here would be to allow max concurrency/queueing limits to be applied by unique tag value.
Assuming tags can be structured as "name -> value" as in the examples, you can have a tag like "OrgId => {ID}" and limit concurrent running jobs to a certain amount per unique OrgId so a single customer cannot use up your entire cloud hardware quota, for example. Already implementable using the database, by having a counter of in-flight jobs, of course - but again could be a feature to consider to simplify fairly common patterns, especially for SaaS applications.

[davidchalifoux]

@pranaygp Something to consider when working on cron support is having an API for programmatically managing them. I'm doing this via Trigger right now where I can allow end-users to schedule when certain tasks run. This would also require being able to attach multiple cron schedules to a single workflow with some sort of context (orgId/etc).