new limits page for v2

docs/jobs/limits.md

@@ -2,75 +2,53 @@
 title: Limits
 ---
 
-Both microservice and OpenFn.org (the iPaaS) have some pre-configured limits to
-ensure smooth operation. Read below for a detailed explanation of those defaults
-and how to adjust the limits.
+OpenFn has a number of configurable limits that help ensure smooth operation.
+This page details what those limits are and how they can be adjusted.
 
-:::tip
+:::tip Increasing limits
 
-If you don't feel like reading much, here's the tl;dr:
-
-1. Don't send more than `8MB` to the `/inbox` API in a single request.
-2. Make sure your `expression.js` and your `state.json` don't total more than
-   `10MB` when you _start_ or _finish_ a run.
-3. Keep memory usage below `512MB` _during_ the run.
+Contact [email protected] to increase limits on standard plans, configure
+custom limits on dedicated deployments, or get help adjusting limits on your
+locally deployment instance.
 
 :::
 
-## NodeVM memory limits
-
-The default memory limit for a Node process is `512MB` and unless you've
-customized it for use on core or have agreed to a high-memory plan on
-OpenFn.org, job runs will be killed by Node (`JavaScript heap out of memory`)
-when they reach that threshold.
-
-### Increase memory limit for an individual execution
-
-For a `1GB` limit for an individual job, use:
-
-```sh
-node --max-old-space-size=1024 core execute ...arguments
-```
-
-### Increase memory limit for all jobs on that machine
-
-For a `4GB` limit for all jobs, set an environment variable with:
-
-```sh
-export NODE_OPTIONS=--max_old_space_size=4096
-```
+## Workflow execution duration (1 hour)
 
-## Request body limit on inboxes
+Each workflow attempt needs to complete in less than `1 hour`. You can view the
+duration of each attempt by clicking on the attempt ID. If an attempt exceeds
+this limit, it will be killed by the worker and you'll see a `Killed:Timeout`
+badge as your attempt state.
 
-The maximum size of the JSON body accepted on the `/inbox/your-uuid` endpoint is
-`8MB`. Note that payloads should be kept below this limit in order to ensure
-that [Job state limits](#job-state-limits) are not hit when a subsequent job run
-is executed.
+> _Instance superusers can control this limit the `MAX_RUN_DURATION` environment
+> variable._
 
-:::note
+## Memory Usage (1GB)
 
-If you're using OpenFn for bulk data processing/periodic batch jobs, you can
-either reconfigure these limits on your microservice deployment or contact
-[email protected] to have special provisions made for your OpenFn.org
-project.
+Each workflow attempt may not use more than `1GB` of memory. You can view the
+maximum memory usage of each attempt by clicking on the attempt ID. If an
+attempt exceeds this limit, it will be killed by the worker and you'll see a
+`Killed:OOM` badge as your attempt state.
 
-:::
+> _Instance superusers can control this limit via the `MAX_RUN_MEMORY`
+> environment variable._
 
-## Job state limits
+## Dataclip Size (10MB)
 
-When a job is to be sent for processing the total size of all job artifacts
-(your job `expression` and the initial `state`) cannot exceed `10MB`. In other
-words, your code (`expression.js`) plus your data and configuration
-(`state.json`) cannot exceed `10MB` in total.
+1. Each **webhook request** to a trigger URL cannot exceed `10MB`.
+2. If you are persisting the final state of each **run** as a dataclip, each
+   dataclip may not exceed `10MB`.
+3. If you are persisting the final state of an **attempt** as a dataclip, it may
+   not exceed `10MB`.
 
-Similarly, when state is saved to cloud storage/disk _after_ execution there is
-a `10MB` limit to the size of a job's final `state`.
+If you send a payload to a webhook trigger URL which breaches this limit, the
+server will respond with a `413` error with a `:request_entity_too_large`
+message. If the dataclips produced by the final state of runs and attempts are
+too large, you will no be able to access them and instead see:
 
-If you're dealing with large amounts of data you've got to (a) keep your total
-memory use below `512MB` during execution and (b) clean up state so that
-whatever you're passing to the next job, whether via FLOW or via saved state for
-a CRON job, remains below `10MB`.
+```json
+{ "__lightning": "Run result too large to save" }
+```
 
-Sometimes, this necessitates streaming data from A -> B in the same job. Other
-times it will necessitate storing data as local variables and then posting data
-to your inbox in chunks before cleaning up state.
+> _Instance superusers can control this limit via the `MAX_DATACLIP_SIZE`
+> environment variable._