diff --git a/articles/2021-07-05-wrapping-my-head-around-jobs.md b/articles/2021-07-05-wrapping-my-head-around-jobs.md
index 8c22b7272bb..65ced6180e7 100644
--- a/articles/2021-07-05-wrapping-my-head-around-jobs.md
+++ b/articles/2021-07-05-wrapping-my-head-around-jobs.md
@@ -86,7 +86,7 @@ And for a fail triggered job like this:
```
No matter what, jobs start with state. See
-["Initial and final state for runs"](/documentation/build/steps/state/) for a detailed
+["Initial and final state for runs"](/documentation/jobs/state) for a detailed
breakdown.
## It ends with `state` too
diff --git a/docs/build-for-developers/jobs.md b/docs/build-for-developers/jobs.md
deleted file mode 100644
index eb9ea945b36..00000000000
--- a/docs/build-for-developers/jobs.md
+++ /dev/null
@@ -1,12 +0,0 @@
----
-title: Jobs
----
-
-Jobs define the specific series of "operations" (think: tasks or database actions) to be performed in an individual Workflow Step. See the [Steps section](../build/steps/steps.md) for detailed documentation and tips for [writing Job expressions](../build/steps/jobs.md).
-
-:::note
-
-In OpenFn V1, there was no concept of `Workflow Steps`--they were referred to as `Jobs`. In V2, we now refer to Jobs as the "job expressions" or "script" that define the business logic and transformation rules of individual `Steps`.
-
-:::
-
diff --git a/docs/build/steps/editing-locally.md b/docs/build/editing-locally.md
similarity index 100%
rename from docs/build/steps/editing-locally.md
rename to docs/build/editing-locally.md
diff --git a/docs/build/steps/each.md b/docs/build/steps/each.md
deleted file mode 100644
index d0c25fe0944..00000000000
--- a/docs/build/steps/each.md
+++ /dev/null
@@ -1,93 +0,0 @@
----
-id: each
-title: The each(...) operation
----
-
-The `each` operation allows you to perform another operation on each item in an
-array.
-
-## Each takes two arguments
-
-In other words, `each(arrayPath, operation)` will _do_ `operation` on each item
-it finds in the `arrayPath` array. It takes just two arguments:
-
-1. an arrayPath
-2. an operation(...)
-
-### arrayPath
-
-Let's look at the first argument in `each`... the path to the array. Consider
-the following code using the Salesforce adaptor:
-
-```js
-each(
- dataPath('form.participants[*]'),
- upsert(
- 'Person__c',
- 'Participant_Identification_Number_PID__c',
- fields(
- field('Participant_Identification_Number_PID__c', dataValue('pid')),
- relationship('RecordType', 'Name', 'Participant'),
- field('First_Name__c', dataValue('participant_first_name')),
- field('Surname__c', dataValue('participant_surname')),
- field('Mobile_Number_1__c', dataValue('mobile_number'))
- field('Sex__c', dataValue('gender')),
- )
- )
-);
-```
-
-This will upsert a `Person__c` resource in Salesforce for each item found in the
-`state.data.form.participants` array. You could specify this path in the
-following ways:
-
-- `'$.data.form.participants[*]'`
-- `dataPath('form.participants[*]')`
-
-Note the JSON path syntax.
-
-### the operation
-
-If there are 5 participants in there, it will execute the `upsert` operation on
-all 5 items, in sequence. `upsert` takes whatever arguments it takes normally
-but it operates _inside_ the array. See below for more details on the _scope_ of
-this operation.
-
-## dataValue(...) _inside_ each(...)
-
-Note that inside the `each(...)` operation, using `dataValue(path)` will
-evaluate a path inside each item in the array.
-
-## merge(...) and bringing data 'down' into an array:
-
-What if you want to access data in your `upsert` operation that does _not_ exist
-in the array itself. You could use a data preparation step (see: `alterState`)
-or make use of `merge(path, data)` which allows you to merge data from the
-initial scope down into your array and access it from the `upsert` operation.
-
-```js
-each(
- merge(
- dataPath('form.participants[*]'),
- fields(
- field('school_id', dataValue('form.school.id')),
- field('intervention_type', dataValue('form.type'))
- )
- ),
- upsert(
- 'Person__c',
- 'Participant_Identification_Number_PID__c',
- fields(
- field('Participant_Identification_Number_PID__c', dataValue('pid')),
- relationship('RecordType', 'Name', 'Participant'),
- field('First_Name__c', dataValue('participant_first_name')),
- field('Surname__c', dataValue('participant_surname')),
- field('Mobile_Number_1__c', dataValue('mobile_number'))
- field('Sex__c', dataValue('gender')),
- // new fields...
- field('School__c', dataValue('school_id')),
- field('Intervention_Type__c', dataValue('intervention_type'))
- )
- )
-);
-```
diff --git a/docs/build/steps/jobs.md b/docs/build/steps/jobs.md
deleted file mode 100644
index cf510cbf7e9..00000000000
--- a/docs/build/steps/jobs.md
+++ /dev/null
@@ -1,601 +0,0 @@
----
-title: Write Jobs
-sidebar_label: Write Jobs
----
-
-To define the business logic and data transformation rules or logic for
-individual `Steps` in your workflow, you will need to write a `Job`. This
-article will provide a basic overview of Job expressions & writing tips.
-
-:::tip
-
-For example Jobs written by the OpenFn core team and other users, check out the
-[Library](/adaptors/library) or other project repositories under
-[Github.com/OpenFn](https://github.com/OpenFn).
-
-:::
-
-## About Jobs
-
-A `Job` is evaluated as a JavaScript expression and primarily defines the
-specific series of [Operations](/docs/build/steps/operations.md) (think: tasks,
-database actions, custom functions) to be performed in a specific Workflow Step.
-
-In most cases, a Job is a series of `create` or `upsert` operations that are
-triggered by a webhook event (e.g., form submission forwarded from ODK mobile
-data collection app) or cron (e.g., daily @ 12:00). See this basic example:
-
-```js
-create(
- 'Patient__c',
- fields(
- field('Name', dataValue('form.surname')),
- field('Other Names', dataValue('form.firstName')),
- field('Age__c', dataValue('form.ageInYears')),
- field('Is_Enrolled__c', true),
- field('Enrollment_Status__c', 3)
- )
-);
-```
-
-This would create a new `Patient__c` record in the connected app. The patient's
-`Name` will be mapped from the form submission forwarded to OpenFn from the
-source mobile data collection app via a webhook request. Then, because we assume
-that all Patients registered in the mobile app are always considered "enrolled",
-we hard code the patient enrollment status as true - see the mapping for
-`Is_Enrolled__c`.
-
-The functions see above is OpenFn's own syntax, and you've got access to dozens
-of common "helper functions" like `dataValue(path)` and destination specific
-functions like `create(object, attributes)`. You can choose to either write jobs
-using this OpenFn syntax or write your own custom, anonymous functions in
-JavaScript to do whatever your heart desires. Remember that **Steps are
-evaluated as JavaScript**.
-
-### dataValue
-
-The most commonly used "helper function" is `dataValue(...)`. This function
-takes a single argument—the _path_ to some data that you're trying to access
-inside the message that has triggered a particular run. In the above example,
-you'll notice that `Is_Enrolled__c` is _always_ set to `true`, but `Name` will
-change for each message that triggers the running of this step. It's set to
-`dataValue('form.surname')` which means it will set `Name` to whatever value is
-present at `state.data.form.surname` for the triggering webhook request. It
-might be Bob for one message, and Alice for another.
-
-:::note
-
-Note that for message-triggered steps, `state` will always have it's `data` key
-(i.e., `state.data`) set to the body of the triggering webhook request (aka HTTP
-request).
-
-I.e., `dataValue('some.path') === state.data.some.path`, as evaluated at the
-time that the operation (`create` in the above expression) is executed.
-
-:::
-
-### A Job with custom JavaScript
-
-To write your own custom JavaScript functions, simply add an `fn(...)` block to
-your code as below.
-
-```js
-fn(state => {
- //write your own function to manipulate/transform state
- return state;
-});
-```
-
-Alternatively, you can add custom JavaScript code in-line any Adaptor-specific
-functions. See example job below where JavaScript was added to transform the
-data value outputted for `Name`.
-
-```js
-create(
- 'Patient__c',
- fields(
- field('Name', state => {
- console.log('Manipulate state to get your desired output.');
- return Array.apply(null, state.data.form.names).join(', ');
- }),
- field('Age__c', 7)
- )
-);
-```
-
-Here, the patient's name will be a comma separated concatenation of all the
-values in the `patient_names` array from our source message.
-
-## Available JavaScript Globals
-
-For security reasons, users start with access to the following standard
-JavaScript globals, and can request more by opening an issue on Github:
-
-- [`Array`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array)
-- [`console`](https://nodejs.org/api/console.html)
-- [`JSON`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON)
-- [`Number`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number)
-- [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise)
-- [`String`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String)
-
-## Examples of adaptor-specific functions
-
-**N.B.: This is just a sample.** There are lots more available in the
-[Adaptors docs](/adaptors/) and
-[repository](https://github.com/OpenFn/adaptors).
-
-### language-common
-
-These are available functions "common" across Adaptors. You can use any of these
-when writing Job expressions.
-
-- `field('destination_field_name__c', 'value')` Returns a key, value pair in an
- array.
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/common/src/Adaptor.js#L364)
-- `fields(list_of_fields)` zips key value pairs into an object.
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/common/src/Adaptor.js#L377)
-- `dataValue('JSON_path')` Picks out a single value from source data.
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/common/src/Adaptor.js#L146)
-- `each(JSON_path, operation(...))` Scopes an array of data based on a JSONPath
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/common/src/Adaptor.js#L262).
- See beta.each when using multiple each()'s in an expression.
-- `each(merge(dataPath("CHILD_ARRAY[*]"),fields(field("metaId", dataValue("*meta-instance-id*")),field("parentId", lastReferenceValue("id")))), create(...))`
- merges data into an array then creates for each item in the array
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/common/src/Adaptor.js#L396)
-- `lastReferenceValue('id')` gets the sfID of the last item created
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/common/src/Adaptor.js#L175)
-- `fn(state){return state.references[state.references.length-N].id})` gets the
- sfID of the nth item created
-
-#### each()
-
-For more on the `each(...)` operation, see
-[this page](/documentation/build/steps/each) and the below example.
-
-```js
-each(
- dataPath('csvData[*]'),
- upsertTEI(
- 'aX5hD4qUpRW', //piirs uid
- {
- trackedEntityType: 'bsDL4dvl2ni',
- orgUnit: dataValue('OrgUnit'),
- attributes: [
- {
- attribute: 'aX5hD4qUpRW',
- value: dataValue('aX5hD4qUpRW'),
- },
- {
- attribute: 'MxQPuS9G7hh',
- value: dataValue('MxQPuS9G7hh'),
- },
- ],
- },
- { strict: false }
- )
-);
-```
-
-### Salesforce
-
-See below for some example functions for the Salesforce Adaptor. See the
-[Adaptor](/adaptors/) for more.
-
-- `create("DEST_OBJECT_NAME__C", fields(...))` Create a new object. Takes 2
- parameters: An object and attributes.
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/salesforce/src/Adaptor.js#L466-L480)
-- `upsert("DEST_OBJECT_NAME__C", "DEST_OBJECT_EXTERNAL_ID__C", fields(...))`
- Creates or updates an object. Takes 3 paraneters: An object, an ID field and
- attributes.
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/salesforce/src/Adaptor.js#L539-L560)
-- `relationship("DEST_RELATIONSHIP_NAME__r", "EXTERNAL_ID_ON_RELATED_OBJECT__C", "SOURCE_DATA_OR_VALUE")`
- Adds a lookup or 'dome insert' to a record.
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/salesforce/src/Adaptor.js#L49-L56)
-
-### dhis2
-
-See below for some example functions for the DHIS2 Adaptor. See the
-[Adaptor](/adaptors/) for more.
-
-- `create('events', {..})` Creates an event.
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/dhis2/src/Adaptor.js#L137-L142)
-- `create('dataValueSet', {..})` Send data values using the dataValueSets
- resource
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/dhis2/src/Adaptor.js#L185-L191)
-
-### OpenMRS
-
-See below for some example functions for the OpenMRS Adaptor. See the
-[Adaptor](/adaptors/) for more.
-
-- `create('person',{..})` Takes a payload of data to create a person
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/openmrs/src/Adaptor.js#L292-L311)
-- `createPatient(...)` Takes a payload of data to create a patient
- [(source)](https://github.com/OpenFn/adaptors/blob/main/packages/openmrs/src/Adaptor.js#L292-L311)
-
-## Snippets and samples
-
-Below you can find some examples of block code for different functions and data
-handling contexts.
-
-
-Step expression (for commcare to SF)
-
-The following step expression will take a matching receipt and use data from
-that receipt to upsert a `Patient__c` record in Salesforce and create multiple
-new `Patient_Visit__c` (child to Patient) records.
-
-```js
-upsert(
- 'Patient__c',
- 'Patient_Id__c',
- fields(
- field('Patient_Id__c', dataValue('form.patient_ID')),
- relationship('Nurse__r', 'Nurse_ID_code__c', dataValue('form.staff_id')),
- field('Phone_Number__c', dataValue('form.mobile_phone'))
- )
-),
- each(
- join('$.data.form.visits[*]', '$.references[0].id', 'Id'),
- create(
- 'Visit__c',
- fields(
- field('Patient__c', dataValue('Id')),
- field('Date__c', dataValue('date')),
- field('Reason__c', dataValue('why_did_they_see_doctor'))
- )
- )
- );
-```
-
-
-
-
-Accessing the "data array" in Open Data Kit submissions
-
-Notice how we use "each" to get data from each item inside the "data array" in
-ODK.
-
-```js
-each(
- '$.data.data[*]',
- create(
- 'ODK_Submission__c',
- fields(
- field('Site_School_ID_Number__c', dataValue('school')),
- field('Date_Completed__c', dataValue('date')),
- field('comments__c', dataValue('comments')),
- field('ODK_Key__c', dataValue('*meta-instance-id*'))
- )
- )
-);
-```
-
-
-
-
-ODK to Salesforce: create parent record with many children from parent data
-
-Here, the user brings `time_end` and `parentId` onto the line items from the
-parent object.
-
-```js
-each(
- dataPath('data[*]'),
- combine(
- create(
- 'transaction__c',
- fields(
- field('Transaction_Date__c', dataValue('today')),
- relationship(
- 'Person_Responsible__r',
- 'Staff_ID_Code__c',
- dataValue('person_code')
- ),
- field('metainstanceid__c', dataValue('*meta-instance-id*'))
- )
- ),
- each(
- merge(
- dataPath('line_items[*]'),
- fields(
- field('end', dataValue('time_end')),
- field('parentId', lastReferenceValue('id'))
- )
- ),
- create(
- 'line_item__c',
- fields(
- field('transaction__c', dataValue('parentId')),
- field('Barcode__c', dataValue('product_barcode')),
- field('ODK_Form_Completed__c', dataValue('end'))
- )
- )
- )
- )
-);
-```
-
-
-
-
-Salesforce: perform an update
-
-```js
-update("Patient__c", fields(
- field("Id", dataValue("pathToSalesforceId")),
- field("Name__c", dataValue("patient.first_name")),
- field(...)
-));
-```
-
-
-
-
-Salesforce: Set record type using 'relationship(...)'
-
-```js
-create(
- 'custom_obj__c',
- fields(
- relationship(
- 'RecordType',
- 'name',
- dataValue('submission_type'),
- field('name', dataValue('Name'))
- )
- )
-);
-```
-
-
-
-
-Salesforce: Set record type using record Type ID
-
-```js
-each(
- '$.data.data[*]',
- create(
- 'fancy_object__c',
- fields(
- field('RecordTypeId', '012110000008s19'),
- field('site_size', dataValue('size'))
- )
- )
-);
-```
-
-
-
-
-Telerivet: Send SMS based on Salesforce workflow alert
-
-```js
-send(
- fields(
- field(
- 'to_number',
- dataValue(
- 'Envelope.Body.notifications.Notification.sObject.phone_number__c'
- )
- ),
- field('message_type', 'sms'),
- field('route_id', ''),
- field('content', function (state) {
- return 'Hey there. Your name is '.concat(
- dataValue('Envelope.Body.notifications.Notification.sObject.name__c')(
- state
- ),
- '.'
- );
- })
- )
-);
-```
-
-
-
-
-Sample DHIS2 events API step
-
-```js
-event(
- fields(
- field('program', 'eBAyeGv0exc'),
- field('orgUnit', 'DiszpKrYNg8'),
- field('eventDate', dataValue('properties.date')),
- field('status', 'COMPLETED'),
- field('storedBy', 'admin'),
- field('coordinate', {
- latitude: '59.8',
- longitude: '10.9',
- }),
- field('dataValues', function (state) {
- return [
- {
- dataElement: 'qrur9Dvnyt5',
- value: dataValue('properties.prop_a')(state),
- },
- {
- dataElement: 'oZg33kd9taw',
- value: dataValue('properties.prop_b')(state),
- },
- {
- dataElement: 'msodh3rEMJa',
- value: dataValue('properties.prop_c')(state),
- },
- ];
- })
- )
-);
-```
-
-
-
-
-merge many values into a child path
-
-```js
-each(
- merge(
- dataPath("CHILD_ARRAY[*]"),
- fields(
- field("metaId", dataValue("*meta-instance-id*")),
- field("parentId", lastReferenceValue("id"))
- )
- ),
- create(...)
-)
-```
-
-
-
-
-arrayToString
-
-```js
-arrayToString(arr, separator_string);
-```
-
-
-
-
-access an image URL from an ODK submission
-
-```js
-// In ODK the image URL is inside an image object...
-field("Photo_URL_text__c", dataValue("image.url")),
-```
-
-
-
-
-Use external ID fields for relationships during a bulk load in Salesforce
-
-```js
-fn(state) => {
- const patients = array.map(item => {
- return {
- Patient_Name__c: item.fullName,
- 'Account.Account_External_ID__c': item.account
- 'Clinic__r.Unique_Clinic_Identifier__c': item.clinicId,
- 'RecordType.Name': item.type,
- };
- });
-
- return {...state, patients}
-}
-```
-
-
-
-
-Bulk upsert with an external ID in salesforce
-
-```js
-bulk(
- 'Visit_new__c',
- 'upsert',
- {
- extIdField: 'commcare_case_id__c',
- failOnError: true,
- allowNoOp: true,
- },
- dataValue('patients')
-);
-```
-
-
-
-## Anonymous Functions
-
-Different to [Named Functions](#examples-of-adaptor-specific-functions),
-Anonymous functions are generic pieces of JavaScript which you can write to suit
-your needs. Read on for some examples of these custom functions.
-
-### Custom replacer
-
-```js
-field('destination__c', state => {
- console.log(something);
- return dataValue('path_to_data')(state).toString().replace('cats', 'dogs');
-});
-```
-
-This will replace all "cats" with "dogs" in the string that lives at
-`path_to_data`.
-
-> **NOTE:** The JavaScript `replace()` function only replaces the first instance
-> of whatever argument you specify. If you're looking for a way to replace all
-> instances, we suggest you use a regex like we did in the
-> [example](#custom-concatenation-of-null-values) below.
-
-### Custom arrayToString
-
-```js
-field("target_specie_list__c", function(state) {
- return Array.apply(
- null, sourceValue("$.data.target_specie_list")(state)
- ).join(', ')
-}),
-```
-
-It will take an array, and concatenate each item into a string with a ", "
-separator.
-
-### Custom concatenation
-
-```js
-field('ODK_Key__c', function (state) {
- return dataValue('metaId')(state).concat('(', dataValue('index')(state), ')');
-});
-```
-
-This will concatenate two values.
-
-### Concatenation of null values
-
-This will concatenate many values, even if one or more are null, writing them to
-a field called Main_Office_City_c.
-
-```js
-...
- field("Main_Office_City__c", function(state) {
- return arrayToString([
- dataValue("Main_Office_City_a")(state) === null ? "" : dataValue("Main_Office_City_a")(state).toString().replace(/-/g, " "),
- dataValue("Main_Office_City_b")(state) === null ? "" : dataValue("Main_Office_City_b")(state).toString().replace(/-/g, " "),
- dataValue("Main_Office_City_c")(state) === null ? "" : dataValue("Main_Office_City_c")(state).toString().replace(/-/g, " "),
- dataValue("Main_Office_City_d")(state) === null ? "" : dataValue("Main_Office_City_d")(state).toString().replace(/-/g, " "),
- ].filter(Boolean), ',')
- })
-```
-
-> Notice how this custom function makes use of the **regex** `/-/g` to ensure
-> that all instances are accounted for (g = global search).
-
-### Custom Nth reference ID
-
-If you ever want to retrieve the FIRST object you created, or the SECOND, or the
-Nth, for that matter, a function like this will do the trick.
-
-```js
-field('parent__c', function (state) {
- return state.references[state.references.length - 1].id;
-});
-```
-
-See how instead of taking the id of the "last" thing that was created in
-Salesforce, you're taking the id of the 1st thing, or 2nd thing if you replace
-"length-1" with "length-2".
-
-### Convert date string to standard ISO date for Salesforce
-
-```js
-field('Payment_Date__c', function (state) {
- return new Date(dataValue('payment_date')(state)).toISOString();
-});
-```
-
-> **NOTE**: The output of this function will always be formatted according to
-> GMT time-zone.
diff --git a/docs/build/steps/multiple-operations.md b/docs/build/steps/multiple-operations.md
deleted file mode 100644
index cad33a9576c..00000000000
--- a/docs/build/steps/multiple-operations.md
+++ /dev/null
@@ -1,54 +0,0 @@
----
-title: Chaining operations in 1 Step
-sidebar_label: Chaining operations
----
-
-This page describes why you might want to chain multiple operations in a single Job when designing your Step.
-
-
-
-### Reasons to use multiple operations in a single Job
-
-- Your desired final `Output` for a Step requires multiple operations to produce
- (e.g., get Patients to check if existing, then update existing Patient
- records).
-- The job must be atomic, you want the whole thing to count as a failure if any
- part of it fails.
-- You run Steps manually and you want a single button to click to retry the
- entire sequence of operations.
-- You update a `cursor` in a series of operations that involve `GET` and `POST`.
- When the `POST` fails, you don't want to update the `cursor` for the
- subsequent job run which contains the `GET`.
-
- See example Job below with 2 "upsert" (update/insert) operations.
-
- ```js
- //first we upsert the Patient
- upsert(
- 'Patient__c',
- 'ExternalId',
- fields(
- field('ExternalId', dataValue('form.patient_id')),
- field('Name', dataValue('form.surname')),
- field('Other Names', dataValue('form.firstName')),
- field('Age__c', dataValue('form.ageInYears')),
- field('Is_Enrolled__c', true),
- field('Enrollment_Status__c', 3)
- )
- );
- //then we upsert the related Visit (child to the Patient record)
- create(
- 'Visit__c',
- 'ExternalId',
- fields(
- field('ExternalId', dataValue('form.visit_id')),
- relationship('Patient__r', dataValue('form.patient_id')),
- field('Name', dataValue('form.surname')),
- field('Other Names', dataValue('form.firstName')),
- field('Age__c', dataValue('form.ageInYears')),
- field('Is_Enrolled__c', true),
- field('Enrollment_Status__c', 3)
- )
- );
- ```
-Check out the Job [Library Examples](/adaptors/library/) for more examples.
diff --git a/docs/build/steps/operations.md b/docs/build/steps/operations.md
deleted file mode 100644
index 8ffad28b07e..00000000000
--- a/docs/build/steps/operations.md
+++ /dev/null
@@ -1,36 +0,0 @@
----
-title: Operations
----
-
-In a Job expression, an Operation is a function which returns a function which takes `state` and
-returns a `Promise` or `state`.
-
-The purpose of an Operation is to act as an unresolved unit of behaviour.
-
-For example, when drafting a Job expression - the code itself doesn't know what the
-state is going to be, only what _it's going to do_.
-
-Adaptors all follow this convention, where the functions that are provided
-all return Operations.
-
-```javascript
-create('My_Custom_Object__c', {
- Custom_Field__c: dataValue('foo'),
-});
-```
-
-In the snippet above, the `create` function doesn't know anything about
-credentials, or any dynamic data that you may be available at runtime.
-
-```javascript
-function create(objectName, data) {
- return function (state) {
- // expand the data argument using state
- // actually do the work
- };
-}
-```
-
-In this snippet is a simple example of what most functions in OpenFn look like.
-The `create` function returns a function that takes state, this is an
-`Operation`. The runtime using `execute` will call all Operations with `state`.
diff --git a/docs/build/steps/state.md b/docs/build/steps/state.md
deleted file mode 100644
index 1796cf5d359..00000000000
--- a/docs/build/steps/state.md
+++ /dev/null
@@ -1,82 +0,0 @@
----
-title: Initial and final state
----
-
-Each Step requires an input state and (in most cases) will produce an output
-state. This article explains these concepts in greater detail.
-
-
-
-## Input & output state for runs
-
-Depending on whether you're running Workflows locally or on the app, the input
-`state` for a Run can be generated differently. When creating a work order by
-hand, you must select or generate your input manually (e.g., by creating a
-custom `Input` on the app or `state.json` file if working locally
-[in the CLI](/docs/build-for-developers/cli-intro.md)). When a work order is
-automatically created via a webhook trigger or cron trigger, state will be
-created as described below.
-
-The final state of a Run is determined by what's returned from the last
-operation. Remember that job expressions are a series of `operations`—they each
-take `state` and return `state`, after creating any number of side effects. You
-can control what is returned at the end of all of these operations.
-
-### Webhook triggered runs
-
-Initial state contains important parts of the inbound **http request**.
-
-```js
-{
- data: httpRequest.body,
- request: { headers: httpRequest.headers },
- configuration: credential.body
-}
-```
-
-### Cron triggered runs
-
-Initiate state is either an empty object `{}` or the final state of the **last
-succesful run** for this workflow.
-
-```js
-{
- ...finalStateOfLastSuccessfulRun,
- configuration: credential.body
-}
-```
-
-## Input & output state for steps
-
-State is also passed between each step in a workflow. The output state of the
-previous step is used as the input state for the next step.
-
-### On success
-
-When a job succeeds, its output state will be whatever is returned by the last
-operation.
-
-```js
-{
- data: "blah",
- references: [1, 2, 3]
-}
-```
-
-### On failure
-
-When a step in a workflow fails, the error will be added to an `errors` object
-on state, keyed by the ID of the job that failed.
-
-```js
-{
- data: "blah",
- references: [1, 2, 3],
- errors: { jobId: error }
-}
-```
-
-See the below diagram for a visual description of how state might be passed
-between Steps in a Workflow.
-
-
diff --git a/docs/build/steps/step-design-intro.md b/docs/build/steps/step-design-intro.md
index 43e9494cf35..7550098f0ff 100644
--- a/docs/build/steps/step-design-intro.md
+++ b/docs/build/steps/step-design-intro.md
@@ -8,8 +8,8 @@ Read on for a brief overview.
:::tip
-Check out the [Workflow Design](/documentation/design/design-overview) docs
-for more details on solution design and links to templates.
+Check out the [Workflow Design](/documentation/design/design-overview) docs for
+more details on solution design and links to templates.
:::
@@ -29,9 +29,9 @@ and consider summarizing your design specifications in a
### 2: Map your data elements
-[See here](/documentation/design/mapping-specs) for detailed guidance on
-mapping data elements or "data dictionaries" between your source and destination
-apps. To get started:
+[See here](/documentation/design/mapping-specs) for detailed guidance on mapping
+data elements or "data dictionaries" between your source and destination apps.
+To get started:
1. Export the metadata (or "form", "field list", or "data elements") of your
source app (input) & destination app (output).
@@ -63,7 +63,7 @@ apps. To get started:
- `updateTEI(...)`
- `upsertTEI(...)`
-See example [Job expression](/docs/build/steps/jobs.md) for a Step
+See example [Job expression](/documentation/jobs/job-writing-guide) for a Step
that will "upsert" (update or insert) records in a SQL database.
```js
diff --git a/docs/build/steps/step-editor.md b/docs/build/steps/step-editor.md
index 02634ba145c..ffb882249a1 100644
--- a/docs/build/steps/step-editor.md
+++ b/docs/build/steps/step-editor.md
@@ -20,8 +20,8 @@ To access this interface:
For more on how to write custom business logic and data transformation rules in
the `Editor`, see the docs on
-[writing Jobs](/documentation/build/steps/jobs) and check out
-the below video.
+[writing Jobs](/documentation/jobs/job-writing-guide) and check out the below
+video.
@@ -40,7 +40,6 @@ include `Logs` and an `Output`.
- `Logs` - A record generated by the workflow execution engine that details the
activities performed when running a Workflow or individual Step.
-See [Writing Jobs docs](/documentation/build/steps/jobs) for
-more on writing custom logic, and see
-[this article](/documentation/build/steps/state) for more on the concept of
-"state" when writing Jobs and building OpenFn Workflows.
+See [Writing Jobs docs](/documentation/jobs/job-writing-guide) for more on
+writing custom logic, and see [this article](/documentation/jobs/state) for more
+on the concept of "state" when writing Jobs and building OpenFn Workflows.
diff --git a/docs/build/steps/steps.md b/docs/build/steps/steps.md
index 506de66354d..ac31dc48470 100644
--- a/docs/build/steps/steps.md
+++ b/docs/build/steps/steps.md
@@ -6,6 +6,14 @@ A Step is a specific task or activity in a workflow. Each Step is linked to an
[Adaptor](/adaptors/) and contains business logic to perform a specific task or
operation in that target app. Read on to learn more.
+:::note
+
+In OpenFn V1, there was no concept of `Workflow Steps`--they were referred to as
+`Jobs`. In V2, we now refer to Jobs as the "job expressions" or "script" that
+define the business logic and transformation rules of individual `Steps`.
+
+:::
+
## Create or edit a Step
Via the Workflow Canvas, click the plus `+` icon to create a _new_ Step, or
@@ -134,4 +142,4 @@ want this and to avoid the risk of accidental upgrades on live Workflows.
Click the code button `` displayed on the configuration panel to write or
edit a Job expression to define the "rules" or the specific tasks to be
completed by your Step. See the pages on [the Inspector](./step-editor.md) and
-[writing Jobs](./jobs.md) to learn more.
+[writing Jobs](/documentation/jobs/job-writing-guide) to learn more.
diff --git a/docs/build/steps/working-with-branches.md b/docs/build/working-with-branches.md
similarity index 80%
rename from docs/build/steps/working-with-branches.md
rename to docs/build/working-with-branches.md
index 636d0c20e88..d58d6299a58 100644
--- a/docs/build/steps/working-with-branches.md
+++ b/docs/build/working-with-branches.md
@@ -3,13 +3,14 @@ title: Manage changes with Github branches
sidebar_label: Manage changes
---
-In the [Edit Steps Locally](/documentation/build/steps/editing-locally)
-section, we walked through the process of creating and adding your changes to
-the `main` branch of a project.
+In the [Edit Steps Locally](../build/editing-locally.md) section, we walked through
+the process of creating and adding your changes to the `main` branch of a
+project.
-However, most code change to workflows involve sharing and reviewing changes before
-deployment. You can do this by creating, testing and sharing your changes on a
-new Github branch, then, once final, merging them into `main` for deployment.
+However, most code change to workflows involve sharing and reviewing changes
+before deployment. You can do this by creating, testing and sharing your changes
+on a new Github branch, then, once final, merging them into `main` for
+deployment.
:::tip
@@ -28,8 +29,8 @@ repo to your local folder.
branch. When you start editing your steps, the changes will be kept on this
branch, managed separately from `main`.
-2. To test the changes locally, check out the [The CLI](/documentation/cli)
- docs.
+2. To test the changes locally, check out the
+ [The CLI](../build-for-developers/cli-intro.md) docs.
3. Just as you've seen when working on `main`, when you're done check which
files you changed with `git status`.
diff --git a/docs/jobs/javascript.md b/docs/jobs/javascript.md
new file mode 100644
index 00000000000..e06140f08db
--- /dev/null
+++ b/docs/jobs/javascript.md
@@ -0,0 +1,267 @@
+---
+title: Javascript Tips
+sidebar_label: Javascript Tips
+---
+
+OpenFn supports all modern JavaScript features.
+
+This section highlights some useful JavaScript features and operators which
+might help make your code cleaner. This is not meant to be an exhaustive guide,
+just a pointer to some good techniques on some of the newer aspects of the
+language.
+
+### Variables: var vs let vs const
+
+JavaScript gives you three different ways to declare variables, and this can be
+a bit confusing.
+
+- `var` is a variable that can have its value reassigned. You can re-declare a
+ `var` multiple times.
+- `let` is basically the same as a var, but cannot be redeclared and has subtly
+ different scoping rules.
+- `const` is used for variable whose values do not change.
+
+It doesn't really matter which style you use (except perhaps if you try to
+assign to a `const`).
+
+Most OpenFn jobs are written in quite a functional style anyway - you may find
+you don't even need to declare variables.
+
+
+What is functional programming?
+Functional programming is a style of programming, increasingly popular in modern Javascript.
+
+Broadly, the idea is to minimize the usage of control flow statements (like
+`if/else`,`for`) and instead use chains of functions. In functional programming
+we pass data through a pipeline to get the result we want sound familiar?).
+
+```js
+const items = [10, 109, 55];
+
+// Imperative JS
+const transformedItems = [];
+for (const i of items) {
+ if (i < 100) {
+ transformedItems.push(i * 2);
+ }
+}
+
+// Functional js
+const transformedItems = items.filter(x => x > 100).map(x => x * 2);
+```
+
+Functional programming tends to be more terse and condensed than regular,
+imperative programming. This is a good and bad thing - but if you're used to the
+style, it tends to be very readable and translates well across languages.
+
+
+
+Most modern, idiomatic JavaScript is written with `const` and `let`. This can
+actually make your code more readable and intentional, and the rules are
+actually pretty simple:
+
+- Use a `const` if you don't want a variable value to change.
+- Use a `let` if you expect variable value to change.
+
+This can get a little tricky with objects and arrays. We can assign an object to
+a const, but still change the properties of the object. The same for arrays.
+This is all to do with pointers and how JavaScript stores variables - the key to
+it is that you're not assigning a new value to the variable, but you are
+modifying the _contents_ of the variable.
+
+Check these examples:
+
+```js
+// Example 1: Objects
+const data = {};
+
+// We can mutate the object here
+// The data variable is still referencing the same object
+data.name = 'OpenFn';
+
+data = { name: 'Lightning' }; // This throws a runtime error because we are re-assigning the variable!
+
+// Example 2: Arrays
+const ids = [1, 2, 3];
+
+// We can call functions on the ids array, which will mutate the array's contents
+ids.push(4);
+ids.pop();
+
+// But we cannot re-assign the variable
+
+ids = [4, 5, 6]; // This throws a runtime error because we are re-assigning the variable!
+```
+
+### Optional chaining
+
+JavaScript is an untyped language - which is very conveient for OpenFn jobs and
+usually makes life easier.
+
+However, a common problem is that when writing long property chains, an
+exception will be thrown if a property is missing. And this happens all the time
+when fetching data from remote servers.
+
+Optional chaning allows JavaScript to stop evaluating a property chain and
+return undefined as the result of that whole expression:
+
+```js
+const x = a.b?.c?.d?.e;
+```
+
+In this example, if `c`, for example, is not defined, then `x` will be given a
+value of `undefined`. No exception will be thrown.
+
+You can do this with string properties too, although the syntax is a bit
+fiddlier:
+
+```js
+const x = a.b['missing-link']?.d?.e;
+```
+
+This can also be used for optional function calls (less useful in job writing
+but included for completeness):
+
+```js
+const x = a.b?.();
+```
+
+You can combine optional chaning with the wonderfully named **"nullish
+coalescing"** operator. This works a bit like a ternary expression or an or - if
+anything to left of the operator returns `null` or `undefined`, the value to the
+right will be returned.
+
+```js
+const x = a.b?.c?.d?.e ?? 22;
+```
+
+In this example, if any of the values in the chain are not defined, `x` will be
+assigned a value of 22.
+
+### Arrow functions
+
+Arrow functions are used throughout this guide and we expect that most users are
+familiar with their usage.
+
+An arrow function is an alternative way to write a JavaScript function. There
+are a few reasons why they are popular in modern JavaScript:
+
+- They feel lightweight, with less syntax required
+- They do not have a `this` scope - although this is largely irrelevant to
+ OpenFn programming (and indeed most modern JS frameworks)
+
+Arrow functions are always anonymous - they have no name - but they can be
+assigned to variables of course.
+
+```js
+function upperCase(name) {
+ return name.toUpperCase();
+}
+
+const getName = () => {
+ return name.toUpperCase();
+};
+```
+
+An arrow function can contain a single expression and no body, and will return
+the expression:
+
+```js
+function getX() {
+ return x;
+}
+
+const getX = () => x;
+```
+
+This pattern makes arrow functions lightweight and elegant, and aligns nicely
+with functional programming paradigms.
+
+:::tip Problems returning an object?
+
+Always wrap objects in brackets when returning an object from an arrow
+expression:
+
+```
+post('wwww', () => ({ id: 'a', name: 'adam' }))
+```
+
+When Javascript sees a brace `{` after an arrow, it expects to see a block of
+statements, not an object. Wrapping the object in brackets tells Javascript to
+parse an expression instead of a block.
+
+:::
+
+### Rest and spread operators
+
+The spread or rest operator `...` can be used for several purposes. It can be
+quite complex to understand, but in OpenFn it has a couple of strong uses.
+
+First, you can **"spread"** or **"apply"** the properties and value of one (or
+more) objects to a new object. This is a really conveient way to shallow clone
+objects.
+
+It works a lot like `Object.assign(obj, first, second, third)`.
+
+Here's how we shallow clone with spread:
+
+```js
+const newState = {
+ ...state,
+};
+```
+
+Properties are declared in sequence, so you can spread an object and then
+declare more properties:
+
+```js
+const newState = {
+ ...state
+ data: {} // create a new data object but keep all other keys of state
+}
+```
+
+You can spread multiple objects, which again will apply in order. This example
+applies some default values, then overwrites with whatever is on state, then
+finally overwrites the data key.
+
+```js
+const newState = {
+ ...defaults,
+ ...state
+ data: {} // create a new data object but keep all other keys of state
+}
+```
+
+Spreading like this does not affect the original object (ie, in the example
+above, `defaults` and `state` are not changed), although remember that this is
+only a shallow clone, and non-primitive values use pointers, not copies.
+
+
+What is a shallow clone?
+To shallow clone an object means to copy all the top-level keys and values of that object onto a new object.
+
+But this ONLY applies to top-level keys. And if a value contains an object,
+you're really just copying a _pointer_ to that object.
+
+```js
+const a = {
+ x: 1,
+ y: {
+ values: [1, 2, 3]
+ }
+};
+
+// declare b as a shallow clone of a
+const b = {
+ ... a
+}
+
+b.x = 2; // a.x is unchanged
+b.y.values = []; // a.y.values is changed
+b.y = 20' // a.y is unchanged
+```
+
+A deep clone means that all properties in the whole object tree are cloned.
+
+
diff --git a/docs/build/steps/job-examples.md b/docs/jobs/job-examples.md
similarity index 71%
rename from docs/build/steps/job-examples.md
rename to docs/jobs/job-examples.md
index 8a9e9fa3635..bf292919d8b 100644
--- a/docs/build/steps/job-examples.md
+++ b/docs/jobs/job-examples.md
@@ -1,16 +1,24 @@
---
title: Job Code Examples
-sidebar_label: Job Code Snippets & Examples
+sidebar_label: Job Examples
---
-## Snippets and samples
+Below you can find some code blocks for different functions and data handling
+contexts to use in your Jobs.
-Below you can find some code block for different functions and data
-handling contexts to use in your Jobs. **Also see the [Library Examples](/adaptors/library) for more Job examples for other Adaptors.**
+:::tip
+
+For example Jobs written by the OpenFn core team and other users, check out the
+[Library](/adaptors/library) or other project repositories under
+[Github.com/OpenFn](https://github.com/OpenFn).
+
+:::
:::info Questions?
-If you have any job-writing questions, ask on [Community](https://community.openfn.org) to seek assistance from the OpenFn core team and other implementers.
+If you have any job-writing questions, ask on
+[Community](https://community.openfn.org) to seek assistance from the OpenFn
+core team and other implementers.
:::
@@ -412,123 +420,3 @@ post(
callback
);
```
-
-## Anonymous Functions
-
-Different to [Named Functions](#examples-of-adaptor-specific-functions),
-Anonymous functions are generic pieces of javascript which you can write to suit
-your needs. Here are some examples of these custom functions:
-
-### Custom replacer
-
-```js
-field('destination__c', state => {
- console.log(something);
- return dataValue('path_to_data')(state).toString().replace('cats', 'dogs');
-});
-```
-
-This will replace all "cats" with "dogs" in the string that lives at
-`path_to_data`.
-
-> **NOTE:** The JavaScript `replace()` function only replaces the first instance
-> of whatever argument you specify. If you're looking for a way to replace all
-> instances, we suggest you use a regex like we did in the
-> [example](#custom-concatenation-of-null-values) below.
-
-### Custom arrayToString
-
-```js
-field("target_specie_list__c", function(state) {
- return Array.apply(
- null, sourceValue("$.data.target_specie_list")(state)
- ).join(', ')
-}),
-```
-
-It will take an array, and concatenate each item into a string with a ", "
-separator.
-
-### Custom concatenation
-
-```js
-field('ODK_Key__c', function (state) {
- return dataValue('metaId')(state).concat('(', dataValue('index')(state), ')');
-});
-```
-
-This will concatenate two values.
-
-### Concatenation of null values
-
-This will concatenate many values, even if one or more are null, writing them to
-a field called Main_Office_City_c.
-
-```js
-...
- field("Main_Office_City__c", function(state) {
- return arrayToString([
- dataValue("Main_Office_City_a")(state) === null ? "" : dataValue("Main_Office_City_a")(state).toString().replace(/-/g, " "),
- dataValue("Main_Office_City_b")(state) === null ? "" : dataValue("Main_Office_City_b")(state).toString().replace(/-/g, " "),
- dataValue("Main_Office_City_c")(state) === null ? "" : dataValue("Main_Office_City_c")(state).toString().replace(/-/g, " "),
- dataValue("Main_Office_City_d")(state) === null ? "" : dataValue("Main_Office_City_d")(state).toString().replace(/-/g, " "),
- ].filter(Boolean), ',')
- })
-```
-
-> Notice how this custom function makes use of the **regex** `/-/g` to ensure
-> that all instances are accounted for (g = global search).
-
-### Custom Nth reference ID
-
-If you ever want to retrieve the FIRST object you created, or the SECOND, or the
-Nth, for that matter, a function like this will do the trick.
-
-```js
-field('parent__c', function (state) {
- return state.references[state.references.length - 1].id;
-});
-```
-
-See how instead of taking the id of the "last" thing that was created in
-Salesforce, you're taking the id of the 1st thing, or 2nd thing if you replace
-"length-1" with "length-2".
-
-### Convert date string to standard ISO date for Salesforce
-
-```js
-field('Payment_Date__c', function (state) {
- return new Date(dataValue('payment_date')(state)).toISOString();
-});
-```
-
-> **NOTE**: The output of this function will always be formatted according to
-> GMT time-zone.
-
-### Use external ID fields for relationships during a bulk load in Salesforce
-
-```js
-array.map(item => {
- return {
- Patient_Name__c: item.fullName,
- 'Account.Account_External_ID__c': item.account
- 'Clinic__r.Unique_Clinic_Identifier__c': item.clinicId,
- 'RecordType.Name': item.type,
- };
-});
-```
-
-### Bulk upsert with an external ID in salesforce
-
-```js
-bulk(
- 'Visit_new__c',
- 'upsert',
- {
- extIdField: 'commcare_case_id__c',
- failOnError: true,
- allowNoOp: true,
- },
- dataValue('patients')
-);
-```
\ No newline at end of file
diff --git a/docs/jobs/job-snippets.md b/docs/jobs/job-snippets.md
new file mode 100644
index 00000000000..fbb54522c79
--- /dev/null
+++ b/docs/jobs/job-snippets.md
@@ -0,0 +1,124 @@
+---
+title: Code Snippets
+sidebar_label: Code Snippets
+---
+
+This section includes a numner of useful JavaScript code snippets which you can
+use in your jobs.
+
+Most snippets are implemented as callbacks to other operations.
+
+You can copy these callbacks and adapt them to suit your own code.
+
+### Custom replacer
+
+```js
+field('destination__c', state => {
+ return dataValue('path_to_data')(state).toString().replace('cats', 'dogs');
+});
+```
+
+This will replace all "cats" with "dogs" in the string that lives at
+`path_to_data`.
+
+> **NOTE:** The JavaScript `replace()` function only replaces the first instance
+> of whatever argument you specify. If you're looking for a way to replace all
+> instances, we suggest you use a regex like we did in the
+> [example](#custom-concatenation-of-null-values) below.
+
+### Custom arrayToString
+
+```js
+field("target_specie_list__c", function(state) {
+ return Array.apply(
+ null, sourceValue("$.data.target_specie_list")(state)
+ ).join(', ')
+}),
+```
+
+It will take an array, and concatenate each item into a string with a ", "
+separator.
+
+### Custom concatenation
+
+```js
+field('ODK_Key__c', function (state) {
+ return dataValue('metaId')(state).concat('(', dataValue('index')(state), ')');
+});
+```
+
+This will concatenate two values.
+
+### Concatenation of null values
+
+This will concatenate many values, even if one or more are null, writing them to
+a field called Main_Office_City_c.
+
+```js
+...
+ field("Main_Office_City__c", function(state) {
+ return arrayToString([
+ dataValue("Main_Office_City_a")(state) === null ? "" : dataValue("Main_Office_City_a")(state).toString().replace(/-/g, " "),
+ dataValue("Main_Office_City_b")(state) === null ? "" : dataValue("Main_Office_City_b")(state).toString().replace(/-/g, " "),
+ dataValue("Main_Office_City_c")(state) === null ? "" : dataValue("Main_Office_City_c")(state).toString().replace(/-/g, " "),
+ dataValue("Main_Office_City_d")(state) === null ? "" : dataValue("Main_Office_City_d")(state).toString().replace(/-/g, " "),
+ ].filter(Boolean), ',')
+ })
+```
+
+> Notice how this custom function makes use of the **regex** `/-/g` to ensure
+> that all instances are accounted for (g = global search).
+
+### Custom Nth reference ID
+
+If you ever want to retrieve the FIRST object you created, or the SECOND, or the
+Nth, for that matter, a function like this will do the trick.
+
+```js
+field('parent__c', function (state) {
+ return state.references[state.references.length - 1].id;
+});
+```
+
+See how instead of taking the id of the "last" thing that was created in
+Salesforce, you're taking the id of the 1st thing, or 2nd thing if you replace
+"length-1" with "length-2".
+
+### Convert date string to standard ISO date for Salesforce
+
+```js
+field('Payment_Date__c', function (state) {
+ return new Date(dataValue('payment_date')(state)).toISOString();
+});
+```
+
+> **NOTE**: The output of this function will always be formatted according to
+> GMT time-zone.
+
+### Use external ID fields for relationships during a bulk load in Salesforce
+
+```js
+array.map(item => {
+ return {
+ Patient_Name__c: item.fullName,
+ 'Account.Account_External_ID__c': item.account
+ 'Clinic__r.Unique_Clinic_Identifier__c': item.clinicId,
+ 'RecordType.Name': item.type,
+ };
+});
+```
+
+### Bulk upsert with an external ID in salesforce
+
+```js
+bulk(
+ 'Visit_new__c',
+ 'upsert',
+ {
+ extIdField: 'commcare_case_id__c',
+ failOnError: true,
+ allowNoOp: true,
+ },
+ dataValue('patients')
+);
+```
diff --git a/docs/jobs/job-writing-guide.md b/docs/jobs/job-writing-guide.md
new file mode 100644
index 00000000000..1f5a10f5a98
--- /dev/null
+++ b/docs/jobs/job-writing-guide.md
@@ -0,0 +1,711 @@
+---
+sidebar_label: Job Writing Guide
+title: Job Writing Guide
+---
+
+
+
+Workflow automation and data integration in OpenFn is realised through the
+creation of Jobs.
+
+This guide will walk you through key concepts and best practices for job
+writing. It is suitable for new coders and experienced JavaScript programmers.
+In fact, even if you're an experienced JavaScript Developer, there a number of
+key patterns in the OpenFn ecosystem which it is important to learn.
+
+A Job is a bunch of JavaScript code which performs a particular task, like
+fetching data from Salesforce or converting some JSON data to FHIR standard.
+
+Each job uses exactly one Adaptor (often called a "connector") to perform its
+task. The Adaptor provides a collection of helper functions (Operations) which
+makes it easy to communicate with a data source.
+
+This guide applies equally to writing Jobs on the app (Lightning) or through the
+CLI.
+
+:::info Workflows
+
+Multiple jobs can be chained together in a Workflow. A common pattern is to use
+one job to fetch data from datasource A, one job to convert or transform that
+data to be compatible with datasource B, and a third job to upload the
+transformed data to datasource B.
+
+To learn more about workflow design and implementation, see
+[Build & Manage Workflows](build/workflows.md)
+
+:::
+
+## Operations and State
+
+Every job is a data transformation pipeline.
+
+It takes some input (a JavaScript object we call State) and executes a set of
+Operations (or functions), which transform that state in series (ie, one after
+the other). The final state object is returned as the output of the pipeline.
+
+
+
+Operations are provided by an Adaptor (connector). Each adaptor exports a list
+of functions designed to interact with a particular data source - for example,
+take a look at the [dhis2](adaptors/packages/dhis2-docs) and
+[salesforce](adaptors/packages/salesforce-docs) adaptors.
+
+Everything you can achieve in OpenFn can be achieve with existing JavaScript
+libraries or calls to REST APIs. The value of Adaptors is that they provide
+functions to make this stuff easier: taking care of authoristaion, providing
+cleaner syntax, and hiding away implementation details for you.
+
+For example, here's how we issue a GET request with the http adaptor:
+
+```js
+get('/patients');
+```
+
+The first argument to `get` is the path to request from (the configuration will
+tell the adaptor what base url to use). In this case we're passing a static
+string, but we can also pass a value from state:
+
+```js
+get(state => state.endpoint);
+```
+
+
+Why the arrow function?
+If you've got some JavaScript experience, you'll notice The example above uses an arrow function to retreive the endpoint key from state.
+
+But why not just do this?
+
+```
+get(state.endpoint);
+```
+
+Well, the problem is that the state value must be resolved lazily (ie, just
+before the get actually runs). Because of how Javascript works, if we just
+inline the value it might read before state.endpoint has been actually been
+assigned.
+
+For more details, jump ahead to [Reading State Lazily](#reading-state-lazily)
+
+
+
+Your job code should only contain Operations at the top level/scope - you should
+NOT include any other JavaScript statements. We'll talk about this more in a
+minute.
+
+## Callbacks and fn()
+
+Many Operations give you access to a callback function.
+
+Callbacks will be invoked with state, will run whatever code you like, and must
+return the next state. Usually your callback will be invoked as the very last
+step in an operation.
+
+This is useful for intercepting and manipulating the return value of a given
+operation.
+
+
+What is a callback?
+A callback is a common pattern in JavaScript.
+
+It's kind of hard to understand in the abstract: a callback is a function which
+you pass into some other function, to be invoked by that function at a
+particular time.
+
+It's best explained with an example. All JavaScript arrays have a function
+called `map`, which takes a single callback argument.
+
+Array.map will iterate over every item in the array, invoke your callback
+function with it, save the result to a new array, and when it's finished, it
+will return that array.
+
+```js
+const array = ['a', 'b', 'c'];
+const result = array.map(item => {
+ return item.toUpperCase();
+});
+console.log(array); // ['a', 'b', 'c'];
+console.log(result); // ['A', 'B', 'C'];
+```
+
+Because functions are data in JavaScript, we can we-write that code like this
+(which might be a bit more readable)
+
+```js
+const array = ['a', 'b', 'c'];
+const upperCase = item => {
+ return item.toUpperCase();
+};
+const result = array.map(upperCase);
+console.log(array); // ['a', 'b', 'c'];
+console.log(result); // ['A', 'B', 'C'];
+```
+
+
+
+The `fn()` function, for example, ONLY allows you define a callback. This is
+useful for running arbitrary code - if you want to drop down to raw JavaScript
+mode, this is how you do it:
+
+```js
+fn(state => {
+ // declare a help function
+ const convertToFhir = item => {
+ /* ... */
+ };
+
+ // Map data into a new format with native Javsacript functions
+ state.transformed = state.data.map(convertToFhir);
+
+ // Always return the state
+ return state;
+});
+```
+
+Many other operations provide a callback argument, in which case, your callback
+will be invoked with state, and most return the final state as a result of the
+operation.
+
+For example, say you fetch some data from a data source and get back a block of
+JSON. Maybe you want to filter this data before passing it to the next
+operation.
+
+You might naively try something like this - but it won't work!
+
+```js
+get('/data'); // writes to state.data
+state.data = state.data.filter(/* ... */); // This is invalid!
+```
+
+You could use another operation, like `fn` or `each` - and often these work
+great!
+
+```js
+get('/data');
+fn(state => {
+ state.data = state.data.filter(/* ... */);
+ return state;
+});
+```
+
+But you can also use a callback function, which is usually a bit neater:
+
+```js
+get('/data', {}, state => {
+ state.data = state.data.filter(/* ... */);
+ return state;
+});
+```
+
+Whatever your callback returns will be used as the input state for the next
+operation (or will be the final state for the job). So remember to ALWAYS return
+state!
+
+Be mindful that some Adaptors will write internal information to state. So you
+should usually `return { ... state }` rather than `return { data: state.data }`.
+
+:::tip
+
+Remember! Always return state from a callback.
+
+:::
+
+## Operations run at the top level
+
+Operations will only work when they are at the top level of your job code, like
+this:
+
+```js
+get('/patients');
+each('$.data.patients[*]', (item, index) => {
+ item.id = `item-${index}`;
+});
+post('/patients', dataValue('patients'));
+```
+
+OpenFn will call your operations in series during workflow execution, ensuring
+the correct state is fed into each one.
+
+If you try to nest an operation inside the callback of another operation, you'll
+quickly run into trouble:
+
+```js
+get('/patients', { headers: { 'content-type': 'application/json' } }, state => {
+ // This will fail because it is nested in a callback
+ each('$.data.patients[*]', (item, index) => {
+ item.id = `item-${index}`;
+ });
+});
+post('/patients', dataValue('patients'));
+```
+
+This is because an operation is actually a "factory" style function - when
+executed, it returns a new function. That new function must be invoked with
+state, and will return state.
+
+
+What is a factory function?
+Factory functions are quite a hard pattern to understand, although you get used it.
+
+Luckily, you don't really need to understand the pattern to understand openfn.
+
+Simply put, a factory function doesn't really do anything. It instead returns a
+function to do something.
+
+Factory functions are useful for deferred execution, lazy loading, constructor
+functions, and scope binding.
+
+They're used by us in open function for deferred execution: our runtime converts
+each factory into an actual operation function, saves all the functions into an
+array, then iterates over each one and passes in state.
+
+
+
+The OpenFn runtime knows how to handle an operation at the top scope, it can run
+it as part of the pipeline and handle state appropriately. But it does not know
+how to deal with a nested operation like this.
+
+You should actually never need to nest an operation anyway. Just bring it to the
+top level and lean in to the pipeline idea. But if you ever find yourself in a
+situation where you absolutely need to use a nested operation, you should pass
+state into it directly, like this:
+
+```js
+get('/patients', { headers: { 'content-type': 'application/json' } }, state => {
+ each('$.data.patients[*]', (item, index) => {
+ item.id = `item-${index}`;
+ })(state); // Immediately invoke the Operation and pass state into it. This is naughty!
+});
+post('/patients', dataValue('patients'));
+```
+
+To be clear, this is considered an anti-pattern and should not be used except in
+emergencies.
+
+## Reading state lazily
+
+A common problem in OpenFn coding is getting hold of the right state value at
+the right time.
+
+Consider this code:
+
+```js
+get('/some-data');
+post('/some-other-data', state.data);
+```
+
+What it's trying to do is call the GET method on some REST service, save the
+result to `state.data`, and then pass that `state.data` value into a post call
+to send it somewhere else.
+
+Can you see the problem?
+
+It happens all the time.
+
+Because of the way JavaScript works, `state.data` will be evaluated before the
+`get()` request has finished. So the post will always receive a value of
+`undefined`.
+
+
+
+Okay, how does JavaScript work?
+JavaScript, like most languages, will evaluate synchronously, executing code line of code one at a time.
+
+Because each Operation is actually a factory function, it will execute instantly
+and return a function - but that function won't actually be executed yet.
+
+The returned function will have the original arguments in scope (trapped as
+closures), and be executed later against the latest state object.
+
+The example above will synchronously create two functions, which are added into
+an array by the compiler (see bellow), and then excuted by the runtime.
+
+So when your code executes, it's doing something like this:
+
+```js
+const getFn = get('/some-data');
+const postFn = post('/some-data', state.data);
+
+return getFn(state).then(nextState => postFn(nextState));
+```
+
+The point is that when the post operation is created, all we've done is _create_
+the `get` function - we haven't actually _run_ it. And so state.data is
+basically uninitialised.
+
+
+
+What we actually need to do is defer the evaluation of `state.data` until the
+`post` operation actually runs. In other words, we work out the value of
+`state.data` at last possible moment.
+
+There are a few ways we can do that. Some jobs use `dataValue`, which is neat if
+a bit verbose (there are many examples in this guide), and some operations
+support JSON path strings. The preferred way in modern OpenFn is to use an
+inline-function:
+
+```js
+get('/some-data');
+post('/some-other-data', state => state.data);
+```
+
+This passes a function into the `post` operator. When the `post()` call actually
+executes, the first thing it'll do is resolve any function into arguments into
+values. It does this by calling the function passing in the latest state, and
+using the return as the value.
+
+These lazy functions are incredibly powerful. Using them effectively is the key
+to writing good OpenFn jobs.
+
+## Mapping Objects
+
+A common use-case in OpenFn fn is to map/convert/transform an object from system
+A to the format of system B.
+
+We often do this in multiple Jobs in the same workflow, so that we can use
+different adaptors. But in this example we'll work with three operations in one
+job with the http adaptor: one to fetch data, one to transform, and one to
+upload:
+
+```js
+// Fetch an object from one system
+get('https://www.system-a.com/api/patients/123');
+
+// Transform it
+fn(state => {
+ // Read the data we fetched
+ const obj = state.data;
+
+ // convert it by mapping properties from one object to the o ther
+ state.uploadData = {
+ id: obj.id,
+ name: `${obj.first_name} ${obj.last_name}`,
+ metadata: obj.user_data,
+ };
+
+ // Don't forget to return state!
+ return state;
+});
+
+// Post it elsewhere
+post('https://system-b.com/api/v1/records/123', () => state.uploadData);
+```
+
+:::tip Batch conversions
+
+These examples show a single object being converted - but sometimes we need to
+convert many objects at once.
+
+See the each() example below to see how we can do this with the each operator.
+
+You can also use a Javascript map() or forEach() function inside a callback or
+fn block.
+
+Generally it's easier to spread your job logic across many top-level operations,
+each responsible for one task, rather than having a few deeply nested
+operations.
+
+:::
+
+This is fine - and actually, having lots of operations which each do a small
+task is usually considered a good thing. It makes code more readable and easier
+to reason about when things go wrong.
+
+But every operation argument accepts a function (allowing lazy state references,
+as described above). This gives us the opportunity to the conversion in-line in
+the post operation:
+
+```js
+// Fetch an object from one system
+get('https://www.system-a.com/api/patients/123');
+
+// Transform and post it elsewhere
+post('https://system-b.com/api/v1/records/123', state => ({
+ id: state.data.id,
+ name: `${state.data.first_name} ${state.data.last_name}`,
+ metadata: state.data.user_data,
+}));
+```
+
+Effective use of these lazily-resolved functions is critical to writing good
+OpenFn jobs.
+
+## Iteration with each()
+
+A typical use-case in data integration in particular is to convert data from one
+format to another. Usually this involves iterating over an array of items,
+converting the values, and mapping them into a new array.
+
+In OpenFn, we can use the `each()` operator to do this.
+
+```js
+each(
+ '$.data.items[*]',
+ get(state => `/patients/${state.data.id}`)
+);
+```
+
+`each()` takes a JSON path string as its first argument, which points to some
+part of state. In JSON path, we use `$` to refer to the root, and dot notation
+to chain a path, and `[*]` to "select" an array of items.
+
+The second argument is an Operation, which will receive each item at the end of
+the json path as `state.data`, but otherwise will receive the rest of the state
+object.
+
+So we can iterate over each item and write it back to state, like this:
+
+```js
+fn((state) => {
+ // Initialize an array into state to use later
+ state.transformed = []
+ return state;
+})
+each("$.items[*]", fn(state) => {
+ // Pull the next item off the state
+ const next = state.data;
+
+ // Transform it
+ const transformed = { ...next };
+
+ // Write it back to the top-level transformed array on state
+ state.transformed.push(transformed)
+
+ // Always return state
+ return state;
+})
+```
+
+Or we can pass in another operation, like this Salesforce example:
+
+```js
+each(
+ '$.form.participants[*]',
+ upsert('Person__c', 'Participant_PID__c', state => ({
+ Participant_PID__c: state.pid,
+ First_Name__c: state.participant_first_name,
+ Surname__c: state.participant_surname,
+ }))
+);
+```
+
+Each participant is upserted into Salesforce, with its salesforce fields mapped
+to values in the `participants` array.
+
+:::info JSON paths
+
+The use of a JSON path string as the first argument to `each()` allows the
+runtime to lazily evaluate the value at that path -
+[See Reading state lazily](#reading-state-lazily).
+
+Not all operations support a JSON path string - refer to
+[individual adaptor docs](/adaptors) for guidance.
+
+:::
+
+## Variable initialisation
+
+
+
+A common pattern is to need to declare some variables at the state of the job.
+These could be static values to use later, functions to be called multiple times
+through the job, or bits of state that we want to return at the end.
+
+It is considered best practice to use an `fn()` block to do this at the start of
+the job, writing all values to state.
+
+```js
+fn(state => {
+ // Create an array to hold the final results of the job
+ state.results = [];
+
+ // Create a lookup index to be used during the job
+ state.lookup = {};
+
+ state.keyMap = {
+ AccountName: 'C__Acc_Name', // capture various static mappings for transformation
+ };
+
+ state.maxPageSize = 200; // Define some config options
+
+ state.convertToSF = item => {
+ /* ... */
+ }; // A function to be re-used
+
+ return state;
+});
+
+// the rest of your job code goes here
+get('/abc');
+
+fn(state => {
+ /* ... */
+
+ // Only return the results array as output from the job
+ return { result: state.results };
+});
+```
+
+## Cleaning final state
+
+When your job has completed, the final state object will be "returned" by the
+openfn runtime.
+
+This final state must be serialisable to JSON.
+
+If there are more steps in the workflow, the state will be passed to those
+steps. If running on the app (Lightning), the final state will be saved as a
+dataclip. Or if running in the CLI, the final state will be written to disk.
+
+It's often desirable to clean up your final state so that any unused information
+is removed. This reduces the size of your saved data, but could also be an
+important security consideration.
+
+The best way to do this is with a closing `fn()` block which returns just the
+keys you want (this is usually best):
+
+```js
+fn(state => {
+ return {
+ data: state.data,
+ };
+});
+```
+
+You could use the spread operator to override some keys:
+
+```js
+fn(state => {
+ return {
+ ...state,
+ secretStuff: null,
+ };
+});
+```
+
+Or use the rest operator:
+
+```js
+fn(state => {
+ const { usename, password, secrets, ...rest } = state;
+ return rest;
+});
+```
+
+:::info Configuration & Functions
+
+OpenFn will automatically scrub the `configuration` key and any functions from
+your final state.
+
+:::
+
+
+
+## Error Handling
+
+If something goes wrong, it's usually best to let your jobs fail.
+
+Failing jobs will generate the right status on the OpenFn app and communicate
+that something is wrong.
+
+Don't worry, errors happen all the time! Even well established workflows will
+occasionally throw an error because of some unexpected data somewhere in the
+pipeline. It's a fact of life, but the most important thing is to be aware of
+it.
+
+Errors should be thrown from the job without much ceremony, and will be caught
+by the runtime to be processed appropriately.
+
+If a Job does throw an error, it will be logged and written to the final state,
+so it should be easy to find and identify the cause.
+
+It is common practice in a Workflow to let a Job error, and then perform some
+task - like emailing a system admin that there was a problem.
+
+When processing batches of data, you might want to catch errors occuring on
+individual items and write them to state. That way one bad item won't ruin a
+whole batch, and you know which items succeeded and which failed. You can then
+throw an exception to recognise that the job has failed.
+
+## Compilation
+
+The code you write isn't technically executable JavaScript. You can't just run
+it through node.js. It needs to be transformed or compliled into portable
+vanilla JS code.
+
+:::warning
+
+This is advanced stuff more focused at JavaScript developers and the technically
+curious. These docs are not intended to be complete - just a nudge in the right
+direction to help understand how jobs work.
+
+:::
+
+The major differences between openfn code and JavaScript are:
+
+- The top level functions in the code are executed synchronously (in sequence),
+ even if they contain asynchronous code
+- OpenFn code does not contain import statements (although technically it can).
+ These are compiled in.
+- Compiled code is a JavaScript ESM module which default-exports an array of
+ async functions. These functions are imported and executed by the runtime.
+
+It shouldn't be necessary to understand compilation in detail, but you should be
+aware that the code you write is not the code you run.
+
+If you're a JavaScript developer, understanding some of these changes might help
+you better understand how OpenFn works. Using the CLI, you can run
+`openfn compile path/to/job.ja -a ` to see compiled code.
+
+Here's an example of how a simple job looks in compilation:
+
+This job:
+
+```js
+get('/patients');
+```
+
+Compiles to this JavaScript module:
+
+```js
+import { get } from '@openfn/language-http';
+export * from '@openfn/language-http';
+export default [get('/patients')];
+```
+
+## Next Steps
+
+The best way to learn how to write OpenFn jobs is to write OpenFn jobs.
+
+You can [get started with CLI](/documentation/cli) and start running jobs
+locally. Then take a look at the [CLI Challenge](/documentation/cli-challenges)
+to really exercise your job writing skills.
+
+If you're ready to start using the app, take a look at this guide to
+[create your first Workflow](documentation/build/workflows).
+
+Workflow design is a non-trivial problem, so you might also like to review the
+Workflow [Design Process docs](documentation/design/design-overview).
+
+:::info Questions?
+
+If you have any job-writing questions, ask on
+[Community](https://community.openfn.org) to seek assistance from the OpenFn
+core team and other implementers.
+
+:::
diff --git a/docs/jobs/state.md b/docs/jobs/state.md
new file mode 100644
index 00000000000..7ec4794d4f1
--- /dev/null
+++ b/docs/jobs/state.md
@@ -0,0 +1,123 @@
+---
+title: Input and output state
+---
+
+Each Job requires an input state and (in most cases) will produce an output
+state. This article explains these concepts in greater detail.
+
+State is just a Javascript object. It is the means via which Jobs share
+information between each other. It also provides a common scope for Operations
+to read from and write to.
+
+The final state form a Job must always be a serializable Javascript object (ie,
+a JSON object). Any non-serializable keys will be removed.
+
+
+
+:::tip A note on terminology
+
+Input state is often referred to as _initial state_, and output state is often
+referred as _final state_. These terms can safely be used interchangeably.
+
+:::
+
+## State Keys
+
+State objects tend to have the following keys:
+
+- `data`: a temporary information store, usually used to save the result of
+ particular operation
+- `configuration`: an object containing credential data
+- `references`: a history of previous `data` values
+- `response`: often used by adaptors (like http) to save the raw http response
+ from a request
+- `errors`: a list of errors generated by a particular Workflow, indexed by Job
+ name.
+
+At the end of a Job, the configuration key will be removed, along with any other
+non serialisable keys.
+
+Adaptors will occasionally write extra information to state during a run - for
+example, database Adaptors tend to write a `client` key to state, used to track
+the database connection. These will eb removed at the end of a Job.
+
+## Input & output state for runs
+
+Depending on whether you're running Workflows locally or on the app, the input
+state for a Run can be generated differently:
+
+- When creating a work order by hand, you must select or generate your input
+ manually (e.g., by creating a custom `Input` on the app or `state.json` file
+ if working locally [in the CLI](../build-for-developers/cli-intro.md)).
+- When a work order is automatically created via a webhook trigger or cron
+ trigger, state will be created as described below.
+
+The final state of a Run is determined by what's returned from the last
+operation. Remember that job expressions are a series of operations: they each
+take state and return state, after creating any number of side effects. You can
+control what is returned at the end of all of these operations.
+
+### Webhook triggered Runs
+
+In the app, when a Run is triggered by a webhook, the input state contains
+important parts of the inbound **http request**.
+
+The input state will look something like this:
+
+```js
+{
+ data: {}, // the body of the http request
+ request: {
+ headers: {} // an object containing the headers of the request
+ },
+}
+```
+
+### Cron triggered Runs
+
+In the app, when a Run is triggered by a cron job, the input state will the
+final state of the **last succesful run** for this workflow.
+
+If this is the first time the workflow has run, the initial state will simply by
+an empty object.
+
+```js
+{
+ ...(finalStateOfLastSuccessfulRun || {}),
+}
+```
+
+## Input & output state for steps
+
+State is also passed between each step in a workflow. The output state of the
+previous step is used as the input state for the next step.
+
+### On success
+
+When a job succeeds, its output state will be whatever is returned by the last
+operation.
+
+```js
+{
+ data: { patients: [ ]},
+ references: [1, 2, 3]
+}
+```
+
+### On failure
+
+When a step in a workflow fails, the error will be added to an `errors` object
+on state, keyed by the ID of the job that failed.
+
+```js
+{
+ data: { patients: [ ]},
+ references: [1, 2, 3],
+ errors: { jobId: { /* error details */} }
+}
+```
+
+See the below diagram for a visual description of how state might be passed
+between Steps in a Workflow.
+
+
diff --git a/docs/tutorials/http-to-googlesheets.md b/docs/tutorials/http-to-googlesheets.md
index 206ba166f5e..52fc900a8d4 100644
--- a/docs/tutorials/http-to-googlesheets.md
+++ b/docs/tutorials/http-to-googlesheets.md
@@ -73,7 +73,7 @@ following options
Check out the docs on the ["http" Adaptor](/adaptors/packages/http-readme),
[configuring Steps](../build/steps/steps.md), and
-[job-writing](../build/steps/jobs.md).
+[job-writing](../jobs/job-writing-guide.md).
:::
diff --git a/docs/tutorials/kobo-to-dhis2.md b/docs/tutorials/kobo-to-dhis2.md
index 551a71fdf30..c48f593736e 100644
--- a/docs/tutorials/kobo-to-dhis2.md
+++ b/docs/tutorials/kobo-to-dhis2.md
@@ -75,7 +75,7 @@ getSubmissions({ formId: 'aBpweTNdaGJQFb5EBBwUeo' });
Check out the docs on the ["kobotoolbox" Adaptor](/adaptors/kobotoolbox),
[configuring Steps](../build/steps/steps.md), and
-[job-writing](../build/steps/jobs.md).
+[job-writing](../jobs/job-writing-guide.md).
:::
@@ -121,7 +121,7 @@ fn(state => {
Check out the docs on the ["common" Adaptor](/adaptors/packages/common-docs),
[configuring Steps](../build/steps/steps.md), and
-[job-writing](../build/steps/jobs.md).
+[job-writing](../jobs/job-writing-guide.md).
:::
@@ -182,7 +182,7 @@ create('dataValueSets', state => ({
Check out the docs on the ["dhis2" Adaptor](/adaptors/dhis2),
[configuring Steps](../build/steps/steps.md), and
-[job-writing](../build/steps/jobs.md).
+[job-writing](../jobs/job-writing-guide.md).
:::
diff --git a/sidebars-main.js b/sidebars-main.js
index a8dd23ec628..9f7282b5ee6 100644
--- a/sidebars-main.js
+++ b/sidebars-main.js
@@ -36,7 +36,17 @@ module.exports = {
'design/workflow-specs',
],
},
-
+ {
+ type: 'category',
+ label: 'Write Jobs',
+ items: [
+ 'jobs/job-writing-guide',
+ 'jobs/state',
+ 'jobs/javascript',
+ 'jobs/job-examples',
+ 'jobs/job-snippets',
+ ],
+ },
{
type: 'category',
label: 'Platform App ⚡',
@@ -53,21 +63,9 @@ module.exports = {
'build/paths',
'build/credentials',
'build/limits',
+ 'build/editing-locally',
+ 'build/working-with-branches',
'build/troubleshooting',
- {
- type: 'category',
- label: 'Jobs',
- items: [
- 'build/steps/jobs',
- 'build/steps/job-examples',
- 'build/steps/operations',
- 'build/steps/multiple-operations',
- 'build/steps/state',
- 'build/steps/each',
- 'build/steps/editing-locally',
- 'build/steps/working-with-branches',
- ],
- },
],
},
{
@@ -116,9 +114,6 @@ module.exports = {
'build-for-developers/cli-usage',
'build-for-developers/cli-walkthrough',
'build-for-developers/cli-challenges',
- // 'build-for-developers/jobs',
- // 'build-for-developers/build-with-api',
- // 'build-for-developers/security-for-devs',
],
},
{
diff --git a/static/img/guide-job-pipeline.webp b/static/img/guide-job-pipeline.webp
new file mode 100644
index 00000000000..f2baac36579
Binary files /dev/null and b/static/img/guide-job-pipeline.webp differ