Revert "Remove reference page from low-code docs" (#32093)

docs/connector-development/config-based/advanced-topics.md

@@ -236,7 +236,7 @@ In addition to passing additional values through the $parameters argument, macro
 For example,
 `"{{ max(2, 3) }}" -> 3`
 
-The macros and variables available in all possible contexts are documented in the [YAML Reference](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L2154).
+The macros and variables available in all possible contexts are documented in the [YAML Reference](./understanding-the-yaml-file/reference.md#variables).
 
 Additional information on jinja templating can be found at [https://jinja.palletsprojects.com/en/3.1.x/templates/#](https://jinja.palletsprojects.com/en/3.1.x/templates/#)
 

docs/connector-development/config-based/low-code-cdk-overview.md

@@ -1,7 +1,6 @@
 # Low-code connector development
 
 Airbyte’s low-code framework enables you to build source connectors for REST APIs via a [connector builder UI](https://docs.airbyte.com/connector-development/connector-builder-ui/overview) or by modifying boilerplate YAML files via terminal or text editor.
-This page provides an overview of the low-code framework and how to use it to build connectors. You can find the full definition of the schema in the [CDK repository](../../../airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml)
 
 :::info
 Developer updates will be announced via our #help-connector-development Slack channel. If you are using the CDK, please join to stay up to date on changes and issues.

docs/connector-development/config-based/understanding-the-yaml-file/reference.md

@@ -0,0 +1,50 @@
+import ManifestYamlDefinitions from '@site/src/components/ManifestYamlDefinitions';
+
+import schema from "../../../../airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml";
+
+
+# YAML Reference
+
+This page lists all components, interpolation variables and interpolation macros that can be used when defining a low code YAML file.
+
+For the technical JSON schema definition that low code manifests are validated against, see [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml).
+
+<ManifestYamlDefinitions />
+
+export const toc = [
+    {
+    "value": "Components:",
+    "id": "components",
+    "level": 2
+  },
+  {
+    value: "DeclarativeSource",
+    id: "/definitions/DeclarativeSource",
+    level: 3
+  },
+  ...Object.keys(schema.definitions).map((id) => ({
+    value: id,
+    id: `/definitions/${id}`,
+    level: 3
+  })),
+    {
+    "value": "Interpolation variables:",
+    "id": "variables",
+    "level": 2
+  },
+  ...schema.interpolation.variables.map((def) => ({
+    value: def.title,
+    id: `/variables/${def.title}`,
+    level: 3
+  })),
+    {
+    "value": "Interpolation macros:",
+    "id": "macros",
+    "level": 2
+  },
+  ...schema.interpolation.macros.map((def) => ({
+    value: def.title,
+    id: `/macros/${def.title}`,
+    level: 3
+  }))
+];

docs/connector-development/connector-builder-ui/incremental-sync.md

@@ -21,7 +21,7 @@ To configure incremental syncs for a stream in the connector builder, you have t
 In the builder UI, these things are specified like this:
 
 - The "Cursor field" is the property in the record that defines the date and time when the record got changed. It's used to decide which records are synced already and which records are "new"
-- The "Datetime format" specifies the format the cursor field is using to specify date and time. Check out the [YAML reference](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L562) for a full list of supported formats.
+- The "Datetime format" specifies the format the cursor field is using to specify date and time. Check out the [YAML reference](/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/DatetimeBasedCursor) for a full list of supported formats.
 - "API time filtering capabilities" specifies if the API allows filtering by start and end datetime or whether it's a "feed" of data going from newest to oldest records. See the "Incremental sync without time filtering" section below for details.
 - The "Start datetime" is the initial start date of the time range to fetch records for. When doing incremental syncs, the second sync will overwrite this date with the last record that got synced so far.
 - The "End datetime" is the end date of the time range to fetch records for. In most cases it's set to the current date and time when the sync is started to sync all changes that happened so far.
@@ -85,7 +85,7 @@ curl 'https://content.guardianapis.com/search?from-date=<b>2023-04-15T07:30:58Z<
 The `from-date` is set to the cutoff date of articles synced already and the `to-date` is set to the current date.
 
 :::info
-In some cases, it's helpful to reference the start and end date of the interval that's currently synced, for example if it needs to be injected into the URL path of the current stream. In these cases it can be referenced using the `{{ stream_interval.start_date }}` and `{{ stream_interval.end_date }}` [placeholders](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L2073). Check out [the tutorial](./tutorial.mdx#adding-incremental-reads) for such a case.
+In some cases, it's helpful to reference the start and end date of the interval that's currently synced, for example if it needs to be injected into the URL path of the current stream. In these cases it can be referenced using the `{{ stream_interval.start_date }}` and `{{ stream_interval.end_date }}` [placeholders](/connector-development/config-based/understanding-the-yaml-file/reference#variables). Check out [the tutorial](./tutorial.mdx#adding-incremental-reads) for such a case.
 :::
 
 ## Incremental sync without time filtering
@@ -162,7 +162,7 @@ Using the "Inject start time / end time into outgoing HTTP request" option in th
 - The value needs to be injected into the URL path
 - Some conditional logic needs to be applied
 
-To handle these cases, disable injection in the incremental sync form and use the generic parameter section at the bottom of the stream configuration form to freely configure query parameters, headers and properties of the JSON body, by using jinja expressions and [available variables](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L2073). You can also use these variables as part of the URL path.
+To handle these cases, disable injection in the incremental sync form and use the generic parameter section at the bottom of the stream configuration form to freely configure query parameters, headers and properties of the JSON body, by using jinja expressions and [available variables](/connector-development/config-based/understanding-the-yaml-file/reference/#/variables). You can also use these variables as part of the URL path.
 
 For example the [Sendgrid API](https://docs.sendgrid.com/api-reference/e-mail-activity/filter-all-messages) requires setting both start and end time in a `query` parameter.
 For this case, you can use the `stream_interval` variable to configure a query parameter with "key" `query` and "value" `last_event_time BETWEEN TIMESTAMP "{{stream_interval.start_time}}" AND TIMESTAMP "{{stream_interval.end_time}}"` to filter down to the right window in time.

docs/connector-development/connector-builder-ui/pagination.md

@@ -276,7 +276,7 @@ Using the "Inject page size / limit / offset into outgoing HTTP request" option
 * The value needs to be injected into the URL path
 * Some conditional logic needs to be applied
 
-To handle these cases, disable injection in the pagination form and use the generic parameter section at the bottom of the stream configuration form to freely configure query parameters, headers and properties of the JSON body, by using jinja expressions and [available variables](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L2073). You can also use these variables as part of the URL path.
+To handle these cases, disable injection in the pagination form and use the generic parameter section at the bottom of the stream configuration form to freely configure query parameters, headers and properties of the JSON body, by using jinja expressions and [available variables](/connector-development/config-based/understanding-the-yaml-file/reference/#/variables). You can also use these variables as part of the URL path.
 
 For example the [Prestashop API](https://devdocs.prestashop-project.org/8/webservice/cheat-sheet/#list-options) requires to set offset and limit separated by a comma into a single query parameter (`?limit=<offset>,<limit>`)
 For this case, you can use the `next_page_token` variable to configure a query parameter with key `limit` and value `{{ next_page_token['next_page_token'] or '0' }},50` to inject the offset from the pagination strategy and a hardcoded limit of 50 into the same parameter.

docs/connector-development/connector-builder-ui/partitioning.md

@@ -21,7 +21,7 @@ The first two options are a "static" form of partition routing (because the part
 ### List partition router
 
 To configure static partitioning, choose the "List" method for the partition router. The following fields have to be configured:
-* The "partition values" can either be set to a list of strings, making the partitions part of the connector itself or delegated to a user input so the end user configuring a Source based on the connector can control which partitions to fetch. When using "user input" mode for the partition values, create a user input of type array and reference it as the value using the [placeholder](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L1371-L1372) value using `{{ config['<your chosen user input name>'] }}`
+* The "partition values" can either be set to a list of strings, making the partitions part of the connector itself or delegated to a user input so the end user configuring a Source based on the connector can control which partitions to fetch. When using "user input" mode for the partition values, create a user input of type array and reference it as the value using the [placeholder](/connector-development/config-based/understanding-the-yaml-file/reference#variables) value using `{{ config['<your chosen user input name>'] }}`
 * The "Current partition value identifier" can be freely choosen and is the identifier of the variable holding the current partition value. It can for example be used in the path of the stream using the `{{ stream_partition.<identifier> }}` syntax.
 * The "Inject partition value into outgoing HTTP request" option allows you to configure how to add the current partition value to the requests
 
@@ -63,7 +63,7 @@ To fetch the list of partitions (in this example surveys or orders) from the API
 The following fields have to be configured to use the substream partition router:
 * The "Parent stream" defines the records of which stream should be used as partitions
 * The "Parent key" is the property on the parent stream record that should become the partition value (in most cases this is some form of id)
-* The "Current partition value identifier" can be freely chosen and is the identifier of the variable holding the current partition value. It can for example be used in the path of the stream using the `{{ stream_partition.<identifier> }}` [interpolation placeholder](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L2137-L2144).
+* The "Current partition value identifier" can be freely choosen and is the identifier of the variable holding the current partition value. It can for example be used in the path of the stream using the `{{ stream_partition.<identifier> }}` [interpolation placeholder](/connector-development/config-based/understanding-the-yaml-file/reference#variables).
 
 #### Example
 
@@ -127,4 +127,4 @@ Using the "Inject partition value into outgoing HTTP request" option in the part
 * The value needs to be injected into the URL path
 * Some conditional logic needs to be applied
 
-To handle these cases, disable injection in the partitioning form and use the generic parameter section at the bottom of the stream configuration form to freely configure query parameters, headers and properties of the JSON body, by using jinja expressions and [available variables](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L2073). You can also use these variables (like `stream_partition`) as part of the URL path as shown in the Woocommerce example above.
+To handle these cases, disable injection in the partitioning form and use the generic parameter section at the bottom of the stream configuration form to freely configure query parameters, headers and properties of the JSON body, by using jinja expressions and [available variables](/connector-development/config-based/understanding-the-yaml-file/reference/#/variables). You can also use these variables (like `stream_partition`) as part of the URL path as shown in the Woocommerce example above.

docs/connector-development/connector-builder-ui/tutorial.mdx

@@ -120,7 +120,7 @@ To do so, follow these steps:
 * Set the name to "Base"
 * Click "Create"
 
-The value input is now set to `{{ config['base'] }}`. When making requests, the connector will replace this placeholder by the user supplied value. This syntax can be used in all fields that have a user icon on the right side, see the full reference [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L2072).
+The value input is now set to `{{ config['base'] }}`. When making requests, the connector will replace this placeholder by the user supplied value. This syntax can be used in all fields that have a user icon on the right side, see the full reference [here](/connector-development/config-based/understanding-the-yaml-file/reference#variables).
 
 
 Now your connector has a second configuration input. To test it, click the "Testing values" button again, set the "Base" to `USD`. Then, click the "Test" button again to issue a new test read.
@@ -150,7 +150,7 @@ In this section, we'll update the source to read historical data instead of only
 According to the API documentation, we can read the exchange rate for a specific date range by querying the `"/exchangerates_data/{date}"` endpoint instead of `"/exchangerates_data/latest"`.
 
 To configure your connector to request every day individually, follow these steps:
-* On top of the form, change the "URL Path" input to `/exchangerates_data/{{ stream_interval.start_time }}` to [inject](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/declarative/declarative_component_schema.yaml#L2131-L2136) the date to fetch data for into the path of the request
+* On top of the form, change the "URL Path" input to `/exchangerates_data/{{ stream_interval.start_time }}` to [inject](/connector-development/config-based/understanding-the-yaml-file/reference#variables) the date to fetch data for into the path of the request
 * Enable "Incremental sync" for the Rates stream
 * Set the "Cursor Field" to `date` - this is the property in our records to check what date got synced last
 * Set the "Cursor Field Datetime Format" to `%Y-%m-%d` to match the format of the date in the record returned from the API

docusaurus/sidebars.js

@@ -175,6 +175,7 @@ const buildAConnector = {
             "connector-development/config-based/understanding-the-yaml-file/pagination",
             "connector-development/config-based/understanding-the-yaml-file/partition-router",
             "connector-development/config-based/understanding-the-yaml-file/record-selector",
+            "connector-development/config-based/understanding-the-yaml-file/reference",
           ],
         },
         "connector-development/config-based/advanced-topics",