-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[4.5][http] Adds url templating and worker-partition-key
- Loading branch information
Showing
4 changed files
with
40 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
16 changes: 16 additions & 0 deletions
16
content/headless/chunk/option-destination-http-url-templating.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
--- | ||
--- | ||
<!-- This file is under the copyright of Axoflow, and licensed under Apache License 2.0, except for using the Axoflow and AxoSyslog trademarks. --> | ||
Available in {{% param "product_name" %}} version 4.5.0 and later. | ||
|
||
In {{% param "product_name" %}}, a template can only be resolved on a single message, because the same template might have different resolutions on different messages. As a batch consists of multiple messages, it's not trivial to decide which message should be used for the resolution. | ||
|
||
When batching is enabled and multiple workers are configured, it's important to add only those messages to a batch which generate identical URLs. To achieve this, set the [`worker-partition-key()` option](#worker-partition-key) with a template that contains all the templates used in the `url()` option, otherwise messages will be mixed. | ||
|
||
For security reasons, all the templated contents in the `url()` option are URL-encoded automatically. The following parts of the URL cannot be templated: | ||
|
||
- scheme | ||
- host | ||
- port | ||
- user | ||
- password |
17 changes: 17 additions & 0 deletions
17
content/headless/chunk/option-destination-http-worker-partition-key.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
--- | ||
--- | ||
<!-- This file is under the copyright of Axoflow, and licensed under Apache License 2.0, except for using the Axoflow and AxoSyslog trademarks. --> | ||
## worker-partition-key() | ||
|
||
| | | | ||
| -------- | ------ | | ||
| Type: | template | | ||
| Default: | | | ||
|
||
*Description:* The `worker-partition-key()` option specifies a template: messages that expand the template to the same value are mapped to the same partition. When batching is enabled and multiple workers are configured, it's important to add only those messages to a batch which generate identical URLs. To achieve this, set the `worker-partition-key()` option with a template that contains all the templates used in the `url()` option, otherwise messages will be mixed. | ||
|
||
For example, you can partition messages based on the destination host: | ||
|
||
```shell | ||
worker-partition-key("$HOST") | ||
``` |