Skip to content

Commit 1bc4092

Browse files
committed
Add processor options
1 parent 451dfca commit 1bc4092

File tree

2 files changed

+109
-0
lines changed

2 files changed

+109
-0
lines changed

packages/azure/data_stream/eventhub/agent/stream/stream.yml.hbs

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,31 @@ storage_account: {{storage_account}}
1919
{{/if}}
2020
{{#if storage_account_key}}
2121
storage_account_key: {{storage_account_key}}
22+
storage_account_connection_string: DefaultEndpointsProtocol=https;AccountName={{storage_account}};AccountKey={{storage_account_key}};EndpointSuffix={{endpoint_suffix}}
2223
{{/if}}
2324
{{#if resource_manager_endpoint}}
2425
resource_manager_endpoint: {{resource_manager_endpoint}}
2526
{{/if}}
27+
28+
{{#if processor_version}}
29+
processor_version: {{processor_version}}
30+
{{/if}}
31+
{{#if migrate_checkpoint}}
32+
migrate_checkpoint: {{migrate_checkpoint}}
33+
{{/if}}
34+
{{#if processor_update_interval}}
35+
processor_update_interval: {{processor_update_interval}}
36+
{{/if}}
37+
{{#if processor_start_position}}
38+
processor_start_position: {{processor_start_position}}
39+
{{/if}}
40+
{{#if partition_receive_timeout}}
41+
partition_receive_timeout: {{partition_receive_timeout}}
42+
{{/if}}
43+
{{#if partition_receive_count}}
44+
partition_receive_count: {{partition_receive_count}}
45+
{{/if}}
46+
2647
data_stream:
2748
dataset: {{data_stream.dataset}}
2849
tags:

packages/azure/data_stream/eventhub/manifest.yml

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,94 @@ streams:
7474
type: bool
7575
multi: false
7676
default: false
77+
78+
#
79+
# Processor v2 only settings
80+
#
81+
- name: processor_version
82+
type: text
83+
title: Processor version
84+
multi: false
85+
required: false
86+
show_user: false
87+
default: v1
88+
description: "The processor version that the integration should use. Possible values are `v1` and `v2` (preview). \nThe v2 event hub processor is in preview, so using the v1 processor is recommended for typical use cases.\nDefault is `v1`."
89+
- name: processor_update_interval
90+
type: text
91+
title: Processor update interval
92+
multi: false
93+
required: false
94+
show_user: false
95+
default: 10s
96+
description: >-
97+
(Processor v2 only) How often the processor should attempt to claim partitions.
98+
99+
Default is `10` seconds.
100+
- name: processor_start_position
101+
type: text
102+
title: Processor start position
103+
multi: false
104+
required: false
105+
show_user: false
106+
default: earliest
107+
description: >-
108+
(Processor v2 only) Controls from what position in the event hub the processor should start processing messages for all partitions.
109+
110+
Possible values are `earliest` and `latest`.
111+
112+
`earliest` starts processing messages from the last checkpoint, or the beginning of the event hub if no checkpoint is available.
113+
114+
`latest` starts processing messages from the the latest event in the event hub and continues to process new events as they arrive.
115+
116+
Default is `earliest`.
117+
- name: partition_receive_timeout
118+
type: text
119+
title: Partition receive timeout
120+
multi: false
121+
required: false
122+
show_user: false
123+
default: 5s
124+
description: >-
125+
(Processor v2 only) Maximum time to wait before processing the messages received from the event hub.
126+
127+
The partition consumer waits up to a "receive count" or a "receive timeout", whichever comes first.
128+
129+
Default is `5` seconds.
130+
- name: partition_receive_count
131+
type: text
132+
title: Partition receive count
133+
multi: false
134+
required: false
135+
show_user: false
136+
default: 100
137+
description: >-
138+
(Processor v2 only) Maximum number of messages from the event hub to wait for before processing them.
139+
140+
The partition consumer waits up to a "receive count" or a "receive timeout", whichever comes first.
141+
142+
Default is `100` messages.
143+
- name: migrate_checkpoint
144+
type: bool
145+
title: Migrate checkpoint information
146+
multi: false
147+
required: false
148+
show_user: false
149+
default: true
150+
description: >-
151+
(Processor v2 only) Flag to control if the processor should perform the checkpoint information migration from processor v1 to v2 at startup.
152+
153+
The checkpoint migration converts the checkpoint information from the v1 format to the v2 format.
154+
155+
Default is `false`, which means the processor will not perform the checkpoint migration.
156+
- name: endpoint_suffix
157+
type: text
158+
default: core.windows.net
159+
required: true
160+
title: Storage account endpoint suffix
161+
show_user: false
162+
description: >-
163+
(Processor v2 only) Override the default storage account endpoint suffix.
164+
77165
# Ensures agents have permissions to write data to `logs-*-*`
78166
elasticsearch:
79167
dynamic_dataset: true

0 commit comments

Comments
 (0)