Skip to content

Commit

Permalink
chore(docs): Clarify behavior of sources when any sink has acks enabl…
Browse files Browse the repository at this point in the history 
…ed (#20910)

Sources wait for all connected sinks to ack even if only one connected sink has acks enabled.

Signed-off-by: Jesse Szwedko <[email protected]>
  • Loading branch information
@jszwedko
jszwedko authored Jul 23, 2024
1 parent 21548fc commit 6a77cc3
Show file tree
Hide file tree
Showing 57 changed files with 114 additions and 114 deletions.
4 changes: 2 additions & 2 deletions lib/vector-core/src/config/mod.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -335,8 +335,8 @@ pub struct AcknowledgementsConfig {
/// Whether or not end-to-end acknowledgements are enabled.
///
/// When enabled for a sink, any source connected to that sink, where the source supports
/// end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
/// before acknowledging them at the source.
/// end-to-end acknowledgements as well, waits for events to be acknowledged by **all
/// connected** sinks before acknowledging them at the source.
///
/// Enabling or disabling acknowledgements at the sink level takes precedence over any global
/// [`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions ...ontent/en/docs/about/under-the-hood/architecture/end-to-end-acknowledgements.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ whether an event was successfully processed or not.

Additionally, Vector ensures that the batch notifier for an event is always updated, whether or not
the event made it to a sink. This ensures that if an event is intentionally dropped (for example, by
using a [`filter`][filter] transform) or even unintentionally dropped (maybe Vector had a bug, uh
oh!), we still update the batch notifier to indicate the processing status of the event.
using a [`filter`][filter_transform] transform) or even unintentionally dropped (maybe Vector had
a bug, uh oh!), we still update the batch notifier to indicate the processing status of the event.

Meanwhile, the source will hold on to the other half of the batch notifiers that it has created, and
is notified when a batch notifier is updated. Once notified, a source will propagate that batch
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/amqp.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: amqp: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/appsignal.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: appsignal: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/aws_cloudwatch_logs.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: aws_cloudwatch_logs: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/aws_cloudwatch_metrics.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: aws_cloudwatch_metrics: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/aws_kinesis_firehose.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: aws_kinesis_firehose: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/aws_kinesis_streams.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: aws_kinesis_streams: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/aws_s3.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: aws_s3: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/aws_sns.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: aws_sns: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/aws_sqs.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: aws_sqs: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/axiom.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: axiom: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/azure_blob.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: azure_blob: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/azure_monitor_logs.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: azure_monitor_logs: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/blackhole.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: blackhole: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/clickhouse.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: clickhouse: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/console.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: console: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/databend.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: databend: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/datadog_events.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: datadog_events: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/datadog_logs.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: datadog_logs: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/datadog_metrics.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: datadog_metrics: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/datadog_traces.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: datadog_traces: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/elasticsearch.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: elasticsearch: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/file.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: file: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/gcp_chronicle_unstructured.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: gcp_chronicle_unstructured: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/gcp_cloud_storage.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: gcp_cloud_storage: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/gcp_pubsub.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: gcp_pubsub: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/gcp_stackdriver_logs.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: gcp_stackdriver_logs: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/gcp_stackdriver_metrics.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: gcp_stackdriver_metrics: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
4 changes: 2 additions & 2 deletions website/cue/reference/components/sinks/base/greptimedb.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ base: components: sinks: greptimedb: configuration: {
Whether or not end-to-end acknowledgements are enabled.
When enabled for a sink, any source connected to that sink, where the source supports
end-to-end acknowledgements as well, waits for events to be acknowledged by the sink
before acknowledging them at the source.
end-to-end acknowledgements as well, waits for events to be acknowledged by **all
connected** sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
[`acknowledgements`][global_acks] configuration.
Expand Down
Loading

0 comments on commit 6a77cc3

Please sign in to comment.