Skip to content

Commit

Permalink
Clarify dead-letter queue behavior (#125)
Browse files Browse the repository at this point in the history
* clarify dead-letter queue behavior

* clarify

---------

Co-authored-by: Raúl Barroso <[email protected]>
  • Loading branch information
lovromazgon and raulb authored Aug 21, 2024
1 parent f0d1dd7 commit 62c0dff
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions docs/features/dead-letter-queue.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -49,18 +49,19 @@ By default, a Conduit pipeline will stop immediately when a record gets nacked.
You can make the pipeline more fault-tolerant by enabling a dead-letter queue.
The first step is to change the window size and nack threshold:

- If the window size is greater than 1 and the nack threshold is 0, then the
- If the window size is greater than 0 and the nack threshold is 0, then the
first nacked record will cause the pipeline to stop running immediately. Such
a configuration essentially disables the dead-letter queue. **This is the
default configuration.**
- If the window size is set to 0, then no records are monitored and all nacked
records will be rerouted to the dead-letter queue without any limits. Such a
configuration essentially disables the nack window and enables the dead-letter
queue.
- If the window size is greater than 1 and the nack threshold is greater than 0,
then nacked records will be sent to the dead-letter queue. Additionally,
Conduit will keep track of the number of nacks in the current window and stop
the pipeline if the threshold gets exceeded.
- If both the window size and the nack threshold are greater than 0, then nacked
records will be sent to the dead-letter queue. Conduit will keep track of the
number of nacks in the current window and stop the pipeline if the threshold
gets exceeded. Note that the window size needs to be greater than the nack
threshold, otherwise the threshold will never be reached.

If the window settings are adjusted accordingly, then Conduit will route nacked
records to the builtin log connector by default, causing nacked records to show
Expand Down Expand Up @@ -90,4 +91,4 @@ Please check
the [pipeline config file documentation](/docs/pipeline-configuration-files/getting-started)
for more information about configuring a pipeline.
![scarf pixel conduit-site-docs-features](https://static.scarf.sh/a.png?x-pxid=8c8ff6d5-2756-41a3-b4ca-3ca2be381842)
![scarf pixel conduit-site-docs-features](https://static.scarf.sh/a.png?x-pxid=8c8ff6d5-2756-41a3-b4ca-3ca2be381842)

0 comments on commit 62c0dff

Please sign in to comment.