From 8129bd156d4c661ea194cd800729803da5e308d8 Mon Sep 17 00:00:00 2001 From: Tomas Ekeli Date: Thu, 30 May 2024 15:56:47 +0200 Subject: [PATCH 1/8] Describe queue-name, policy and dead-letter queue aspects Describe how to get past stumbling-blocks I met when setting up a service-bus queue pubsub. Signed-off-by: Tomas Ekeli --- .../supported-pubsub/setup-azure-servicebus-queues.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md index 57e3b92868d..a8d07cd4f0c 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md @@ -181,6 +181,17 @@ When subscribing to a topic, you can configure `bulkSubscribe` options. Refer to Follow the instructions [here](https://learn.microsoft.com/azure/service-bus-messaging/service-bus-quickstart-portal) on setting up Azure Service Bus Queues. +{{% alert title="Note" color="primary" %}} +Your queue will need to have the same name as the Topic you are publishing to with Dapr. E.g. if you are publishing to the pubsb "myPubsub" on the topic "orders" your queue must be named "orders". +If you are using a Shared access policy to connect to the queue that policy must have the ability to "Manage" the queue. To work with a dead-letter queue the policy must live on the Service Bus Namespace that contains both the main queue and the dead-letter queue. +{{% /alert %}} + +### Retry policy and dead-letter queues + +An Azure ServiceBus Queue has a dead-letter quue by default. If you do not modify it the message will be retried the "Max delivery count" number of times (defaults to 10, can be set up to 2000). These retries will happen with very little delay before the message is put in the dead-letter queue. + +Since Dapr Pubsub has its own dead-letter queue -concept you can use this instead if you need control over retry-policy and you need to subscribe to the dead-letter queue (you probably should) you should set up a separate queue as that dead-letter queue in the same namespace, and a resilience component that defines how to retry on input. + ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}}) From e912c50d01d51550f9787c1fa20eaab4b876af52 Mon Sep 17 00:00:00 2001 From: Tomas Ekeli Date: Thu, 30 May 2024 16:02:04 +0200 Subject: [PATCH 2/8] Update setup-azure-servicebus-queues.md Signed-off-by: Tomas Ekeli --- .../supported-pubsub/setup-azure-servicebus-queues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md index a8d07cd4f0c..ac428de2ac3 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md @@ -190,7 +190,7 @@ If you are using a Shared access policy to connect to the queue that policy must An Azure ServiceBus Queue has a dead-letter quue by default. If you do not modify it the message will be retried the "Max delivery count" number of times (defaults to 10, can be set up to 2000). These retries will happen with very little delay before the message is put in the dead-letter queue. -Since Dapr Pubsub has its own dead-letter queue -concept you can use this instead if you need control over retry-policy and you need to subscribe to the dead-letter queue (you probably should) you should set up a separate queue as that dead-letter queue in the same namespace, and a resilience component that defines how to retry on input. +Since Dapr Pubsub has its own dead-letter queue -concept you can use this instead if you need control over retry-policy and you need to subscribe to the dead-letter queue (you probably should). Set up a separate queue as that dead-letter queue in the same namespace, and a resilience component that defines how to retry on input. Then subscribe to that queue (topic) to get the failed messages and deal with them. ## Related links From b924132113ef153ffd7414bacdfb10f6650afbd3 Mon Sep 17 00:00:00 2001 From: Tomas Ekeli Date: Thu, 30 May 2024 16:12:06 +0200 Subject: [PATCH 3/8] Update setup-azure-servicebus-queues.md Signed-off-by: Tomas Ekeli --- .../supported-pubsub/setup-azure-servicebus-queues.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md index ac428de2ac3..7057c27c4fe 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md @@ -182,15 +182,15 @@ When subscribing to a topic, you can configure `bulkSubscribe` options. Refer to Follow the instructions [here](https://learn.microsoft.com/azure/service-bus-messaging/service-bus-quickstart-portal) on setting up Azure Service Bus Queues. {{% alert title="Note" color="primary" %}} -Your queue will need to have the same name as the Topic you are publishing to with Dapr. E.g. if you are publishing to the pubsb "myPubsub" on the topic "orders" your queue must be named "orders". +Your queue must have the same name as the Topic you are publishing to with Dapr. E.g. if you are publishing to the pubsb "myPubsub" on the topic "orders" your queue must be named "orders". If you are using a Shared access policy to connect to the queue that policy must have the ability to "Manage" the queue. To work with a dead-letter queue the policy must live on the Service Bus Namespace that contains both the main queue and the dead-letter queue. {{% /alert %}} ### Retry policy and dead-letter queues -An Azure ServiceBus Queue has a dead-letter quue by default. If you do not modify it the message will be retried the "Max delivery count" number of times (defaults to 10, can be set up to 2000). These retries will happen with very little delay before the message is put in the dead-letter queue. +An Azure ServiceBus Queue has a dead-letter queue by default. By default the messages are retried the "Max delivery count" number of times (defaults to 10, can be set up to 2000). These retries happen very rapidly and the message is put in the dead-letter queue if no success is returned. -Since Dapr Pubsub has its own dead-letter queue -concept you can use this instead if you need control over retry-policy and you need to subscribe to the dead-letter queue (you probably should). Set up a separate queue as that dead-letter queue in the same namespace, and a resilience component that defines how to retry on input. Then subscribe to that queue (topic) to get the failed messages and deal with them. +Dapr Pubsub has its own dead-letter queue -concept you can use instead. This lets you control the retry-policy and you can subscribe to the dead-letter queue through Dapr. Set up a separate queue as that dead-letter queue in the Azure Service Bus Namespace, and a resilience component that defines how to retry. You can subscribe to that (topic) to get the failed messages and deal with them. E.g. setting up a dead-letter queue `orders-dlq` in the subscription and a resilience lets your subscribe to the topic `orders-dlq` to handle failed messages. ## Related links From 1d91ccd727a5ddc4d0a7644920b6bd9a7f814f7d Mon Sep 17 00:00:00 2001 From: Tomas Ekeli Date: Fri, 31 May 2024 00:09:59 +0200 Subject: [PATCH 4/8] change resilience component to policy and link pubsub-deadletter Signed-off-by: Tomas Ekeli --- .../supported-pubsub/setup-azure-servicebus-queues.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md index 7057c27c4fe..fc6d95915f9 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md @@ -190,7 +190,9 @@ If you are using a Shared access policy to connect to the queue that policy must An Azure ServiceBus Queue has a dead-letter queue by default. By default the messages are retried the "Max delivery count" number of times (defaults to 10, can be set up to 2000). These retries happen very rapidly and the message is put in the dead-letter queue if no success is returned. -Dapr Pubsub has its own dead-letter queue -concept you can use instead. This lets you control the retry-policy and you can subscribe to the dead-letter queue through Dapr. Set up a separate queue as that dead-letter queue in the Azure Service Bus Namespace, and a resilience component that defines how to retry. You can subscribe to that (topic) to get the failed messages and deal with them. E.g. setting up a dead-letter queue `orders-dlq` in the subscription and a resilience lets your subscribe to the topic `orders-dlq` to handle failed messages. +Dapr Pubsub has its own dead-letter queue -concept you can use instead. This lets you control the retry-policy and you can subscribe to the dead-letter queue through Dapr. Set up a separate queue as that dead-letter queue in the Azure Service Bus Namespace, and a resilience policy that defines how to retry. You can subscribe to that (topic) to get the failed messages and deal with them. E.g. setting up a dead-letter queue `orders-dlq` in the subscription and a resilience lets your subscribe to the topic `orders-dlq` to handle failed messages. + +For more details on setting up dead-letter queues, see the [dead-letter article]({{< ref pubsub-deadletter >}}). ## Related links From db646a0ed77af44113b01e8c4b71e42aa7054bd6 Mon Sep 17 00:00:00 2001 From: Tomas Ekeli Date: Sun, 30 Jun 2024 20:28:46 +0200 Subject: [PATCH 5/8] Update daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md Co-authored-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com> Signed-off-by: Tomas Ekeli --- .../supported-pubsub/setup-azure-servicebus-queues.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md index cb9a30954f3..20057ee3572 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md @@ -190,7 +190,11 @@ If you are using a Shared access policy to connect to the queue that policy must An Azure ServiceBus Queue has a dead-letter queue by default. By default the messages are retried the "Max delivery count" number of times (defaults to 10, can be set up to 2000). These retries happen very rapidly and the message is put in the dead-letter queue if no success is returned. -Dapr Pubsub has its own dead-letter queue -concept you can use instead. This lets you control the retry-policy and you can subscribe to the dead-letter queue through Dapr. Set up a separate queue as that dead-letter queue in the Azure Service Bus Namespace, and a resilience policy that defines how to retry. You can subscribe to that (topic) to get the failed messages and deal with them. E.g. setting up a dead-letter queue `orders-dlq` in the subscription and a resilience lets your subscribe to the topic `orders-dlq` to handle failed messages. +Dapr Pub/sub offers its own dead-letter queue concept that lets you control the retry policy and subscribe to the dead-letter queue through Dapr. +1. Set up a separate queue as that dead-letter queue in the Azure Service Bus namespace, and a resilience policy that defines how to retry. +1. Subscribe to the topic to get the failed messages and deal with them. + +For example, setting up a dead-letter queue `orders-dlq` in the subscription and a resiliency policy lets you subscribe to the topic `orders-dlq` to handle failed messages. For more details on setting up dead-letter queues, see the [dead-letter article]({{< ref pubsub-deadletter >}}). From 1fed5a0922469f9a33f88b953d55b90f93e5d1ab Mon Sep 17 00:00:00 2001 From: Tomas Ekeli Date: Sun, 30 Jun 2024 20:28:57 +0200 Subject: [PATCH 6/8] Update daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md Co-authored-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com> Signed-off-by: Tomas Ekeli --- .../supported-pubsub/setup-azure-servicebus-queues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md index 20057ee3572..fe25df6828a 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md @@ -183,7 +183,7 @@ Follow the instructions [here](https://learn.microsoft.com/azure/service-bus-mes {{% alert title="Note" color="primary" %}} Your queue must have the same name as the Topic you are publishing to with Dapr. E.g. if you are publishing to the pubsb "myPubsub" on the topic "orders" your queue must be named "orders". -If you are using a Shared access policy to connect to the queue that policy must have the ability to "Manage" the queue. To work with a dead-letter queue the policy must live on the Service Bus Namespace that contains both the main queue and the dead-letter queue. +If you are using a shared access policy to connect to the queue, that policy must be able to "manage" the queue. To work with a dead-letter queue, the policy must live on the Service Bus Namespace that contains both the main queue and the dead-letter queue. {{% /alert %}} ### Retry policy and dead-letter queues From 74a87a7c3f62717328df9bfe5f1a670059881889 Mon Sep 17 00:00:00 2001 From: Tomas Ekeli Date: Sun, 30 Jun 2024 20:29:05 +0200 Subject: [PATCH 7/8] Update daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md Co-authored-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com> Signed-off-by: Tomas Ekeli --- .../supported-pubsub/setup-azure-servicebus-queues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md index fe25df6828a..baf09362515 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md @@ -182,7 +182,7 @@ When subscribing to a topic, you can configure `bulkSubscribe` options. Refer to Follow the instructions [here](https://learn.microsoft.com/azure/service-bus-messaging/service-bus-quickstart-portal) on setting up Azure Service Bus Queues. {{% alert title="Note" color="primary" %}} -Your queue must have the same name as the Topic you are publishing to with Dapr. E.g. if you are publishing to the pubsb "myPubsub" on the topic "orders" your queue must be named "orders". +Your queue must have the same name as the topic you are publishing to with Dapr. For example, if you are publishing to the pub/sub `"myPubsub"` on the topic `"orders"`, your queue must be named `"orders"`. If you are using a shared access policy to connect to the queue, that policy must be able to "manage" the queue. To work with a dead-letter queue, the policy must live on the Service Bus Namespace that contains both the main queue and the dead-letter queue. {{% /alert %}} From b7587634afd7e2c82e601bcba0c50ff568f1b54d Mon Sep 17 00:00:00 2001 From: Tomas Ekeli Date: Sun, 30 Jun 2024 20:29:13 +0200 Subject: [PATCH 8/8] Update daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md Co-authored-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com> Signed-off-by: Tomas Ekeli --- .../supported-pubsub/setup-azure-servicebus-queues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md index baf09362515..2bc7ca5f984 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-azure-servicebus-queues.md @@ -188,7 +188,7 @@ If you are using a shared access policy to connect to the queue, that policy mus ### Retry policy and dead-letter queues -An Azure ServiceBus Queue has a dead-letter queue by default. By default the messages are retried the "Max delivery count" number of times (defaults to 10, can be set up to 2000). These retries happen very rapidly and the message is put in the dead-letter queue if no success is returned. +By default, an Azure Service Bus Queue has a dead-letter queue. The messages are retried the amount given for `maxDeliveryCount`. The default `maxDeliveryCount` value defaults to 10, but can be set up to 2000. These retries happen very rapidly and the message is put in the dead-letter queue if no success is returned. Dapr Pub/sub offers its own dead-letter queue concept that lets you control the retry policy and subscribe to the dead-letter queue through Dapr. 1. Set up a separate queue as that dead-letter queue in the Azure Service Bus namespace, and a resilience policy that defines how to retry.