Skip to content

Commit

Permalink
Add a note about increases in backlog when replicated subscription is…
Browse files Browse the repository at this point in the history
… activated (#916)

* Solves apache/pulsar/issues/22926

Tries to clarify a possible increase in backlog and from where it is
coming

* Update version 3.3
  • Loading branch information
AlvaroStream committed Jul 5, 2024
1 parent 8d01016 commit 8a94924
Show file tree
Hide file tree
Showing 5 changed files with 30 additions and 0 deletions.
6 changes: 6 additions & 0 deletions docs/administration-geo.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,12 @@ The limitations of replicated subscription are as follows.
* When you enable replicated subscriptions, you're creating a consistent distributed snapshot to establish an association between message ids from different clusters. The snapshots are taken periodically. The default value is `1 second`. It means that a consumer failing over to a different cluster can potentially receive 1 second of duplicates. You can also configure the frequency of the snapshot in the `broker.conf` file.
* Only the base line cursor position is synced in replicated subscriptions while the individual acknowledgments are not synced. This means the messages acknowledged out-of-order could end up getting delivered again, in the case of a cluster failover.
:::note
* This replicated subscription will add a new special message every second, it will contains the [snapshot](https://github.com/apache/pulsar/wiki/PIP-33:-Replicated-subscriptions#storing-snapshots) of the subscription. That means, if there are inactive subscriptions over the topic there can be an increase in backlog in source and destination clusters.
:::
## Migrate data between clusters using geo-replication
Using geo-replication to migrate data between clusters is a special use case of the [active-active replication pattern](concepts-replication.md#active-active-replication) when you don't have a large amount of data.
Expand Down
6 changes: 6 additions & 0 deletions versioned_docs/version-2.11.x/administration-geo.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,12 @@ If you want to use replicated subscriptions in Pulsar:
* When you enable replicated subscriptions, you're creating a consistent distributed snapshot to establish an association between message ids from different clusters. The snapshots are taken periodically. The default value is `1 second`. It means that a consumer failing over to a different cluster can potentially receive 1 second of duplicates. You can also configure the frequency of the snapshot in the `broker.conf` file.
* Only the base line cursor position is synced in replicated subscriptions while the individual acknowledgments are not synced. This means the messages acknowledged out-of-order could end up getting delivered again, in the case of a cluster failover.
:::note
* This replicated subscription will add a new special message every second, it will contains the [snapshot](https://github.com/apache/pulsar/wiki/PIP-33:-Replicated-subscriptions#storing-snapshots) of the subscription. That means, if there are inactive subscriptions over the topic there can be an increase in backlog in source and destination clusters.
:::
## Migrate data between clusters using geo-replication
Using geo-replication to migrate data between clusters is a special use case of the [active-active replication pattern](concepts-replication.md#active-active-replication) when you don't have a large amount of data.
Expand Down
6 changes: 6 additions & 0 deletions versioned_docs/version-3.0.x/administration-geo.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,12 @@ If you want to use replicated subscriptions in Pulsar:
* When you enable replicated subscriptions, you're creating a consistent distributed snapshot to establish an association between message ids from different clusters. The snapshots are taken periodically. The default value is `1 second`. It means that a consumer failing over to a different cluster can potentially receive 1 second of duplicates. You can also configure the frequency of the snapshot in the `broker.conf` file.
* Only the base line cursor position is synced in replicated subscriptions while the individual acknowledgments are not synced. This means the messages acknowledged out-of-order could end up getting delivered again, in the case of a cluster failover.

:::note

* This replicated subscription will add a new special message every second, it will contains the [snapshot](https://github.com/apache/pulsar/wiki/PIP-33:-Replicated-subscriptions#storing-snapshots) of the subscription. That means, if there are inactive subscriptions over the topic there can be an increase in backlog in source and destination clusters.

:::


## Migrate data between clusters using geo-replication

Expand Down
6 changes: 6 additions & 0 deletions versioned_docs/version-3.2.x/administration-geo.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,12 @@ The limitations of replicated subscription are as follows.
* When you enable replicated subscriptions, you're creating a consistent distributed snapshot to establish an association between message ids from different clusters. The snapshots are taken periodically. The default value is `1 second`. It means that a consumer failing over to a different cluster can potentially receive 1 second of duplicates. You can also configure the frequency of the snapshot in the `broker.conf` file.
* Only the base line cursor position is synced in replicated subscriptions while the individual acknowledgments are not synced. This means the messages acknowledged out-of-order could end up getting delivered again, in the case of a cluster failover.
:::note
* This replicated subscription will add a new special message every second, it will contains the [snapshot](https://github.com/apache/pulsar/wiki/PIP-33:-Replicated-subscriptions#storing-snapshots) of the subscription. That means, if there are inactive subscriptions over the topic there can be an increase in backlog in source and destination clusters.
:::
## Migrate data between clusters using geo-replication
Using geo-replication to migrate data between clusters is a special use case of the [active-active replication pattern](concepts-replication.md#active-active-replication) when you don't have a large amount of data.
Expand Down
6 changes: 6 additions & 0 deletions versioned_docs/version-3.3.x/administration-geo.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,12 @@ The limitations of replicated subscription are as follows.
* When you enable replicated subscriptions, you're creating a consistent distributed snapshot to establish an association between message ids from different clusters. The snapshots are taken periodically. The default value is `1 second`. It means that a consumer failing over to a different cluster can potentially receive 1 second of duplicates. You can also configure the frequency of the snapshot in the `broker.conf` file.
* Only the base line cursor position is synced in replicated subscriptions while the individual acknowledgments are not synced. This means the messages acknowledged out-of-order could end up getting delivered again, in the case of a cluster failover.
:::note
* This replicated subscription will add a new special message every second, it will contains the [snapshot](https://github.com/apache/pulsar/wiki/PIP-33:-Replicated-subscriptions#storing-snapshots) of the subscription. That means, if there are inactive subscriptions over the topic there can be an increase in backlog in source and destination clusters.
:::
## Migrate data between clusters using geo-replication
Using geo-replication to migrate data between clusters is a special use case of the [active-active replication pattern](concepts-replication.md#active-active-replication) when you don't have a large amount of data.
Expand Down

0 comments on commit 8a94924

Please sign in to comment.