Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HLD of group consumption. #200

Open
6 of 9 tasks
Tracked by #185
anuj-flipkart opened this issue Dec 13, 2024 · 2 comments
Open
6 of 9 tasks
Tracked by #185

HLD of group consumption. #200

anuj-flipkart opened this issue Dec 13, 2024 · 2 comments
Assignees
Milestone

Comments

@anuj-flipkart
Copy link
Collaborator

anuj-flipkart commented Dec 13, 2024

Local Region / What happens locally

  • Message Consumption. Normal Flow.

  • Offset reset to previous point.

  1. In MQ

  2. In RQ / DLQ

  • Offset reset to future point.
  1. In MQ
  2. In RQ / DLQ
  • Group State storage and serving

  • Subscription Stop / Start

  • Shard count change

Global / Geo Replication. What happens globally

  • What are the possible topologies?
    1 -> N. active passive replicated topic. Its not a global topic, but just a replicated one for DR scenarios.

Not tackling:
N -> N. truly global topic. produce anywhere, consume anywhere. No ordering by default.
N -> N. n sets of active passive replicated topic. What is this good for? Cant really provide the ordering post failover like we can give in case of (a).

  • Produce Failover.

  • Subscription Failover.

@anuj-flipkart anuj-flipkart converted this from a draft issue Dec 13, 2024
@anuj-flipkart anuj-flipkart moved this to Todo in Varadhi Dec 13, 2024
@anuj-flipkart anuj-flipkart added this to the Consumer milestone Dec 19, 2024
@anuj-flipkart anuj-flipkart moved this from Todo to In Progress in Varadhi Dec 20, 2024
@gaurav-ashok
Copy link
Contributor

@gauravAshok
Copy link
Collaborator

Currently exploring the issues due to replication lag in geo-replication.,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Progress
Development

No branches or pull requests

3 participants