Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat][doc] add conceptual docs (bundle assignment/splitting/unloading...) for broker load balancing #644

Merged
merged 2 commits into from
Jul 20, 2023

Conversation

Anonymitaet
Copy link
Member

@Anonymitaet Anonymitaet commented Jul 18, 2023

What change does this PR introduce?

This PR adds conceptual docs for broker load balancing (topics in the red box below)

image

Doc development plan - Read before review

For the current doc of Broker Load Balancer, we want to tackle the following challenges:

  • Create new documentation to cover all aspects of the feature (especially for the new extensible load balancer).
  • Fill in any content gaps that may exist.
  • Consolidate existing documentation into the new IA, as it is currently scattered and disorganized.
  • Organize the documentation in a cohesive manner, ensuring it flows logically.

Consequently, we’ve created the following content development plans:

Ackowledgements

@Demogorgon314 @heesung-sn @D-2-Ed many thanks for your technical guidance and great support!

@Anonymitaet Anonymitaet added the doc Improvements or additions to documentation label Jul 18, 2023
@Anonymitaet Anonymitaet added this to the 3.1.0 milestone Jul 18, 2023
@Anonymitaet Anonymitaet self-assigned this Jul 18, 2023
@Anonymitaet
Copy link
Member Author

Previews go as expected:

image

FireShot Capture 008 - Concepts - Apache Pulsar - localhost

Copy link
Member Author

@Anonymitaet Anonymitaet Jul 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reviewers:

Here are some known issues of illustrations:

Bundle splitting threshold can be set based on various conditions. Any existing bundle that exceeds any of the thresholds is a candidate to be split. The load balancer assigns the newly split bundles to other brokers, to facilitate the traffic distribution.

For how to enable bundle split and set bundle split thresholds automatically, see TBD (the docs is WIP, stay tuned!).
Copy link
Member Author

@Anonymitaet Anonymitaet Jul 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reviewers:

I write this on purpose. Reasons:

  • Give users a positive expectation
  • Call for a targeted list of early adopters interested in this new feature
  • Improve SEO by adding relevant keywords

see TBD (the docs is WIP, stay tuned!).

@Anonymitaet
Copy link
Member Author

@Demogorgon314 @heesung-sn @D-2-Ed this doc has been reviewed by you in Broker Load Balancing | Concepts, can you take a quick look at this PR?

@momo-jun @Huanli-Meng could you please help review this PR?

Thank you all!

@Anonymitaet Anonymitaet changed the title [feat][doc] add conceptual docs (bundle assignment/splitting/algorithm...) for broker load balancing [feat][doc] add conceptual docs (bundle assignment/splitting/unloading...) for broker load balancing Jul 18, 2023
Copy link
Contributor

@Huanli-Meng Huanli-Meng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments.

docs/concepts-broker-load-balancing-concepts.md Outdated Show resolved Hide resolved
docs/concepts-broker-load-balancing-concepts.md Outdated Show resolved Hide resolved
@Anonymitaet Anonymitaet merged commit 9b4b3d3 into apache:main Jul 20, 2023
@Technoboy- Technoboy- mentioned this pull request Aug 9, 2023
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants