Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: optimize heartbeat process with concurrent runner - part 2 #8052

Merged
merged 5 commits into from
Apr 24, 2024

Conversation

nolouch
Copy link
Contributor

@nolouch nolouch commented Apr 10, 2024

What problem does this PR solve?

Issue Number: Ref #7897

What is changed and how does it work?

Optimize heartbeat process
- Split the statics updates on the subtree

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

Before:
image
image

After:
image
image

The duration of the pending state on tikvs has been reduced from 10 minutes to 3 minutes

Release note

None.

Copy link
Contributor

ti-chi-bot bot commented Apr 10, 2024

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • JmPotato
  • rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Apr 10, 2024
@ti-chi-bot ti-chi-bot bot requested review from HunDunDM and rleungx April 10, 2024 09:37
@ti-chi-bot ti-chi-bot bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 10, 2024
@nolouch nolouch changed the title *: optimize heartbeat process with async runner *: optimize heartbeat process with async runner - part 2 Apr 10, 2024
@ti-chi-bot ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 12, 2024
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 15, 2024
@nolouch nolouch changed the title *: optimize heartbeat process with async runner - part 2 *: optimize heartbeat process with concurrent runner - part 2 Apr 15, 2024
Copy link

codecov bot commented Apr 15, 2024

Codecov Report

Merging #8052 (3007e31) into master (5cdf252) will increase coverage by 0.05%.
Report is 2 commits behind head on master.
The diff coverage is 97.69%.

❗ Current head 3007e31 differs from pull request most recent head 15b1979. Consider uploading reports for the commit 15b1979 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8052      +/-   ##
==========================================
+ Coverage   77.29%   77.34%   +0.05%     
==========================================
  Files         470      470              
  Lines       61143    61225      +82     
==========================================
+ Hits        47260    47355      +95     
+ Misses      10320    10309      -11     
+ Partials     3563     3561       -2     
Flag Coverage Δ
unittests 77.34% <97.69%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@nolouch nolouch requested review from JmPotato and HuSharp and removed request for HunDunDM April 16, 2024 02:08
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 16, 2024
@nolouch
Copy link
Contributor Author

nolouch commented Apr 17, 2024

PTAL @rleungx @JmPotato @HuSharp

server/cluster/cluster.go Outdated Show resolved Hide resolved
pkg/ratelimit/metrics.go Outdated Show resolved Hide resolved
Comment on lines +1155 to +1158
collectFromItemSlice(r.leaders, storeID)
collectFromItemSlice(r.followers, storeID)
collectFromItemSlice(r.learners, storeID)
collectFromItemSlice(r.witnesses, storeID)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a possibility that the following situation exists: I detected an overlap in the leaders tree and added item.RegionInfo, then, before I check other role trees, something like a leader transfer occurs. Will I not be able to get the latest item.RegionInfo because of this? Will there be any side effects?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here checks hold the lock, which means the new changes have not updated the subtree yet. So I think it's ok, then will handle the new updates.

pkg/core/region.go Outdated Show resolved Hide resolved
Signed-off-by: nolouch <[email protected]>
Signed-off-by: nolouch <[email protected]>
@ti-chi-bot ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 24, 2024
@ti-chi-bot ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Apr 24, 2024
@JmPotato
Copy link
Member

/merge

Copy link
Contributor

ti-chi-bot bot commented Apr 24, 2024

@JmPotato: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Copy link
Contributor

ti-chi-bot bot commented Apr 24, 2024

This pull request has been accepted and is ready to merge.

Commit hash: f67c8a8

@ti-chi-bot ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 24, 2024
Copy link
Contributor

ti-chi-bot bot commented Apr 24, 2024

@nolouch: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot bot merged commit 141186e into tikv:master Apr 24, 2024
22 checks passed
@nolouch nolouch deleted the optmize-heartbeat branch April 24, 2024 06:54
nolouch added a commit to nolouch/pd that referenced this pull request Apr 29, 2024
)

ref tikv#7897

Optimize heartbeat process
- Split the statics updates on the subtree

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants