Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

txnkv: introduce ballast object to batch-get worker #1535

Merged
merged 4 commits into from
Dec 25, 2024

Conversation

zyguan
Copy link
Contributor

@zyguan zyguan commented Dec 21, 2024

Batch-get may spawn multiple goroutines to fetch keys. There is notable cost on stack growing.
image

After some analysis, it shows the batch-get worker requires 8KB stack size currently, that is, the stack will grow to 4KB, then 8KB for each worker groutine. To reduce the overhead, we can introduce a ballast object to let the stack grow to 8KB directly at the begining of the goroutine.
image

The benchmark results show that it helps improve performance.

workload threads baseline this PR diff
batchget 15 19450.06 19548.51 0.51%
batchget 30 24507.40 24914.81 1.66%
batchget 60 26797.00 27451.43 2.44%

BTW, I also tried to use the internal goroutine pool of kvstore, but the test result is not good, the overhead of selectgo is significant, and default 128 pool size is not enough to prevent new goroutines.

20241221-121605

@ti-chi-bot ti-chi-bot bot added dco-signoff: yes Indicates the PR's author has signed the dco. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Dec 21, 2024
@zyguan zyguan changed the title txnkv: introduce ballast object for batch-get worker txnkv: introduce ballast object to batch-get worker Dec 24, 2024
@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Dec 24, 2024
@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 24, 2024
@ti-chi-bot ti-chi-bot bot removed the lgtm label Dec 24, 2024
@cfzjywxk cfzjywxk requested a review from crazycs520 December 24, 2024 09:59
@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Dec 24, 2024
Copy link

ti-chi-bot bot commented Dec 25, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfzjywxk, crazycs520

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [cfzjywxk,crazycs520]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 25, 2024
Copy link

ti-chi-bot bot commented Dec 25, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-12-24 04:01:21.515633967 +0000 UTC m=+1534271.604436515: ☑️ agreed by cfzjywxk.
  • 2024-12-24 04:39:29.161587168 +0000 UTC m=+1536559.250389710: ☑️ agreed by crazycs520.
  • 2024-12-24 08:30:53.525275697 +0000 UTC m=+1550443.614078233: ✖️🔁 reset by zyguan.
  • 2024-12-24 10:00:07.935927534 +0000 UTC m=+1555798.024730073: ☑️ agreed by cfzjywxk.
  • 2024-12-25 10:06:53.03591573 +0000 UTC m=+1642603.124718270: ☑️ agreed by crazycs520.

@ti-chi-bot ti-chi-bot bot merged commit e0415dc into tikv:master Dec 25, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved dco-signoff: yes Indicates the PR's author has signed the dco. lgtm size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants