Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pkg/ratelimit: introduce an executor that can run with a limiter #8024

Merged
merged 8 commits into from
Apr 9, 2024
Merged

pkg/ratelimit: introduce an executor that can run with a limiter #8024

merged 8 commits into from
Apr 9, 2024

Conversation

nolouch
Copy link
Contributor

@nolouch nolouch commented Apr 3, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: Ref #7897

part of #7898

What is changed and how does it work?

pkg/ratelimit: Introduce an executor that can run with a limiter
- async runner can run the task with the limiter, the task will parallel run.
- sync runner used to keep the behavior like before

Check List

Tests

  • Unit test
None.

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Apr 3, 2024 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • CabinfeverB
  • rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Apr 3, 2024
@nolouch nolouch changed the title pkg/ratelimit: introduce an executor that can run with a rate limiter pkg/ratelimit: introduce an executor that can run with a limiter Apr 3, 2024
@nolouch nolouch requested a review from HuSharp April 3, 2024 10:59
@nolouch nolouch force-pushed the introduce-runner branch 2 times, most recently from f71c46a to df0a9d3 Compare April 3, 2024 11:05
@nolouch nolouch mentioned this pull request Apr 3, 2024
Closed
7 tasks
@codecov Codecov
Copy link

codecov bot commented Apr 3, 2024 •
edited
Loading

Codecov Report

Merging #8024 (ee8d546) into master (ad3387c) will increase coverage by 8.30%.
The diff coverage is 81.25%.

❗ Current head ee8d546 differs from pull request most recent head ef574fe. Consider uploading reports for the commit ef574fe to get more accurate results

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #8024      +/-   ##
==========================================
+ Coverage   65.33%   73.63%   +8.30%     
==========================================
  Files         464      437      -27     
  Lines       52389    48618    -3771     
==========================================
+ Hits        34226    35798    +1572     
+ Misses      15152     9755    -5397     
- Partials     3011     3065      +54
Flag Coverage Δ
unittests 73.63% <81.25%> (+8.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@nolouch
Copy link
Contributor Author

nolouch commented Apr 7, 2024

PTAL @rleungx @HuSharp

CabinfeverB
CabinfeverB reviewed Apr 9, 2024
View reviewed changes
pkg/ratelimit/runner.go Show resolved Hide resolved
pkg/ratelimit/concurrency_limiter.go Show resolved Hide resolved
pkg/ratelimit/runner.go
// Task is a task to be run.
type Task struct {
Ctx context.Context
Opts TaskOpts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to use pointer?

Copy link
Contributor Author

@nolouch nolouch Apr 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok now. it is not allocated in the heap.

pkg/ratelimit/runner.go Outdated Show resolved Hide resolved
rleungx
rleungx reviewed Apr 9, 2024
View reviewed changes
pkg/ratelimit/concurrency_limiter.go Outdated Show resolved Hide resolved
pkg/ratelimit/concurrency_limiter.go Outdated Show resolved Hide resolved
rleungx
rleungx reviewed Apr 9, 2024
View reviewed changes
pkg/ratelimit/concurrency_limiter.go
}
tt.released = true
tt.limiter.current--
if len(tt.limiter.queue) < int(tt.limiter.limit) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current and the queue share the same limit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes.

@nolouch
address
aa88f60 
Signed-off-by: nolouch <[email protected]>
@ti-chi-bot ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 9, 2024
@ti-chi-bot ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Apr 9, 2024
@nolouch
Copy link
Contributor Author

nolouch commented Apr 9, 2024

/merge

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Apr 9, 2024

@nolouch: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Apr 9, 2024

This pull request has been accepted and is ready to merge.

Commit hash: aa88f60

@ti-chi-bot ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 9, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Apr 9, 2024

@nolouch: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot bot merged commit 35e8e95 into tikv:master Apr 9, 2024
22 checks passed
@nolouch nolouch deleted the introduce-runner branch April 9, 2024 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CabinfeverB CabinfeverB CabinfeverB approved these changes

@rleungx rleungx rleungx approved these changes

@HuSharp HuSharp Awaiting requested review from HuSharp

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nolouch @CabinfeverB @rleungx