Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

checker: use ttl cache #8505

Merged
merged 3 commits into from
Aug 8, 2024
Merged

checker: use ttl cache #8505

merged 3 commits into from
Aug 8, 2024

Conversation

rleungx
Copy link
Member

@rleungx rleungx commented Aug 8, 2024

What problem does this PR solve?

Issue Number: Close #8500

What is changed and how does it work?

Check List

Tests

  • Unit test

Release note

None.

Signed-off-by: Ryan Leung <[email protected]>
@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has signed the dco. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 8, 2024
Copy link

codecov bot commented Aug 8, 2024

Codecov Report

Attention: Patch coverage is 80.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 77.43%. Comparing base (7ef2be0) to head (779f7bc).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8505      +/-   ##
==========================================
+ Coverage   77.34%   77.43%   +0.08%     
==========================================
  Files         472      472              
  Lines       61796    61795       -1     
==========================================
+ Hits        47799    47850      +51     
+ Misses      10433    10371      -62     
- Partials     3564     3574      +10     
Flag Coverage Δ
unittests 77.43% <80.00%> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@ -88,7 +88,7 @@ type Controller struct {

// NewController create a new Controller.
func NewController(ctx context.Context, cluster sche.CheckerCluster, conf config.CheckerConfigProvider, ruleManager *placement.RuleManager, labeler *labeler.RegionLabeler, opController *operator.Controller) *Controller {
pendingProcessedRegions := cache.NewDefaultCache(DefaultPendingRegionCacheSize)
pendingProcessedRegions := cache.NewIDTTL(ctx, time.Minute, 3*time.Minute)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, previous suspectRegions is *cache.TTLUint64

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to add a test for this issue?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not just the problem of suspectRegions. The waiting list also has the same problem previously.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, then do we need to add a test for this issue?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why 3 mins?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original setting is 3 min.

}

// AddPendingProcessedRegions adds the pending processed region into the cache.
func (c *Controller) AddPendingProcessedRegions(ids ...uint64) {
for _, id := range ids {
if c.pendingProcessedRegions.Len() > DefaultPendingRegionCacheSize {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why to add it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To prevent the cache from becoming too large.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If so, some regions, such as those from checkSuspectRanges or AccelerateRegionsScheduleInRange, may not be put successfully.

Should we only skip adding to pendingProcessedRegions in tryAddOperators when pendingProcessedRegions is full?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Signed-off-by: Ryan Leung <[email protected]>
@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Aug 8, 2024
@ti-chi-bot ti-chi-bot bot added the lgtm label Aug 8, 2024
Copy link
Contributor

ti-chi-bot bot commented Aug 8, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: HuSharp, lhy1024

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot removed the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Aug 8, 2024
Copy link
Contributor

ti-chi-bot bot commented Aug 8, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-08-08 06:00:09.638267812 +0000 UTC m=+504539.505366892: ☑️ agreed by lhy1024.
  • 2024-08-08 06:21:50.865451757 +0000 UTC m=+505840.732550844: ☑️ agreed by HuSharp.

Copy link
Contributor

ti-chi-bot bot commented Aug 8, 2024

@rleungx: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot bot merged commit ff71512 into tikv:master Aug 8, 2024
21 checks passed
@rleungx rleungx deleted the use-ttl branch August 8, 2024 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved dco-signoff: yes Indicates the PR's author has signed the dco. lgtm release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Duration of analyze partition table increased more than 50x
3 participants