fix pruner timing issue with batch pruning #14929
Open
+285
−182
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dB2510
commented
Feb 13, 2025
| @@ -336,7 +325,7 @@ func (s *Store) SaveBlock(ctx context.Context, signed interfaces.ReadOnlySignedB | |||
| // if a `saveBlindedBeaconBlocks` key exists in the database. Otherwise, we check if the last | |||
| // blocked stored to check if it is blinded, and then write that `saveBlindedBeaconBlocks` key | |||
| // to the DB for future checks. | |||
| func (s *Store) shouldSaveBlinded(ctx context.Context) (bool, error) { | |||
| func (s *Store) shouldSaveBlinded() (bool, error) { | |||
There was a problem hiding this comment.
Removed this ctx variable as it was unused
| @@ -159,6 +169,24 @@ func (p *Service) prune(slot primitives.Slot) error { | |||
| return nil | |||
| } | |||
|
|
|||
| func (p *Service) pruneBatches(pruneUpto primitives.Slot) (int, error) { | |||
| ctx, cancel := context.WithDeadline(p.ctx, time.Now().Add(defaultPruningWindow)) | |||
There was a problem hiding this comment.
Suggested change
| ctx, cancel := context.WithDeadline(p.ctx, time.Now().Add(defaultPruningWindow)) | |
| ctx, cancel := context.WithTimeout(p.ctx, defaultPruningWindow) |
A little easier this way and achieves the same result
Comment on lines
+177
to
+180
| for { | ||
| select { | ||
| case <-ctx.Done(): | ||
| return numBatches, nil |
There was a problem hiding this comment.
Does this routine always way until the context has expired?
What happens when there is nothing to do? Seems like this routine should exit the loop early if there are no batches left to prune.
There was a problem hiding this comment.
Here I haven't mentioned the number of batches to prune. The idea is to prune as many batches in 3 seconds.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Bug fix
What does this PR do? Why is it needed?
This PR fixes a bug in pruner where it was taking a lot of time to prune a large database therefore blocking everything else because boltDB locks the database until pruning is completed.
Other notes for review
Solved the above by pruning in batches.
Acknowledgements