feat(event cache): avoid storing useless prev-batch tokens #4427
+751
−150
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have a problem right now, with the event cache storage: as soon as you receive a previous-batch token, either from the sync or from a previous back-pagination, we consider that we may have more events to retrieve in the past. Later, after running back-paginations, we may realize those events are duplicated. But since a back-pagination will return another previous-batch token, until we hit the start of the timeline, then we're getting stuck in back-pagination mode, until we've reached the start of the timeline again.
That's bad, because it makes the event cache store basically useless: every time you restore a session, you may receive a previous-batch token from sync (even more so with sliding sync, which will give one previous-batch token when timeline_limit=1, then another one when the timeline limit expands to 20). And so you back-paginate back until the start of the history.
This series of commits fixes that, by introducing two new rules:
With these two rules, we're not storing useless previous-batch tokens anymore, and we'll back-paginate at most one, for a given room. Interestingly, this might uncover some bugs related to back-pagination orderings, so we'll desperately want #4408 <3
Part of #3280.