Sliding Sync: Get bump_stamp from new sliding sync tables because it's faster
#17658
Conversation
MadLittleMods
changed the title
bump_stamp for new sliding sync tablesbump_stamp from new sliding sync tables
| # `stream_ordering` should be assigned for persisted events | ||
| assert max_stream_ordering is not None | ||
| # Check if the event is a backfilled event (with a negative `stream_ordering`). | ||
| # If one event is backfilled, we assume this whole batch was backfilled. | ||
| if max_stream_ordering < 0: | ||
| # We only update the sliding sync tables for non-backfilled events. | ||
| return |
There was a problem hiding this comment.
This is unrelated to the change to using new tables or the bugfix.
Just re-arranging to be a little more sane. Should have no functional difference from before.
| # We can quickly query for the latest bump event in the room using the | ||
| # sliding sync tables. | ||
| latest_room_bump_stamp = await self.store.get_latest_bump_stamp_for_room( | ||
| room_id | ||
| ) |
There was a problem hiding this comment.
Here is the new functionality that this PR is introducing. We're fetching the bump_stamp from the new sliding sync tables which should be faster than flipping through the latest events in the room.
| # If we've just joined a remote room, then the last bump event may | ||
| # have been backfilled (and so have a negative stream ordering). | ||
| # These negative stream orderings can't sensibly be compared, so | ||
| # instead just leave it as `None` in the table and we will use their | ||
| # membership event position as the bump event position in the | ||
| # Sliding Sync API. | ||
| if new_bump_event_pos.stream > 0: | ||
| bump_stamp_to_fully_insert = new_bump_event_pos.stream |
There was a problem hiding this comment.
Had to add this bug fix here because now that we're using the new tables, the rest-layer tests would fail with the previous flawed logic storing the negative backfilled stream_ordering value (tests.rest.client.sliding_sync.test_rooms_meta.SlidingSyncRoomsMetaTestCase_new.test_rooms_bump_stamp_backfill).
I also added a new test, test_joined_room_bump_stamp_backfill, to do the same thing but just make sure our database tables are correct.
MadLittleMods
changed the title
bump_stamp from new sliding sync tablesbump_stamp from new sliding sync tables because it's faster
| # `SCHEMA_COMPAT_VERSION` and run the foreground update for | ||
| # `sliding_sync_joined_rooms`/`sliding_sync_membership_snapshots` | ||
| # (tracked by https://github.com/element-hq/synapse/issues/17623) | ||
| await self.store.have_finished_sliding_sync_background_jobs() |
There was a problem hiding this comment.
Don't we need to check this for the second clause too?
There was a problem hiding this comment.
We only need it for the first clause.
If the value is None, we can only trust it if the tables are reliable. If the background updates haven't finished yet, we can't tell None as a valid value from the row just not being there yet.
But if we have some value in the table, we can use it (second clause).
Instead of using simple_select_one_onecol, we could use simple_select_one and return an Optional[Tuple[int]] to be able to make this distinction. It's just a bit obtuse and we probably want it the way it's currently written after the gradual migration anyway with the few tweaks already called out.
| if room_membership_for_user_at_to_token.membership == Membership.JOIN: | ||
| # We can quickly query for the latest bump event in the room using the | ||
| # sliding sync tables. | ||
| latest_room_bump_stamp = await self.store.get_latest_bump_stamp_for_room( |
There was a problem hiding this comment.
As a separate PR maybe: we should also check in timeline for events that are bump events and use the stream ordering from that.
…ng-sync-bump-stamp-from-new-tables
erikjohnston
deleted the
madlitlemods/sliding-sync-bump-stamp-from-new-tables
branch
|
Thanks for the review and merge @erikjohnston 🐠 |
Get
bump_stampfrom new sliding sync tables which should be faster (performance) than flipping through the latest events in the room.Pull Request Checklist
EventStoretoEventWorkerStore.".code blocks.(run the linters)