Sliding Sync: Pre-populate room data for quick filtering/sorting #17512
Commits on Aug 1, 2024
Commits on Aug 5, 2024
Commits on Aug 6, 2024
Commits on Aug 8, 2024
-
-
-
-
-
-
-
Fill in
stream_ordering/bump_stampwhen we add current state to t……he joined rooms table
-
Commits on Aug 12, 2024
Commits on Aug 13, 2024
-
-
-
-
-
-
-
-
-
-
-
Fix running into
StopIterationMore context about how/why `StopIteration` was being ignored silently which made this problem harder to debug. See #17512 (comment)
-
-
Commits on Aug 15, 2024
Commits on Aug 16, 2024
-
-
Move background updates to
StateBackgroundUpdateStoreSo we can access `_get_state_groups_from_groups_txn(...)`
-
-
-
-
-
-
-
Commits on Aug 19, 2024
Commits on Aug 20, 2024
-
Prefer `_update_sliding_sync_tables_with_new_persisted_events_txn(...…
…)` to do the right thing See #17512 (comment)
-
-
-
-
-
-
-
Commits on Aug 21, 2024
-
-
-
-
Move
_calculate_sliding_sync_table_changes(...)after we assign `st……ream_ordering` to events See #17512 (comment)
-
-
-
Commits on Aug 22, 2024
Commits on Aug 23, 2024
-
Fix partial-stated room re-syncing state but nothing has changed
Fixes failing test in CI: `tests.handlers.test_federation.PartialJoinTestCase.test_failed_partial_join_is_clean` ``` 2024-08-22 18:57:22-0500 [-] synapse.metrics.background_process_metrics - 253 - ERROR - sync_partial_state_room-0 - Background process 'sync_partial_state_room' threw an exception Traceback (most recent call last): File "synapse/synapse/metrics/background_process_metrics.py", line 251, in run return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "synapse/synapse/handlers/federation.py", line 1842, in _sync_partial_state_room_wrapper await self._sync_partial_state_room( File "synapse/synapse/handlers/federation.py", line 1933, in _sync_partial_state_room await self.state_handler.update_current_state(room_id) File "synapse/synapse/state/__init__.py", line 554, in update_current_state await self._storage_controllers.persistence.update_current_state(room_id) File "synapse/synapse/storage/controllers/persist_events.py", line 491, in update_current_state await self._event_persist_queue.add_to_queue( File "synapse/synapse/storage/controllers/persist_events.py", line 245, in add_to_queue res = await make_deferred_yieldable(end_item.deferred.observe()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "synapse/synapse/storage/controllers/persist_events.py", line 288, in handle_queue_loop ret = await self._per_item_callback(room_id, item.task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "synapse/synapse/storage/controllers/persist_events.py", line 370, in _process_event_persist_queue_task await self._update_current_state(room_id, task) File "synapse/synapse/storage/controllers/persist_events.py", line 507, in _update_current_state await self.persist_events_store._calculate_sliding_sync_table_changes( File "synapse/synapse/storage/databases/main/events.py", line 624, in _calculate_sliding_sync_table_changes assert most_recent_event_pos_results, ( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: We should not be seeing `None` here because we are still in the room (!room:example.com) and it should at-least have a join membership event that's keeping us here. ```
-
Use
simple_upsert_txnforsliding_sync_membership_snapshotsin ba……ckground update
Commits on Aug 26, 2024
-
-
-
-
-
-
-
Adapt
sliding_sync_joined_roomsbackground update to use `event_str……eam_ordering` for progress This way we can re-use it for the catch-up background process
-
Commits on Aug 27, 2024
Commits on Aug 28, 2024
-
-
-
-
Use
stream_idof some point before we fetch the current stateThis is simpler and some rooms are so old that they don't have `current_state_delta_stream` yet. It's easier if we just get a general max `stream_id` of the whole table than the max `stream_id` for the specific room anyway. Thanks @erikjohnston
-
-
Add unique index right away for `sliding_sync_joined_rooms_to_recalcu…
…late` This makes it so we can always `upsert` to avoid duplicates otherwise I'm not sure of how to not insert duplicates in certain situations (see FIXME in the diff) which would cause problems down the line for the unique index being added later.
-
-
-
-
-
-
-
-
-
-
Fix join condition not working in Postgres
``` common column name "room_id" appears more than once in left table ```
-
Add
instance_nametosliding_sync_membership_snapshotsSo we can craft `PersistedEventPosition(...)`
Commits on Aug 29, 2024
-
Fully-insert
sliding_sync_joined_roomsrowsThis way if the row exists, we can rely on the information in it. And only use a fallback for rows that don't exist.
-
Merge branch 'develop' into madlittlemods/sliding-sync-pre-populate-r…
…oom-meta-data Conflicts: synapse/handlers/sliding_sync/__init__.py
-