This repository has been archived by the owner on Apr 26, 2024. It is now read-only.
Don't waste time processing backfill events that we never show to the client #15632
Labels
A-Messages-Endpoint
/messages client API endpoint (`RoomMessageListRestServlet`) (which also triggers /backfill)
T-Enhancement
New features, changes in functionality, improvements in performance, or user-facing enhancements.
Milestone
As discovered while working on #15617 and a natural extension of #15585 to process previously failed backfill events in the background (follow-up to #13623),
It turns out, when we make a
/messages
request and trigger a backfill, Synapse will happily waste time trying hard to process events (_process_pulled_events
) in the foreground that we end up never showing to clients. The one particularly bad culprit is theorg.matrix.dummy_event
which Synapse automatically puts in the DAG to resolve outstanding forward extremities. Since it has 10x randomprev_events
, it takes a while to fetch the state for eachprev_event
and persist. We determine whether some events should be shown to clients in the_check_filter_send_to_client(...)
functionsynapse/synapse/visibility.py
Lines 482 to 519 in ad50510
org.matrix.dummy_event
can be particularly bad because they usually include a lot of disparateprev_events
which take a while for us to work out the state for (long_get_state_groups_from_groups
times).Potential solution
We should use this same
_check_filter_send_to_client(...)
criteria when determining whether we should backfill a given event in the background or foreground.We should be mindful an event that is filtered from the client, might still be used as a
prev_event
from another event in the backfill response that we will show to clients. And if we kick off the function to processorg.matrix.dummy_event
in the background, while the other event in the foreground, we will probably end up duplicating the work depending on the race.We don't really have these same race -> duplicate work concerns with #15585 (events which previously failed to backfill) because it's a "fool me once" sort of situation and they are already potentially disconnected from what we were trying to backfill anyway given they failed before.
Also of interest,
filter_events_for_server(...)
The text was updated successfully, but these errors were encountered: