Expected UTDs: Hide device-relative historical UTDs

[BillCarsonFr]

A freshly logged in session will have access to event in the joined rooms, but not to the keys for the messages as the device didn't exist when the event were sent.
The new device has to go through the verification process in order to get access to the keys and properly display history.

We are talking here about Device-Relative history:

Currently EIX is hiding such history until the device has access to backup:

Technical Inputs

How to detect when an event is historical to the current device?

Currently EIX is using events origin_server_ts of events (isItemInEncryptionHistory). It's comparing the local creation date of the device ( since the unix epoch) to the event origin_server_ts (Timestamp in milliseconds since the unix epoch on originating homeserver when this event was sent.)

Some downside on this approach is that if the device local time or server local time is wrong it could messed up things.

Another approach could be to use the API end-point used to get the messages?

• From sync: If not an initial sync it's not an historical message.
• From sync: If an initial sync, could be a mix of it, but mainly historical? (unless a clear cache is done)
• From /message (back pagination). Would be mostly historical to the current device (unless there was a clear cache?)
• From /relation (threads?). Would be mostly historical?

It's anyhow hard to see if something is really device relative history, as the process of sending is inherently racy.
There might be some events that are shown were they shouldn't (sent around the same time the device is logged in but before propagated to anyone)

Proposal

• Hide historical messages (expected UTD) until the user has access to backup
• Hide historical messages (expected UTD) while keys are beeing downoaled.
• Once the keys have been downloaded, show the events.

If an UTD has not be displayed on screen, it should not be reported to analytics.

Tasks

Tasks

Preview Give feedback

1. EX Hide device relative historical messages until the device has access to backup.
2. WebR: Hide device relative historical messages until the backup has been downloaded.
3. WebR: make sure that UTDs due to this are reported separately in Posthog

Known possible issues

• If we hide the device relative historical messages, and the users manually import keys from a file, what should we do?
• After the backup import, some messages might still be in UTDs (key was not in the used backup). That's fine, we should just display the UTD and report to analytics

[richvdh]

Probably requires a fix to element-hq/element-web#27009?

Edit: not really. The two are orthogonal.

[richvdh]

There might be some events that are shown were they shouldn't (sent around the same time the device is logged in but before propagated to anyone)

This is somewhat related to #2314, and dealing with the race condition might have a similar solution.

[richvdh]

As a first step, we will not hide the events altogether, but display a placeholder.

[BillCarsonFr]

Note: we should ensure that it's also hiding historical UTDs in thread view.

[richvdh]

Rough idea for approach here:

• Add a new code value to DecryptionError which reflects a message that is older than the device, when we don't have backup enabled.
• Update DecryptionFailureBody to display the failure differently to the user
• Update DecryptionFailureTracker to report the failure differently to posthog.

More detailed work:

• Move DecryptionError out of legacy crypto code and into crypto-api. Replace its code with an enum: Clean up code for handling decryption failures matrix-org/matrix-js-sdk#4126
• Refactor EventDecryptor.attemptEventDecryption so that all the logic isn't inside a catch block. Refactoring and simplification in decryption error handling matrix-org/matrix-js-sdk#4138
• Expose a new method PerSessionKeyBackupDownloader.isBackupConfigured().
• When we get a MissingRoomKey or UnknownMessageIndex, compare the timestamp on the event with the creation time of our own device (available via Device.firstTimeSeen), and check isBackupConfigured. If it's an old event, and backup is not configured, use different codes on DecryptionError. Crypto: use a new error code for UTDs from device-relative historical events matrix-org/matrix-js-sdk#4139
• DecryptionFailureBody: use different text for new error codes. Expected UTDs: use a different message for UTDs sent before login matrix-org/matrix-react-sdk#12391
• DecryptionFailureTracker: use different error code. Expected UTDs: report a different Posthog code matrix-org/matrix-react-sdk#12389
• Playwright tests (some may exist and just need updates). Expected UTDs: use a different message for UTDs sent before login matrix-org/matrix-react-sdk#12391
  • Old event, with backup disabled.
  • Enabling backup changes the UI for the event even if the key isn't in the backup.

[BillCarsonFr]

Some design exploration https://www.figma.com/file/a1NWmg6mOi2VZuSgaJx6nD/Displaying-UISI?type=design&node-id=1-12313&mode=design&t=6hb0XIH9Ftv5SI46-0

[richvdh]

Decision tree for the text to be displayed:

flowchart TD
A{Timestamp on the message postdates the creation of our device}
A -- Yes --> X
A -- No --> B
B{Is there a backup}
B -- No --> Y
B -- Yes --> C
C{Is the device verified}
C -- No --> Z
C -- Yes --> X

X([Normal UTD error])
Y([History is not available on this device])
Z([You need to verify this device])

Loading

[andybalaam]

Discussion today suggests the full flow diagram implemented looks like this:

flowchart TD
A{Is the message newer than the device?}
A -- No --> B
A -- Yes --> X
B{Is there a backup on the server?}
B -- No --> Y
B -- Yes --> C
C{Is backup working on this device?}
C -- No --> D
C -- Yes --> X
D{Is this device verified?}
D -- No --> Z
D -- Yes --> X

X([Normal UTD error])
Y([History is not available on this device])
Z([You need to verify this device])

Loading

In matrix-js-sdk most of this is implemented when we choose a DecryptionFailureCode and some ("Is this device verified?") when we display a message to the user.

I am looking at this as I am implementing similar logic in matrix-rust-sdk, probably in UtdCause::determine.

[andybalaam]

(I am working on #2638 )