Speech synthesis does not read the content of pop-ups

[CLnordcompo]

Speech synthesis does not read the content of pop-ups (notes and description texts placed in pop-ups).

Note_pop-up.mp4

[danielweck]

Thank you for reporting this problem. This is both a feature and a bug: it is a feature because our code explicitly excludes the information inside the popup footnote dialog during the TTS readloud (because it is considered UI, not publication contents), but obviously this is also a bug because the footnote contents should be announced after the user activates the hyperlink, and TTS readloud should resume at the calling site once the popup dialog is closed.

The current status is that this feature is quite simply not supported, we have not started implementing it yet. From the top of my head, I can see some obvious technical challenges but also some UX questions. For example Thorium offers a "simple view" for TTS utterances that displays on top of the publication's content, and which offers no interaction with footnote references at all (in other words, the user cannot even navigate to linked footnotes).

Final thought: one temporary solution could be to disable popup footnotes during TTS readloud, as this would cause hyperlinks to behave normally (i.e. jump to the note contents in the current document or in another "chapter") in which case the user would be able to resume playback at the location of the note reference by using Thorium's navigation history or with authored backlinks. Still, even in this case there is a UX problem with TTS readaloud not being able to identify the scope / end of the targeted note content (typically, speech utterances are streamed from beginning to end of document automatically unless the user interrupts the process)

[danielweck]

Related issue by @LaZay

#1923

This only nice to have.

So far, footnotes are not read aloud when TSS is activated. It would be nice to let people choose their own user preferences for TTS on this (as assistive technologies do for the blinds).

- epub:type=noteref, footnotes, footnote, endnotes, endnote
- role=doc-noteref, doc-footnote, doc-endnotes, doc-endnote

[ddfreiling]

+1 from Nota for reading footnotes aloud, perhaps with a skipability toggle.

[danielweck]

Related discussion: #2655

[danielweck]

For now, as a sensible workaround (which makes logical sense given how the popup viewport is implemented), during EPUB Media Overlays and TTS readaloud playback, the popup footnotes functionality is automatically disabled.

Unfortunately this does not suffice, the user must manually ensure to disable skippability otherwise note contents will be jumped over during playback. The solution to this UX problem would be for Thorium to detect when the user "explicitly requests" a skippable item, for example when following a note reference hyperlink, or even during playback when clicking/tapping on a skippable item (which Thorium could infer as the user's desire to play the item, even if skippable and skippability is enabled).

See code change:
fee5fda