Accessibility (a11y) in diagrams

[weedySeaDragon]

There has been some great discussion and information about accessibility (a11y) in mermaid diagrams (flowcharts, specifically) in issue # 2395 . I'm starting this discussion to uncover the ideas and discussion.

Here are some of the ideas and questions raised. This is not exhaustive -- it's just a rough start.

• what roles should nodes, lines, connectors, and labels have?
• what aria roles should nodes, lines, connectors, and labels have?
• how to represent the relationships in the accessibility DOM?
• how to implement navigation in screen readers and the keyboard? How to navigate "forward" and "backward"?
  • what is the order for navigation?
  • what aria tags/properties to use for navigation? There is no currently defined aria standard ('aria-flow' is the closest, but is vague and barely supported)

What are links and references that will help explain the issues and potential solutions?

I'm willing to put together examples (I'll create branches and try to implement ideas, then put the resulting SVG images in this discussion).

[weedySeaDragon]

Gleaned from #2732

from @aardrian in Issue #2732

Some reference material for working on making the SVGs more accessible:

Tips for Creating Accessible SVG by Léonie Watson, 2014.
Accessible SVGs by Heather Migliorisi, 2016.
Accessible SVG flowcharts by Léonie Watson, 2018.
Creating Accessible SVGs by Carie Fisher, 2019.
Accessible SVGs: Inclusiveness Beyond Patterns by Carie Fisher, 2020.
Accessible SVGs: Perfect Patterns For Screen Reader Users by Carie Fisher, 2021.

---

also from @aardrian in Issue #2732, in a later comment on the example flowchart in the issue:

Despite all this, since many of the charts that can be generated do not have the relatively simple narrative flow of the one I added above, I still recommend an auto-generated plain text equivalent alongside the chart. Especially when the chart is small, the contrast is poor, or rendering breaks.

For example, the alternative text I wrote for the chart above:
The decision tree: Using a flow chart in GitHub → Must it be accessible? If no: Wrong, try again. If yes → Into what format does it render? If JPG/PNG/GIF: Does it have an alt? If no: Wrong, try again. If Yes, and it is lengthy: Wrong, try again. If yes: Manually add a plain text description of the chart. Back up to Into what format does it render? If SVG → What role does it have? If none: Wrong, try again. If table: Wrong, try again. If grid: Wrong, try again. If presentation: Manually add a plain text description of the chart. If image → Does it have an aria-label? If no: Wrong, try again. If Yes, and it is lengthy: Wrong, try again. If yes: Manually add a plain text description of the chart. Back up to What role does it have? If tree → Does it contain any treeitem roles for the text nodes? If no: Wrong, try again. If yes → Are they contained in / owned by group roles beyond the first level? If no: Wrong, try again. If yes → Have you tested in any screen reader? Have you tested in any screen reader? If no: Wrong, try again. If yes → Did it accurately convey the relationships and structure of the chart? If no: Wrong, try again. If yes → You should write a CSUN talk about it.

As you can see, that is a bit verbose for a plain text description and lacks the nuance and structure you could represent with things like lists.

[brennanyoung]

Core Concept: Semantics and ARIA roles

Semantics are the primary way we humans make sense of things. A semantic role manages human expectations about web content, even in cases where sensory cues are arbitrary or unavailable. (If a greeting is called-for, then even a grunt or a wave will serve the purpose).

You may know that bumping up the font size in CSS does not make a paragraph into a heading. Yes, it might look like a heading (the larger font is a sensory cue), but semantically it will remain a paragraph. Accessible content should aim to get the semantics -and nesting- right and then there is a good chance that many of the finer-grained details presented visually will make sense by themselves.

HTML5 provides implicit semantics for most of its elements, such as headings, lists or nav, and these should certainly be used when available. For other host languages (especially including SVG) then ARIA roles may be used.

Explicit ARIA roles vs. implicit semantics from the host language.

If the host language provides a semantic implicitly (for example <a> has the semantic role "link" by default in both SVG and HTML) then it is not necessary to provide the same semantic using the role attribute, unless you intend to change that semantic.

For this reason, in general, role attributes are most likely to appear on generic elements such as <div> or <span>, but sometimes a non-generic role will be overridden. Here are some examples:

• <a role="link" href="foo.html"> - Avoid this. Superfluous role will raise a warning in automated a11y audit.
• <a role="button"> - link becomes button, a subtle distinction, and possibly a 'code smell' (why not just <button>?)
• <div role="spinbutton"> - providing a widget semantic to a generic element, common practice if the semantic is not available in the host language
• <text role="heading" aria-level="4"> - Using a visible SVG text as a semantic heading.

Roles and the Accessibility Tree

Assistive tech does not process the DOM directly. Instead, the browser generates an "accessibility tree", which is a subset of the DOM, excluding any 'generic' structures or presentational content. The accessibility tree is as close as we get to an API for assistive technology and roles play a key role in its construction. The nodes of the accessibility tree may be inspected in modern browsers' built-in Developer Tools, where accessible names and ARIA roles and values are typically found. Implementations of the accessibility tree view in the dev tools differ quite a lot by browser. The Firefox one is perhaps the easiest to understand at time of writing.

There are some bleeding-edge ways to inspect and manipulate the accessibility tree using javaScript if you are looking for high adventure.

Generic, Presentational and hidden content

• role="generic" is the default semantic role for anything that does not have any specific meaning. This may be <div> or <span> for example, often provided as hooks for CSS, or just to make the code more manageable.
  Elements with a generic role do not permit naming, and are not included in the accessibility tree. Authors should not set this role, but may override it with something more semantically meaningful that does support naming, e.g. <div role="navigation" aria-label="quick links">.
• role="presentation" (synonym role="none") indicates content that is "purely presentational" - i.e. content which is concerned only with layout or appearance, often "purely decorative" or "visual affordances" (such as background images or tick marks along an axis). If these are allowed to appear in the accessibility tree, the AT experience can be very "chatty" and distracting, so role="presentation" can clean things up. This role also masks the semantics (but not the text nodes) of any 'child' elements, so that they become (effectively) generic.
• hidden is a boolean HTML5 attribute (always true if present, even if it has a value of "false") which hides content (and any subtree) from view and from the accessibility tree. Note that while CSS is generally regarded as "purely presentational", the rules display:none and visibility:hidden will nonetheless remove content from the accessibility tree.
• aria-hidden="true" is an ARIA attribute which removes content (and any subtree) from the accessibility tree, but does not hide it from view. This can be useful to reduce screen reader chatter.

W3C has documented how specific SVG element types may be included in or excluded from the accessibility tree. Spoiler: In general, they are excluded from the tree by default, and only reliably appear if appropriate ARIA attributes have been added.

Roles intended for content "in which the visual appearance or layout of content conveys meaning"

For SVG content (such as mermaid charts), there is a vey important implicit semantic ("graphics-document"), but explicit semantic roles (e.g. for individual datapoints) may be provided. <text> also has an implicit semantic of "group".

Here are the three roles offered by graphics ARIA.

• role="graphics-document" - the default/implicit role of any SVG element. According to spec, there is no need to add it, but not all browsers follow the spec.
• role="graphics-object" - for parts of a chart that may have 'child' parts (groups and subgroups)
• role="graphics-symbol" - for parts of a chart that do not have 'child' parts

These roles do a decent job of structuring content, but they give very little guidance about what is really meant. It may be valuable to provide further cues about the kind of content in the chart. (e.g. is it a pie chart, or a sequence diagram or something else?)

By adding aria-roledescription you can guide user expectations about the chart without writing a lot of extra descriptive copy. Warnings about misusing aria-roledescription are common, but the graphics ARIA spec clearly recommends using this attribute,

Here is a small example:

<svg aria-roledescription="pie chart">
<title>Unique visits by device</title>
<g role="graphics-object" aria-roledescription="segment" aria-label="iPhone 38%">...

Some work is afoot to add appropriate aria-roledescription strings to mermaid for all the standard chart types. This will be a significant improvement. (Note: internationalisation will be relevant here).

In addition, all the other ARIA roles are available within SVG if necessary.

<text> and <foreignObject>

SVG <text> elements will appear in the accessibility tree by default (implicit role="group"), but <text> does a pretty poor job of reflowing text. <foreignObject> is available, permitting HTML markup to be inserted into SVG code, with all the reflow and styling you may want. But <foreignObject> has a default role of "presentation". You'll need to override that with another role if you want to include <foreignObject> content in the accessibility tree.

Headings

Some roles are considered so important that assistive tech provides special affordances (e.g. keyboard shortcuts) for getting to them. These are called "landmark roles". Most of these are out of scope for this discussion, but headings are worth mentioning.

The most important of all is the main heading, available in HTML as <h1> or in aria as role="heading" with aria-level="1". A main heading indicates clearly what is the main topic, purpose or activity on the current page or document. Mermaid is probably off-the-hook for this, except perhaps in the popup dialog views.

The legendary "outline algorithm" which was supposed to be central to HTML5 is not implemented anywhere outside the lab and the effort has been abandoned. In theory, subsequent heading levels should increase by one as the content tree becomes more fine-grained. In practice it is acceptable to skip a level, if unavoidable, but do not distort the tree by putting (e.g.) a H1 inside a section that begins with H2.

It is considered acceptable to begin a <section> or <dialog> with a new <H1>, which is sometimes a lot easier than inspecting the DOM context to discover the appropriate heading level in relation to the document as a whole. Similar liberties may be taken by the opaque subtrees of HTML components.

A "graphics-document" is a subclass of "document", so it within spec for the first heading in a mermaid view to be level 1.

Or else if your content is part of a larger structure, you can just pick up after the most recent heading level, whatever that may be.

The important thing is to be consistent. If you implement axis labels as headings, then your X-axis should have a heading with the same level as your Y-axis, because they are at the same level of granularity.

Note that the SVG <title> element does not appear as a distinct heading node in the accessibility tree. Instead, its child text nodes are used as the accessible name for the SVG graphics-document that does appear in the tree. Adding role="heading" to the <title> is likely to produce duplicate announcements from ATs.

[weedySeaDragon]

Also adding this comment I made in #2395 :
(@brennanyoung posted a reply, but I'll let them copy/paste it in this discussion, or not)

Given the example flowchart, how could we show the possible sequence?
It seems that aria-flow is intended for this purpose but is not well supported. (Is that correct?)

Could aria-owns be used?

In a simple example: How should the sequence for a choice and the options be represented? ex:

flowchart TD
    B --> C{Let me think}
    C -->|One| D[Laptop]
    C -->|Two| E[iPhone]
    C -->|Three| F[fa:fa-car Car]

Loading

I think that if we can come up with some ideas for aria labels and sequencing (order), I can implement some ideas so we can see how they play out.

[brennanyoung]

Regarding aria-flowto and aria-owns, I'll direct interested folks to my reply on #2395

But I would like to urge mermaid users to think very carefully about whether we'll be better off replacing aria-flowto which is part of ARIA today, and, while flawed, promises to do exactly what we need here, (except that there is only one half-assed implementation in the wild) with aria-details (draft spec here), a much more general purpose attribute still in discussion, with no finished implementations and questionable applicability as a connector mechanism for charting elements that are at the same level of granularity.

I trust that my own sympathies are clear from the above paragraph. We'll lose aria-flowto unless we make a case that tightening and clarifying its spec, not deprecation is what needs to happen.

I've made my arguments on the ARIA thread. Please contribute with your take, if this is important to you.

[brennanyoung]

Still anxious about the prospect of aria-flowto getting deprecated, I created a counter proposal on the ARIA board.

[weedySeaDragon]

Saw your counter proposal.
I will do my best to post on the ARIA thread # 630. (Depends on my body cooperating. [rolls eyes].
In short (in case I can't post): we finally have the ability to provide a much truer experience with flow charts and other diagrams: the coding tools and the browser support are now such that we can provide the information within the diagram itself that affords the same flow -- the same navigation experience. No longer have to hope that the diagram authors will spend the time to write out a text version (which affords no navigation at all and thus is a poor substitute). But it depends on aria-flow. ...

[brennanyoung]

Some news: The ARIA guys have agreed not to deprecate aria-flowto, with the caveat that it is not (yet) fit for purpose. In the long term, a deprecation may occur, but not until we have a viable replacement (which might be a souped-up aria-flowto). I will write more about this soon.

[brennanyoung]

A very interesting new document was just published. Essential reading for all working in the accessible chart space.

https://www.urban.org/sites/default/files/2022-12/Do%20No%20Harm%20Guide%20Centering%20Accessibility%20in%20Data%20Visualization.pdf