Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal for various cross platform a11y improvements. #56

Closed
wants to merge 1 commit into from
Closed

Proposal for various cross platform a11y improvements. #56

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 133 additions & 0 deletions proposals/0000-a11y-cross-platform.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
title: Cross-platform Accessibility Improvements
author:
- Marc Mulcahy
date: 2018-10-31
---

# RFC0000: Cross-platform Accessibility Improvements

## Summary

Several accessibility-related properties are only available on one platform (either iOS or Android, but not both). This proposal attempts to make as many properties available on both Android and iOS as possible.

Properties only available on iOS include:

* accessibilityViewIsModal
* accessibilityElementsHidden
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is similar to Android importantForAccessibility prop. It would be good to consolidate these together and learn from how web handles these cases

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps something like ARIA hiden would be adequate. Android's importantForAccessibility is a bad model to emulate-- it includes yes, no, and auto as values. Auto is the default I believe, and from the developer's point of view, is hard to use, since the rules for when something is automatically marked as unimportant for accessibility aren't clear. I think a better model is to assume everything is important for accessibility, unless explicitly marked otherwise. So it seems like some notion of hidden is adequate.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should make accessibilityElementsHidden cross-platform? Mapping to aria-hidden on web and equivalent on Android?


Properties only available on Android include:

* accessibilityLiveRegion
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also available on web

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could standardize on ARIA's notion of a live region, and then dumb down the implementation to the various platforms. Android's notion of a live region is pretty basic (polite or assertive). iOS has no notion of live regions as far as I know, so we'd have to use iOS primitives to implement a subset of live region functionality.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accessibilityLiveRegion is essentially already identical to ARIA live region, there's just a different in what the none state is called. So you could extend support for the existing API to iOS and then we'll have the 3 main platforms covered.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on the accessibilityLiveRegion property being more universal. We just implemented this for Windows (see PR here).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for accessibilityLiveRegion on iOS


Several properties are missing from both platforms:

* accessibilityCompoundComponent: replace accessible-- indicates to assistive technologies that all the components inside this one should be treated as a single component.
* accessibilityRelationships: describes how this component is related to others
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a proposal internally that is based on Web's ARIA properties: accessibilityLabelID (equivalent to aria-labelledby), accessibilityDescriptionID (equivalent to aria-describedby), and accessibilityFocusID (equivalent to aria-activedescendent). All these props would take the value of the nativeID of the element they are associated with.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. From naming point of view, are you in favor of using the ARIA namespace, or creating our own properties based on ARIA?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "ARIA namespace"? Something like accessibilityLabelledBy vs accessibilityLabelID?

* accessibilityRange: expose information about range-based components such as sliders
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can probably learn from web here and expose the range props to the underlying accessibility layer of the platform. Or model this off ARIA props for progressbar etc

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've found that in practice, exposing range information in a consumable way is fairly difficult. Exposing a minimum, current, and maximum value allow you to speak a percentage, which is fine for some types of progress bars. But for others, where the actual value is relevant (say a color contrast setting, stereo balance control, movie timeline slider, etc.) it would be nice to include some information that could cause the screen reader to speak either a string or the current value instead of a percentage. Slider's also imply programatic actions to adjust them.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But for others, where the actual value is relevant (say a color contrast setting, stereo balance control, movie timeline slider, etc.) it would be nice to include some information that could cause the screen reader to speak either a string or the current value instead of a percentage.

ARIA has a property for that aria-valuetext - https://www.w3.org/TR/wai-aria-practices/examples/slider/slider-2.html

I do like how understandable and extensible the accessibilityRelationships and accessibilityRange props names are vs having many more individual props ARIA.

Are you thinking it would look something like this (property names tbh for relationships)?

<View
  accessibilityRange={{ min, max, current, text }}
  accessibilityRelationships={{ labelID, descriptionID, focusID }}
/>


## Motivation

All accessibility properties should be available on all supported platforms. This reduces the burden on developers to understand the nuances of accessibility support on various platforms. It also allows JavaScript components to expose rich accessibility information, rather than relying on the accessibility support in native components.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


## Detailed design

### accessibilityViewIsModal

#### Android/Fire OS

When a component marks itself as modal, code in the view manager will need to traverse all other components in the view tree and mark them as importantForAccessibility=no. A window content changed event with content change types set to subtree will then need to be fired on the root window to ensure that assistive technologies receive the tree updates.

It may also be necessary to install a view tree observer once the modal component is active to prevent background components from appearing. We'll need to do some real-world testing to understand how realistic this scenario might be, and whether it's worth the performance cost of managing such an observer.

#### iOS

This property is currently supported on iOS, so no change to iOS support is needed.

### accessibilityElementsHidden

#### Android/Fire OS

Supporting this property in Android may be as simple as marking the component as importantForAccessibility=noHideDescendants

If this doesn't match the iOS behavior, we may need to build an accessibility delegate to hide the children. This will likely be tricky, since the children will still be generating events.

#### iOS

This property is already supported on iOS, so no iOS changes are required.

### accessibilityLiveRegion

This allows components to tell assistive technologies how to react to changes in their content.

#### Android/Fire OS

This property is already supported on Android, so no changes in Android support are required.

#### iOS

We believe this functionality can be implemented using accessibility notifications on iOS. For assertive live regions, the updated text of the component can be sent immediately to iOS as an announcement notification. For polite live regions, code can wait for the AnnouncementDidFinish notification before posting the updated text as an announcement notification.

### accessibilityCompoundComponent

We think the current implementation, which is called "accessible", should be renamed to accessibilityCompoundComponent, to more accurately represent what it does. The functionality should remain the same.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The behaviour of accessible has always felt a bit ambiguous to me, as on Android and Web it's equivalent to focusable. I don't think accessibilityCompoundComponent is any clearer a name than accessible

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the whole notion of compound component handling needs quite a bit more thought. My main opposition to calling this accessible is that it's misleading-- it has little to do with whether the component is accessible, and almost everything to do with how assistive technologies should treat the component.

Maybe I should drop this from this proposal, and make a separate proposal covering compound component handling? I'm thinking as a start about containers with multiple children, which we want to treat as a single item, but depending on the container layout, perhaps only one child is clickable and should be manipulated when the screen reader double taps, etc.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I should drop this from this proposal, and make a separate proposal covering compound component handling?

I think that sounds good. The idea of an accessibility group is something that's a little complicated on web too, because you have both screen-reader accessibility and keyboard accessibility.


### accessibilityRelationships
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this API look like in practice? Is the value an object?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're referring to compound component handling, I think the answer is yes. perhaps a separate proposal would let us discuss and flesh out exactly how to do this.

Relationships could be an object, or each relationship could be a separate property. We've found it sometimes useful for a relationship to include multiple targets-- for example, cases where multiple components describe a single component.


This property describes how this component is related to others. Relationship types include:

* labelFor: This component labels the specified component
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a use-case you had in mind for having the relationship be defined in either direction like this?

* labeledBy: this component is labeled by the specified component
* describedBy: this component is described by the specified components.

When referring to other components, we may need to add an additional property to view-- namely accessibilityId. We need to find the best forward-looking approach for referencing other components.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have nativeID that we can use for this

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, when was this nativeID property introduced? Trying to understand backward compatibility.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it was introduced for both platforms here in June 2017 facebook/react-native@70e0455


#### Android/Fire OS

For labelFor and labeledBy, use the standard Android view and AccessibilityNodeInfo methods.

For describedBy, which is only supported on Fire OS, put the appropriate key/values in the extras bundle.

#### iOS

When associating labels with components, include the label text in the target component's accessibilityLabel.

Don't support describedBy on iOS for now. In future, we could explore adding the accessibilityLabels of the describedBy targets to the accessibilityHint of the component.

### accessibilityRange

This property allows components (such as sliders) to expose range-based information. The value of the property is an object containing the following values:

* min: the minimum value of the range
* max: the maximum value of the range
* current: the current value of the component (within the range specified by min and max)

#### Android/Fire OS

This maps directly to Android's RangeInfo interface.

#### iOS

iOS supports a property called accessibilityValue. This value is a simple string. accessibilityRange will be converted to a localized string and placed into accessibilityValue.

## Drawbacks

It will be challenging to ensure that the behavior of these properties is the same between iOS and Android. Building cross-platform properties will likely involve sacrificing a small amount of functionality on each target platform to ensure consistency.

## Alternatives

Components could continue to use the platform-specific properties. This has the drawback of requiring developers to have at least a cursory
understanding of accessibility on both iOS and Android. And in the case of accessibilityViewIsModal, we expect this to be somewhat difficult to implement correctly on Android.

## Adoption strategy

We don't believe this to be a breaking change. Developers who currently use these properties on Android or iOS can continue to do so, and remove conditional use to allow functionality on other platforms.

## How we teach this

The same documentation that is used to describe the properties on either iOS or Android can be marked as cross-platform for use on either. For the accessibilityRelationships property, labeledBy and labelFor are common paradigms across accessibility support on multiple platforms including web.

## Unresolved questions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a related question (and issue) about content that is hidden visually but visible to screen readers. Typically it will be hidden headings so they show up in the screen reader headings list (e.g., mobile.twitter.com does this) and it is visually hidden with a particular CSS incantation. Would you / how would you do something like this in native apps?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something like this?

import React, { useEffect, useState } from 'react';
import { AccessibilityInfo, Text } from 'react-native';

const useIsScreenReaderEnabled = () => {
  const [isScreenReaderEnabled, setIsScreenReaderEnabled] = useState(false);

  useEffect(() => {
    AccessibilityInfo.addEventListener('screenReaderChanged', setIsScreenReaderEnabled);
    AccessibilityInfo.isScreenReaderEnabled().then(setIsScreenReaderEnabled);
    return () => AccessibilityInfo.removeEventListener('screenReaderChanged', setIsScreenReaderEnabled);
  }, []);

  return isScreenReaderEnabled;
};

const Example = () => {
  const isScreenReaderEnabled = useIsScreenReaderEnabled();
  return <Text>I {isScreenReaderEnabled ? 'love' : '❤️'} React Native for Web</Text>;
};


* Exactly what is the behavior for accessibilityElementsHidden on iOS? Does this hide the component itself from accessibility, or only its children?
* What is the best mechanism for one component to refer to another for the purpose of specifying labels, targets, descriptive components, etc.?