accesskit has no effect unless screen readers are enabled

[drewcrawford]

For the dedicated and often overburdened OSS maintainers: this is actually a check-in to gauge whether it would be helpful and appropriate to contribute by sending PRs, or if it's more expedient to deal with things on my own. I truly appreciate the time and effort you put into maintaining this project and for anyone who tries to improve AX generally, and ideally everything I would say would be interpreted in that light.

I am evaluating accesskit's viability for application and application-related frameworks I have in the works and everything I can say comes from an application-developer's POV.

My basic strategy to get to know the crate was to add deps to a representative project, to call accesskit_unix::Adapter::new(...) and pass it todo!()-based implementation of the required traits. Then to perform panic-oriented-programming to better understand the crate and to see how I can implement core behaviors etc.

Unfortunately calling new in this way did not trigger my panics or have seemingly any effect, and I got stuck here for a long while. I spent a fair bit of time trying to understand what "else" I needed to do to get the accesskit family of crates to "do" anything (I read and tried all of the 4 examples on GitHub that use this API, I read a lot of API docs to try to find out how who I should call to 'activate' my adapter, I looked into drop issues, I set breakpoints inside the accesskit family of crates to step through them interactively, etc.) to eventually discover my issue is #324 and #314 and I need to run my program with my screenreader enabled. So in that sense I know the workaround and so my problem is "solved".

In another sense this experience raised some practical questions about if accesskit is a good fit for my usecase:

1. This behavior is very hard to debug for new users. There's a) no mention of this behavior in the docs for accesskit_unix::Adapter::new(...), b) no print at runtime when this happens, and c) the names of properties I saw deep in my debugger (IsEnabled and ScreenReaderEnabled) seem to be dead ends for searching for answers in the codebase itself, comments, etc.
2. The detection of whether accesskit is needed: it's wrong in practice. As a sighted developer, I mostly develop with my screenreader off and using a tool like Accerciser to check my tree. Unfortunately, whatever Accerciser does, doesn't enable the accesskit_unix adapter. I'm not familiar enough with accessibility stuff on linux to understand "whose bug" that is, but my outsider perspective is that if accesskit apps can't be debugged with the standard tool from the standard desktop environment, like other apps in the desktop environment, it feels like a bug in accesskit. (I have not tried KDE's tool but I wonder if it is similar?)
3. Obviously there are access technologies beyond screen readers or accessibility tree inspectors but I don't know linux well and whether they are affected by this behavior. My googling does turn up that folks much more knowledgeable than myself wonder if they could be affected.
4. While I understand the advantage of disabling unneeded stuff at runtime, I also understand this places the burden of performance issues disproportionately onto disabled users. In the backing material an application developer expressed that "the performance issue we're seeing is severe enough that also those relying on the accessibility feature would be bothered by this". I suppose I foresee myself being in their shoes trying to solve similar issues.

Here's a list of possible PRs I could contribute on these topics that maintainers could easily approve:

1. docfix, per 1a
2. print fix, per 1b (do we call log or eprintln?)
3. code comments, per 1c
4. Force enable unix adapter for development builds, i.e. when #[cfg(debug_assertions)]
5. Force enable unix adapter by default in release builds, ship escape hatch via env such as NO_ACCESSKIT=1 for those affected by problems with this
6. API for application developers like myself to programmatically configure whether accesskit is enabled on some basis sensible to ourselves

Some unrelated longer-term priorities I would have, and I'd like to understand if my PRs can land here, would be:

• Support for alternative windowing systems that are in my development tree
• Support for the canvas element (purportedly 'planned' but I don't see active PRs or public work in this area)
• Advancing the schema as-needed for my custom control(s) as supported by various platform(s) APIs and/or mainline screen readers
• Support for other platforms I need, primarily the Apple platforms but possibly various proprietary platforms

[mwcampbell]

Hi @drewcrawford! Thanks for the thoughtful message!

We certainly want to eliminate as many stumbling blocks as possible for developers who are trying to make their software accessible. I wasn't aware that the current behavior of the Unix adapter was a problem for developers who don't use a screen reader or other assistive technology but are trying to test their work using Accerciser. For that reason alone, I think it's worthwhile for us to revisit that behavior, regardless of whose bug is giving you trouble.

Unfortunately, AT-SPI doesn't allow as clean a solution as the Windows and macOS APIs do. On Windows and macOS, we can enable lazy initialization with practically zero overhead simply by responding to a particular window message or Objective-C selectors. Crucially, because those platforms themselves raise focus events that cause accessibility clients to send those initial requests, we don't have to proactively raise focus events ourselves. If I remember correctly from my work on GTK last year, this is what prevents GTK 4 from lazily initializing its accessibility subsystem as AccessKit tries to do. So, I don't think we can arrive at a win-win solution by doing something simple, such as proactively registering the root object with the AT-SPI bus regardless of the state of the IsEnabled property, and lazily requesting a full tree from the application only if we actually get a request on the root object. Maybe we could make that work if we raise a focus event on the root object, in place of whatever descendant actually got focus, and then raise another focus event once the full tree is actually initialized in response to an AT querying the root object. I don't know if that would result in excess verbiage for screen reader users, though. We'd have to try it to find out.

I want to respond more directly to this point:

While I understand the advantage of disabling unneeded stuff at runtime, I also understand this places the burden of performance issues disproportionately onto disabled users.

That's a valid concern. Perhaps I'm showing my jadedness here, as a (partially) blind person, a long-time assistive technology developer, and a former member of an accessibility team at a mainstream company (I was on the Windows accessibility team at Microsoft for ~3.5 years). But it seems to me that for accessibility solutions to be accepted and implemented by the non-disabled mainstream, it's important to minimize the impact of those solutions on that group. So, I worry that if a non-disabled developer notices any performance impact caused by AccessKit when running their applications without any assistive technology, they'll be less likely to use AccessKit, and by extension, possibly less likely to make their thing accessible at all. I figured that if we can get more developers to make their stuff accessible at all, even if disabled users then disproportionately bear the burden of performance issues, that's a net win. Of course, in a more fair world, there wouldn't be performance issues that disproportionately affect disabled users, and non-disabled developers wouldn't be put off by being immediately exposed to performance issues that come with implementing accessibility. Perhaps I overestimate the number of developers who would have such a strong reaction to being immediately confronted with such issues. And of course it's true that, since performance issues like the one you linked will ultimately affect assistive technology users, they really ought to be addressed. So, I don't think there's a simple correct answer to this. And again, maybe I'm carrying a chip on my shoulder and allowed that to cloud my judgment.

Since @DataTriny is the primary developer of the Unix adapter, I'll defer to him on specific solutions. But I would certainly welcome a PR that documents the current behavior.

Thanks again for taking the time to share your thoughts!

[DataTriny]

Hello @drewcrawford, indeed, thank you for this nice message.

Your understanding of the situation is correct. Your code searches just didn't return anything because we use the zbus crate, which wraps D-Bus objects inside Rust types. The monitoring of the IsEnabled property is here.

We obviously don't expect UI toolkit developers to perform most of their tests with a screen reader on, and Accerciser is the preferred way of exploring the accessibility tree exposed by applications on the Linux desktop. Unfortunately I don't use this tool myself very often (always with a screen reader on anyway) and I didn't even realize that I broke AccessKit for this use case. I think that what happens is that Accerciser doesn't set the IsEnabled D-Bus property on startup, like Orca does here. I have searched Accerciser's repository for issues or merge requests mentioning this but found nothing, so I wonder if noone simply offered to implement it. I know that other AT-SPI implementations read environment variables to force initialization, but I'd rather avoid doing this if there is no technical roadblocks for Accerciser to emit this signal.

While we figure this out, you can run this command after opening Accerciser to avoid running a screen reader:

busctl --user set-property org.a11y.Bus /org/a11y/bus org.a11y.Status IsEnabled b true

This will tell AccessKit that accessibility is enabled on the system and hopefully Accerciser will be able to fetch the tree.

[drewcrawford]

Thanks so much for getting back to me so quickly, especially with workarounds!

I worry that if a non-disabled developer notices any performance impact caused by AccessKit when running their applications without any assistive technology, they'll be less likely to use AccessKit... I figured that if we can get more developers to make their stuff accessible at all, even if disabled users then disproportionately bear the burden of performance issues, that's a net win.

I understand this line of thinking, and I can't speak to what is the most viable strategy in the Microsoft orbit at all. But I also think different kinds of developers will have different kinds of concerns.

My main concern is "if I break my software, will I find out?" Ultimately the more configurations I have, the less likely that is. Of particular concern and at large risk are things that I need to support but don't use personally. I'm also uncertain what the best practice is to find out if accessible technologies are contributing to e.g. a performance issue someone files on GitHub. For all those reasons "always on" would be easier for me to ship personally.

With that said, having software that doesn't work as expected out of the box because of how I interact with assistive technologies was certainly "eye-opening" for me, and gave me a better understanding of how other people go through the world so that part was kinda cool actually.

I'd rather avoid doing this if there is no technical roadblocks for Accerciser to emit this signal.

I can't speak to this technically, but I can mention the distribution story for accerciser is less than excellent; the distro I normally use doesn't package it at all and I'm not sure how hard it will be to build on my own. I'd be interested to know what KDE's tool does here, I wound up on accerciser because I just figured the GNOME tooling would be more popular and well-traveled.

[drewcrawford]

I'd be interested to know what KDE's tool does here

Found out today. KDE's is like Accerciser by default, but there is the attached setting that does wake up accesskit. It also seems to wake up a lot of other KDE-based trees so I guess lazy initialization is more popular here.

At a glance I'd say KDE's tooling is ahead here, and might be the better way for me to develop...

[mwcampbell]

I'm now much less sure that I made the right decision about lazy initialization, as you can see in the last comment I posted on that slint issue. I'm tempted to do a complete 180 now. After all, especially at this stage of AccessKit's development (more or less usable, but still incomplete, and pre-1.0), it might be good if the developers (presumably mostly sighted) who are integrating AccessKit into their toolkits and applications immediately run into performance problems.

At the same time, lazy initialization of accessibility is something that every serious GUI toolkit (possibly excluding GTK 4) and browser engine I know of attempts to do. Chromium does it for sure, because accessibility has an easily noticeable impact on performance in Chromium. A sighted person once told me that he did some testing with a screen reader, then after he turned the screen reader off, Chromium remained sluggish, particularly on a large web page, because Windows and macOS don't have a clean way for the application to find out when assistive technologies are turned off. So, with all of that in mind, in addition to the more hypothetical concern I raised yesterday, I figured lazy initialization was a foundational part of the design that I should do fairly early on (I did the first implementation on Windows in late 2021).

Maybe the best solution would be to let the toolkit/application developer decide. So, for instance, Adapter::new could take an enum with two variants: an ActivationHandler implementation, or an initial TreeUpdate. If the developer chooses the latter, AccessKit would always be active, and any performance problems that come with pushing tree updates would be there, regardless of whether assistive technology is active.

Or maybe, as @drewcrawford originally suggested, the adapter should always be active in debug builds.

[DataTriny]

Thank you @drewcrawford for reporting your findings on KDE Accessibility Inspector. I never use KDE apps myself and tend to ignore them but if one is simply better than the GNOME alternative, this must be known.

Unfortunately Accerciser doesn't just have a packaging issue. I even doubt we could get it fixed to support the changes described earlier. I'd honestly write something new if I had the time.

[DataTriny]

I'd strongly be against unconditionally pushing and diffing tree updates. This could help ensure performance stays acceptable, but basically not much more. To focus on Linux: keep in mind that:

• we have to register D-Bus objects on a private bus, but this bus is not guaranteed to be available on all systems. What would we do then?
• significant performance costs are due to the back and forth between the app and ATs, and by ATs calling specific D-Bus methods on exposed objects. All of this can only be discovered by running an assistive technology. This diminishes the value of forcing accessibility for the purpose of keeping performance in check IMO.

I hope the issue will simply disappear once stuff using AccessKit start being widely used, at which point the pool of real world testers should expand. I'm also not in favor of making our API more complex to let UI toolkit developers decide whether they want accessibility to be lazily initialized or not.

I can get behind disabling lazy initialization on debug builds though. I think this would play well with the rest of the ecosystem.