Programatically disabling hardware encoding/decoding in WebRTC

[fippo]

Request for position on an emerging web specification

• WebKittens who can provide input: @youennf

Information about the specification

• Title: Programatically disabling hardware encoding/decoding in WebRTC
• URL: https://w3c.github.io/webrtc-extensions/#disable-hardware
• GitHub repository: https://github.com/w3c/webrtc-extensions

Design reviews and vendor positions

• TAG Design Review:
• Mozilla standards-positions issue: Programatically disabling hardware encoding/decoding in WebRTC mozilla/standards-positions#728

Bugs tracking this feature

• WebKit Bugzilla:
• Radar:

Anything else we need to know

The hardware encoders used by Safari are problematic as well as seen from these issues from the list-of-doom:
iPadOS 15 / iOS 15 unable to decode VP9 stream
https://bugs.webkit.org/show_bug.cgi?id=230604#c11

Safari: VP9-SVS no video stream from remote peer on some devices
https://bugs.webkit.org/show_bug.cgi?id=231074

Apple Silicon: Black screen share on remote side of peerconnection
https://bugs.webkit.org/show_bug.cgi?id=236870

and last but not least the infamous https://bugs.webkit.org/show_bug.cgi?id=232006

[annevk]

cc @youennf

[youennf]

The hardware encoders used by Safari are problematic

I do not think this is true for any of the bugs mentioned here.
For one of them, there is indeed an issue with the HW decoder support but a software workaround was put in place.

The justification of this API seems to be browser bugs.
Given they have been fixed and websites can already decode in JavaScript themselves as a temporary stopgap, I do not think this API is worth pursuing, especially in UAs that have a limited number of HW configurations.

The note The methods specified in this section should be used sparingly and not for extended amounts of time. indicates this could be a footgun for web developers as well.

Also, the potential fingerprinting issues are not really mentioned. And the proposal is not aligned with Media Capabilities or WebCodecs. Doing something closer to https://www.w3.org/TR/webcodecs/#enumdef-hardwareacceleration might be better in that sense.

[nils-ohlmeier]

Also, the potential fingerprinting issues are not really mentioned. And the proposal is not aligned with Media Capabilities or WebCodecs.

Just as word of caution: Media Capabilities increases the fingerprinting as well. Especially since the playback API's are not guarded with anything as far as I know.

But I do agree that it would make more sense to continue extending the Media Capabilities API for encoding purposes, especially if it this could get leveraged in WebRTC and WebCodecs.

[youennf]

Until the proposal is reworked to better align with existing APIs, I would go with position: oppose given the potential perf and fingerprinting issues.

[hober]

Barring objections, we'll mark this position: oppose in a week.