Add a blackFrames counter to MediaStreamTrackVideoStats

[guidou]

VC applications currently detect black frames by inspecting the payload of each frame.
In some cases, the system can provide information that the frames are black due to a camera occlusion.
Exposing this via a stat would allow the application to implement the same functionality with lower resource usage.

cc @youennf @jan-ivar

[youennf]

In some cases, the system can provide information that the frames are black due to a camera occlusion.

If some OSes already expose that information, this seems ok to me.
The question we could discuss is whether a track should be muted instead in that case.

Another question is how web application would know whether the OS supports it or not.
One option would be to have the counter property be undefined for OSes that do not support this.
For OSes that expose that information, the counter property would be exposed with 0 as the default value.

[eladalon1983]

The question we could discuss is whether a track should be muted instead in that case.

I'd say that it's likely helpful to mute in this case (more efficient), *if* we ensure that the app can detect the reason.

[guidou]

If some OSes already expose that information, this seems ok to me. The question we could discuss is whether a track should be muted instead in that case.

Some OSes do for some machines. I think muting would be useful since that would give a signal to the application to stop using the less efficient frame inspection and look at the counter.

Another question is how web application would know whether the OS supports it or not. One option would be to have the counter property be undefined for OSes that do not support this. For OSes that expose that information, the counter property would be exposed with 0 as the default value.

That makes sense to me.

[eladalon1983]

I think muting would be useful since that would give a signal to the application to stop using the less efficient frame inspection and look at the counter.

Ideally, an app running on a UA that would mute in this case, would not even need to run the less efficient black-frame-detection at all. Maybe we should give the app the ability to detect that.

[guidou]

I think muting would be useful since that would give a signal to the application to stop using the less efficient frame inspection and look at the counter.

Ideally, an app running on a UA that would mute in this case, would not even need to run the less efficient black-frame-detection at all. Maybe we should give the app the ability to detect that.

Correct. @youennf 's idea of not exposing the counter on systems that don't support this would be a way to detect it.

[alvestrand]

Are we sure a browser can always tell if the OS exposes this information or not?
Or might it be camera driver specific?

[guidou]

Are we sure a browser can always tell if the OS exposes this information or not? Or might it be camera driver specific?

I think we can expose that the OS can potentially expose this, but I guess it's camera/laptop/driver/somethingelse specific. So this is would not be a full substitute for applications detecting black frames, but a way to do it more efficiently (and perhaps more accurately) in some cases.