Does PTZ look good to you?

[beaufortfrancois]

We are ready to ship Camera Pan-Tilt-Zoom support in Chromium with latest spec changes (#261, #260) and we'd love to hear what WebKit (@youennf) and Gecko (@jan-ivar) folks think about this.
In other words, do you folks agree with the overall consensus found at the previous virtual interim event?
And more specifically, is this realistic to think spec authors will have merged remaining PTZ related PRs in the coming days?

For info, you can play with Camera PTZ in Chrome Canary by going to https://ptz.glitch.me and enabling the experimental-web-platform-features flag in chrome://flags.

[jan-ivar]

@beaufortfrancois Great to hear you're close! I missed the meeting, but I agree with the resolutions posted in the minutes.

I haven't had time to play with it, and this is unfortunately an area where we don't have good web platform tests, so it's a bit hard to judge.

I've commented in the PRs where I feel details still need to be sorted out, so I'm not exactly sure which interpretation of the consensus Chrome aligns with. For instance, I'm going to guess it doesn't implement fitness distance, since #257 got closed.

[eehakkin]

Chrome implements fitness distance so that {pan: {}} increases fitness distance for pan incapable devices by 1 in the basic set and effectively by infinity in the advanced sets. Thus navigator.mediaDevices.getUserMedia({video: {pan: {}}}) prefers pan capable devices and navigator.mediaDevices.getUserMedia({video: {advanced: [{pan: {}, ...}]}}) either selects a pan capable device or ignores the advanced constraint set completely.

[jan-ivar]

@eehakkin Thanks for the info. Had you said {pan: true} instead of {pan: {}} then I think that would have been perfect.

But {} does not traditionally register in the constraints algorithm. That's a bug in the spec still (#256).

My preference here would be to revive #257 and deprecate {}.

I approve of the behavior for true. Can we agree to at least remove all mention of {} in docs and examples until this has been resolved?

[beaufortfrancois]

I approve of the behavior for true. Can we agree to at least remove all mention of {} in docs and examples until this has been resolved?

{} is not used in docs and examples. I guess we're good then.

[beaufortfrancois]

For info, we've shared our intent to ship Camera PTZ support in Chrome 87.
See https://twitter.com/ChromiumDev/status/1313036318105055234

[louisparks]

Does anyone know of a list of compatible cameras that will support these capabilities. I have a Logitech brio which seems to support it digitally, but it doesn't seem to work fully with Logitech PTZ PRO2. It seems there is partial support, for zoom, but not pan or tilt. Any idea why it doesn't report the full capabilities?

[beaufortfrancois]

@louisparks I'm surprised the Logitech PTZ PRO2 doesn't report pan & tilt capabilities when looking at the spec.
May you go to the internal about://media-internals page and check out the "Pan-Tilt-Zoom" column in the "Video Capture" tab?

[louisparks]

@beaufortfrancois thank you so much for responding. This is what the camera reports and it's consistent with what I see. Zoom is controllable but not pan or tilt. Is this something Logitech needs to address? Would this require firmware or driver update?

[beaufortfrancois]

I'm surprised only "zoom" is advertised. Which operating system do you use? Can you reproduce on others (Windows, macOS, Linux)?

[beaufortfrancois]

I believe you use macOS as the Capture API is "AV Foundation". Can you reproduce on Windows, Linux, or Chrome OS by any chance? In other words, do you see "pan" and "tilt" on other desktop platforms?

[louisparks]

Tried Windows 10

[beaufortfrancois]

I guess this is a Logitech issue then. @ogay Would you be able to help with this?

[louisparks]

I don't have a linux bare metal install, but this is how it reports it in a Parallel's ubuntu VM. seems to see the same

Thanks for any help on this. We have contacted Logitech many times, but it's been impossible to get a hold of anyone that can really understand the issue, which of course leads them to just blame it on Chrome.

[beaufortfrancois]

It seems like the PTZ Pro 2 (and Rally) cameras use the VISCA protocol which explain why Pan and Tilt are not exposed in Chromium. It only uses UVC to detect Pan, Tilt, and Zoom.

[riju]

We did think about VISCA here. It would be great if someone picks up the work.

[louisparks-sta]

Thank you again @beaufortfrancois for the response,

Are you saying that Chrome uses UVC for the detection?

FWIW, we have an electron container that controls this cameras pan, tilt and zoom on these cameras via UVC, so I know it does implement UVC as well. Our goal is to get rid of the container if we can get Chrome to do it natively. We just can't get this one specific camera to work

Thanks..

[beaufortfrancois]

Thank you again @beaufortfrancois for the response,

Are you saying that Chrome uses UVC for the detection?

Yes.

FWIW, we have an electron container that controls this cameras pan, tilt and zoom on these cameras via UVC, so I know it does implement UVC as well. Our goal is to get rid of the container if we can get Chrome to do it natively. We just can't get this one specific camera to work

Can you share public electron code that uses UVC (ant not VISCA) to control PTZ?

Thanks..

[louisparks-sta]

we are using this library, https://www.npmjs.com/package/uvc-control2
which is abstracting out the actual commands, but they in fact controlling all aspects of the PTZ. would it help if we dug a little deeper into what that library is sending?

Thanks again for your help

[beaufortfrancois]

Can you share the code you're using to control PTZ when using this library?
It should look likenew UVCControl(...).

[beaufortfrancois]

When controlling PTZ, do you use absolutePanTilt or relativePanTilt?

[louisparks-sta]

I know relative was more what we finally implemented, but let me see if we got absolute working as well.

Below is the code for UVCControl lib and that Logitech camera

this.uvcCamera = new UVCControl(1133, 2143);

ptz(action: string): void {
switch (action) {
case "up":
this.ptzCommandRelative(0, 1);
break;
case "down":
this.ptzCommandRelative(0, -1);
break;
case "left":
this.ptzCommandRelative(1, 0);
break;
case "right":
this.ptzCommandRelative(-1, 0);
break;
case "stop":
this.ptzCommandRelative(0, 0);
break;
case "zoomIn":
this.ptzCommandZoomRelative(1);
break;
case "zoomOut":
this.ptzCommandZoomRelative(-1);
break;
case "zoomStop":
this.zoomStop();
break;

  default:
}

}
ptzCommandRelative(pan: number, tilt: number): void {
// PAN 1=left -1=right
// TILT 1=up -1=down
const buffer = Buffer.alloc(4);
buffer.writeIntLE(pan, 0, 2);
buffer.writeIntLE(tilt, 2, 2);
this.uvcCamera.setRaw("relativePanTilt", buffer).catch(console.error);
}

[beaufortfrancois]

Chromium only uses pan and tilt absolute controls, not relative ones.

[beaufortfrancois]

I've added the paragraph below to https://web.dev/camera-pan-tilt-zoom/

To know whether a Chromium-based browser supports PTZ for a camera, go to the internal about://media-internals page and check out the "Pan-Tilt-Zoom" column in the "Video Capture" tab; "pan tilt" and "zoom" respectively mean the camera supports the "PanTilt (Absolute)" and "Zoom (Absolute)" UVC controls. The "PanTilt (Relative)" and "Zoom (Relative)" UVC controls are not supported in Chromium-based browsers.

[jan-ivar]

All PTZ related PRs have been merged, so this can be closed.

[JeffreyTaylor]

Been playing around with this - really great work here. As mentioned above -- any suggestions on what to do with the cameras that just respond to relative tilt & pan controls (e.g Logitech Rally and PTZ Pro 2)? These cameras just report out zoom in about://media-internals. Are there plans to add support for relative pan and tilt values? If not, any workaround ideas?

[naher94]

Been playing around with this - really great work here. As mentioned above -- any suggestions on what to do with the cameras that just respond to relative tilt & pan controls (e.g Logitech Rally and PTZ Pro 2)? These cameras just report out zoom in about://media-internals. Are there plans to add support for relative pan and tilt values? If not, any workaround ideas?

Great question.. would love to hear about work arounds

[jan-ivar]

Do these cameras offer a way to reset pan and tilt (to straight ahead)?

If so, user agents might be able to map some virtual coordinate system ontop of this, and keep track of absolute position and tilt.

[JeffreyTaylor]

@jan-ivar I'm looking, but the capabilities returned by these two cameras only seem to show that zoom is supported. Looking at about://media-internals also just shows zoom. I'm not seeing a way to reset pan/tilt from a javascript interface ATM. These Logitech (Rally and PTZ Pro 2) are most definitely PTZ cameras. Mapping a virtual coordinate system(from relative to absolute) sounds like a plan, but after the mapping is made, I'm not seeing a mechanism to send a javascript command to the device.

[jan-ivar]

Sorry, I'm not suggesting there is support or providing a workaround. My comment was that browsers should be able to add support using the technique I mentioned, without requiring that we change the API.

This issue tracker is for discussing the specification, not individual browser implementations.

[JeffreyTaylor]

Yes sounds good. In that spirit, I've opened up #281 to discuss introducing some type/any type of support for the relative pan/tilt cases.

[dannyvargas23]

Hi, this is for anyone who is looking for a good PTZ camera. I can confirm that the Atlona AT-HDVS-CAM PTZ Camera with USB works in chrome.

[moiz4]

dannyvargas23, thanks for your comment.

Hey everyone, Please give me more recommendations for the ptz cameras that would work in chrome with WebUSB API. I've to urgently buy a camera that would work with absolute Pan/tilt commands supported by chrome and webUSB API. Your recommendations would be highly appreciated.

[recursivecodes]

chrome://media-internals is reporting "N/A" for both a Logitech Brio and C920 in Chrome 107 on MacOS. Both of these cameras support at least zoom (confirmed via Logitech software and third-party software). Is there somewhere I can file a bug for this?

[beaufortfrancois]

I think you're hitting https://bugs.chromium.org/p/chromium/issues/detail?id=1270335 sadly.

[recursivecodes]

Thanks for the heads up. This is unfortunate. :-/