Selection of camera for Phase 2

[andyburras]

The intention is for Phase 2 to be a more automated approach and to include audio and AV sync testing. This places more stringent requirements on the camera used for the AV capture. The options today seem very limited.

Minimum criteria used are:

1. Publicly published camera control API that supports: start/stop recording, identifying a recording, downloading the recording, deleting the recording.
2. Record video at 120/100fps (to allow capture at twice maximum presentation framerate).
3. Sufficient quality lens/sensor to allow QR code capture at low resolutions on small screen devices.
4. Record synchronized audio and video with frame accuracy.
5. Acceptable price point.

Point (1) rules out a number of manufacturers altogether, as they only support camera control from their own proprietary applications.

Point (2) narrows the range of possible models.

Experimentation with a GoPro camera failed on point (3). Recordings of large screen TVs were acceptable, but low resolution presentations on phones produced blurred images and QR code identification failed. We believe this is because the lenses and sensors in such "action" cameras do not allow sufficiently sharp focussing at the required short distances.

Point (4) looks to rule out manufacturing/surveillance type cameras.

Also a number of candidate cameras from e.g. Canon and BlackMagic turned out to either not capture audio at all when recording at 100/120fps, or to not capture synchronised AV. (the typical use case for high framerate capture is to be played back at lower framerates to give smooth slow motion).

An exception to this is Sony which classes 200+fps as high framerate, and 100/120fps as a standard recording, i.e. with synchronised AV.
However there is currently a limitation with Sony. They used to have a camera control API (which is still supported by some models) but this has now been archived and is receiving no further support (https://developer.sony.com/develop/cameras/#overview-content ).
In 2020 Sony announced a new Camera Remote SDK (https://support.d-imaging.sony.co.jp/app/sdk/en/index.html ) This is available for both Linux and Windows and is free to use. However, because this is so new currently only 4 camera models support it. The cheapest of these, which on paper appears to meet all our criteria, is the Sony ILCE-7C at around $2,100 including lens (https://www.sony.com/electronics/interchangeable-lens-cameras/ilce-7c ).

We believe this situation will resolve itself in time. It seems reasonable to expect that as manufacturers release more advanced models then there will be wider support for 120/100fps AV recording, and we can expect that Sony will be releasing newer models that also support their new SDK.

We will be monitoring the situation to assess what is being released. We would also welcome any suggestions/recommendations from members of the group.

[jpiesing]

As well as the webcam utility, GoPro have an open source library for communication with a GoPro including some demos. https://github.com/gopro/OpenGoPro

[jpiesing]

This is also relevant ...

https://www.youtube.com/watch?v=PCgB2Y5girY

There's a lengthy discussion in the comments about this special mode for indoor use on a tripod where all sorts of processing is turned off to avoid overheating.

[jpiesing]

This is also relevant ...

https://gopro.com/en/rs/news/guide-to-hero10-black-frame-rates

[jpiesing]

And another interesting article.

https://camerajabber.com/buyersguides/which-cameras-shoot-4k-at-120fps/

The ones I consider affordable in a WAVE context are;

GoPro Hero10 Black

Kandao QooCam 8K ("Vloggers will also appreciate the QooCam 8K’s 3.5mm mic port, making it one of the few 360 cameras to offer one. ")

[jpiesing]

Here are some questions for people to consider ....

• What is a reasonable / maximum price for a camera for DPCTF testing? <=$500? <=$1000, <=$2000, <=$5000?
  For reference, I'm using my personal Sony RX10M3 which is $1500-$2000 and I consider that too expensive for general use.
• For phase 2, the requirements go up - streaming and/or the ability to control the camera from a PC and stop/start recording. What if there's no affordable camera which meets these requirements & can be used for all the tests?
• We may find the best candidate cannot stream 1080p60 / 1080p120 at all or can only do so for a short time without over-heating.
• This would mean some tests could be streamed and recorded on the PC but others (p50/p60 content) had to be recorded on memory in the camera and downloaded.
• If there's no candidate that meets all requirements, is it worth spending any money on a partial solution? Should we give up and stick with the phase 1 camera?

[yanj-github]

GoPro does not support high frame rate streaming, it only supports 30fps and 60fps.

[pmaness]

I don't think a consumer product will cut it. Go Pro can only do it when recording to an internal storage device.
For 120Hz capture, I think it's tough to find a high speed laboratory grade camera with integrated audio.
Particularly, search this on Amazon: "High Speed USB3.0 Color 2.3MP 1/1.2" Machine Vision Global Shutter Digital Industrial Camera C-Mouth Supports Windows and Linux SDK Camera 1920x1200 165FPS"
Audio capture would then be a separate path, but it would be a simple matter to have a "flash/ping" pre-roll sync reference to align audio and video captures.

[jpiesing]

@pmaness That's a really great reference. Having read the article, if you search for "USB 3.0 Vision" you can find lots of industrial cameras.

[andyburras]

When we looked into this a while ago, our concerns with the separate capture paths for audio and video approach was that if the sync signalling is being output by the device-under-test and the device is out of sync from the start then any automatic aligning of the video and audio inputs will effectively "fix" the synchronisation, leading to a false result. I guess you could introduce a manual step to the start of any session that includes AV sync tests, and require the user to manually confirm the AV sync of the test signal prior to the actual tests being run.

Our other concern was that if the AV was in sync at the start but then drifted out during a test session, could we be sure that it was due to the device and not to the test rig set up, the input processing on the PC of the audio and video signals, etc.

The aim of using a camera was to ensure that the recorded audio and video was being synchronised at the point of capture. Part of the camera selection process would be to record a known "good" test media and manually verify that the candidate camera model was successfully synchronising the AV over a lengthy recording.

However, if there is a reliable way of using industrial cameras then they would be an option.

[yanj-github]

Choosing a camera depends on requirements details can be found https://github.com/cta-wave/dpctf-deploy/wiki/Choosing-a-Camera.

[jpiesing]

Choosing a camera depends on requirements details can be found https://github.com/cta-wave/dpctf-deploy/wiki/Choosing-a-Camera.

Here are a few comments for your consideration ...

1. The requirement "Streaming, local recording or streams can be downloaded to local" is confusing and the responses in each cell are also confusing. Surely this is something like "Streaming to PC or moving memory card from camera to PC" and each cell will be identically either "Streaming" or "Moving memory card".

2. For "Test audio/video sync" where it says "shall be tested upfront", I think it would be better to say "shall be calibrated upfront".

3. For mobile camera, the heat requirements in "Test playback of movie-length content" can be mitigated by using a mains power supply and, in the case of a GoPro, removing the battery. This could be added as a (2).

4. A small cosmetic issue, "Limited by battery capacity and overheat" should be "Limited by battery capacity and overheating". (i.e. add "ing").

[yanj-github]

Thanks @jpiesing for the comments.
We have updated wiki and addressed all of your comments.
Can you have a look please? And if you happy feel free to close the issue.

[jpiesing]

The row for streaming or local recording is IMHO still confusing. I was expecting something like this;

Requirement	Mid-range cell phone	High-end cell phone	Webcam	Mobile camera	Low-end professional camera
Streaming or local recording	Local recording	Local recording	Streaming	Streaming video only, local recording for both video and audio	Streaming