Front camera mirroring issue

[mrtndat]

I am working on a front camera mirroring feature.
The goal is to keep the local preview mirrored, but ensure the viewer sees the true (non-mirrored) view, so that any text shown by the host is readable.

The following API works as expected:

if (isFrontCamera) {
    setIsStreamHorizontalFlip(true)
} else {
    // The rear camera should not be mirrored.
    setIsStreamHorizontalFlip(false)
}

I call this code after switching the camera to keep the front camera output non-mirrored.

However, there is an issue on the viewer side. Right before the camera frame updates, the last frame from the previous camera gets flipped incorrectly.

From what I observed, the StreamHorizontalFlip flag is handled at the encoder layer. When switching the camera, there is a short delay, but the StreamHorizontalFlip flag is updated immediately. This causes the in-progress frames to be flipped.

Here are the steps:

1. Front camera (StreamHorizontalFlip(true)) capturing: F1, F2, F3, F4, F5

2. Press switch camera

3. Rear camera (StreamHorizontalFlip(false)) capturing: F6, F7, F8...

At the viewer side, frame F5 is flipped incorrectly because the StreamHorizontalFlip flag changed from true to false.

Question:
Do you have any idea how to fix this issue?

Thanks!

[pedroSG94]

Hello,

This is difficult to sync but maybe using the camera callback to know the moment that the camera change is a better solution:

(genericStream.videoSource as Camera2Source).setCameraCallback(object: CameraCallbacks {
      override fun onCameraChanged(facing: CameraHelper.Facing) {
        //camera switch finished here, you can flip now
      }

      override fun onCameraError(error: String) {
        TODO("Not yet implemented")
      }

      override fun onCameraOpened() {
        TODO("Not yet implemented")
      }

      override fun onCameraDisconnected() {
        TODO("Not yet implemented")
      }

    })

[mrtndat]

Hi @pedroSG94
Thanks for the suggestion, I already used that callback to detect camera change and apply flipping.
But the issue is still there.
The workaround that worked for me is to change the video source No Video Source, then delay for a short amount of time (250ms), after that, change the video source to camera source with new camera facing.
But I would expect a better solution.
Do you have any idea?

[pedroSG94]

Hello,

Sorry for the late response

I don't know other way. That should be exact.
Also, you can use the BlackFilterRender instead of change the video source if you want an alternative

[mrtndat]

Thanks. It might work but now the switch camera API becomes not useful when the mirroring is on.

[mrtndat]

@pedroSG94 do we know if a frame is coming from front or rear camera?

[pedroSG94]

No,

I have no way to know it. Camera2 only has a callback when the camera is opened/closed. You have the callbacks in the setCameraCallbacks method

[mrtndat]

@pedroSG94
The reason that we don't know about it is because the camera frame is drawn directly to a encoder surface and then we process frame from this encoder surface, so there is no information about front/rear camera for the frame?
am i understand correctly?

[pedroSG94]

Yes, that is correct.

Camera2 works rendering surfaces so you don't have a way to know metadata info about the frame because you only obtain image data in the render.

In this case the surface is not the encoder surface. It is the GlInterface surface but the logic is the same.

[pedroSG94]

You can do other way that maybe works better.

Using Camera2Source, instead of change to front camera you can try:

• close camera
• wait the disconnect camera callback
• flip the view
• reopen but in front camera directly.

[pedroSG94]

Also, keep in mind that maybe the surface is not cleared after close the camera getting the last frame in the surface until you reopen the camera.
You can try clear the surface too if this is a problem for you.

[mrtndat]

yeah, the problem is the last frame in the surface is still persisted after switching the camera.
do you think you can support this internally?
when switching the camera with flip frame is on, you clear the surface also?
what do you think?

[pedroSG94]

I can try it. For now do it manually and let me know if the problem is fixed. This is the method to do the clear:

fun SurfaceTexture.tryClear() {
  val surface = Surface(this)
  try {
    val canvas = surface.lockCanvas(null)
    canvas.drawColor(Color.TRANSPARENT, PorterDuff.Mode.CLEAR)
    surface.unlockCanvasAndPost(canvas)
  } catch (_: Exception) {} finally {
    surface.release()
  }
}