Joining channel without calling "start preview" on web doesn't listen for remote video stream

[spideythewebhead]

Version of the agora_rtc_engine

6.3.1

Platforms affected

• Android
• iOS
• macOS
• Windows
• Web

Steps to reproduce

1. Initialise rtc engine => createAgoraRtcEngineEx & engine.initialize
2. Request camera permission and decline
3. Enable video module => engine.enableVideo
4. Join a channel engine.joinChannel => doesn't receive remote video stream
5. Request camera permission again and accept it => Start preview and enable local video stream => engine.startPreview & enableLocalVideo => remote users don't receive video stream & local user doesn't receive remote video stream

Expected results

In our app request the necessary permissions for microphone/camera based on some conditions.
Then the app calls the enableVideo method and if the user has accepted the request for accessing the camera, then the app calls startPreview and followed by enableLocalVideo.

If the user has declined the camera access before joining the channel - which means startPreview and enableLocalVideo won't be called- the user doesn't seem to receive events related to the video stream from a remote user.

Now if you requested the camera permission, get access and call these 2 methods nothing happens.

Actual results

1. Being able to receive remote video stream without needing to join a channel with camera permissions/self preview enabled.
2. Being able to grant/open camera after joining a channel

Code sample

Code sample

[Paste your code here]

Screenshots or Video

Screenshots / Video demonstration

[Upload media here]

Logs

Logs

[Paste your logs here]

Flutter Doctor output

Doctor output

[Paste your output here]

[littleGnAl]

Does it work on native?

[qiuyanli1990]

Hi,
I have just tested Flutter web 6.3.1 on my end. I decline camera permission and call enableVideo() and enableLocalVideo(false) and NOT call startPreview(). Flutter web receives remote video streams properly after joining the channel.

Could you please share the log of your testing session in which you encountered the issue?

It would be helpful if you could clean the console log before testing the issue for the first time and avoid any extra actions during the test. To retrieve the console log, right click on the web page, select "inspect" and then go to the "console" tab. Please copy and send us the complete console SDK log for further investigation. Thank you.

[spideythewebhead]

Does it work on native?

Yes native is working fine

[spideythewebhead]

Hi, I have just tested Flutter web 6.3.1 on my end. I decline camera permission and call enableVideo() and enableLocalVideo(false) and NOT call startPreview(). Flutter web receives remote video streams properly after joining the channel.

Could you please share the log of your testing session in which you encountered the issue?

It would be helpful if you could clean the console log before testing the issue for the first time and avoid any extra actions during the test. To retrieve the console log, right click on the web page, select "inspect" and then go to the "console" tab. Please copy and send us the complete console SDK log for further investigation. Thank you.

Forgot to mention the problem is mainly on mobile browsers. I will add the logs later in the day, thanks.

[qiuyanli1990]

I tested the same on mobile web and it can also receive remote video stream in the same test scenario mentioned above.

I'll be waiting for your log to check further.

[spideythewebhead]

Hello, after testing again on the example agora app, it seems it works fine and the issue remaining on our app. I will further investigate what could cause this issue and will come back with more info. Thanks for the help

[spideythewebhead]

I am attaching the piece of code that initializes rtc and joining the channel (also this code was working fine on agora v5.x)

try {
      final rtcEngine = createAgoraRtcEngine();
      await rtcEngine.initialize(RtcEngineContext(
        appId: appId,
        areaCode: AreaCode.areaCodeEu.value(),
        logConfig: const LogConfig(level: LogLevel.logLevelInfo),
      ));
      _setRtcListeners(rtcEngine);
      disposableContainer.addDisposable(rtcEngine.release);

      _rtcEngine = rtcEngine;
      
      // Explain to user why we need the microphone permission
      await Permission.microphone.isGranted.then<void>((bool hasMicPermission) async {
        if (!hasMicPermission) {
          final completer = Completer<void>();
          emitSideEffectEvent(LiveSessionEvent.showPermissionsExplanationDialog(
            onComplete: completer.complete,
          ));
          await completer.future;
        }
      });

      // request the appropriate permissions based on the appointment type
      final permissionsResult = await [
        Permission.microphone,
        if (appointment.type == AppointmentType.video) Permission.camera,
      ].request();

      // cancel further initialization as microphone permission is rquired
      if (!permissionsResult[Permission.microphone]!.isGranted) {
        emitSideEffectEvent(const LiveSessionEvent.showPermissionsDeniedDialog());
        return const _RtcInitResult(initSuccessfully: false);
      }

      await rtcEngine.muteLocalAudioStream(false);

      await rtcEngine.setVideoEncoderConfiguration(VideoEncoderConfiguration(
        orientationMode: OrientationMode.orientationModeFixedPortrait,
        degradationPreference: DegradationPreference.maintainBalanced,
        frameRate: FrameRate.frameRateFps30.value(),
        mirrorMode: VideoMirrorModeType.videoMirrorModeEnabled,
      ));
      await rtcEngine.enableVideo();

      if (permissionsResult[Permission.camera]?.isGranted ?? false) {
        await _enableLocalVideo(true);
        await rtcEngine.startPreview();
      } else {
        await _enableLocalVideo(false);
      }

      return _RtcInitResult(
        initSuccessfully: true,
        grantedCameraPermission: permissionsResult[Permission.camera]?.isGranted ?? false,
      );
    } catch (exception, stackTrace) {
      Logger.error(
        'live-session-init',
        exception: exception,
        stackTrace: stackTrace,
      );
      emitSideEffectEvent(const LiveSessionEvent.failedToInitializeSession());
      return const _RtcInitResult(initSuccessfully: false);
    }

If initializatio is succesful

_rtcEngine!.joinChannel(
             token: ok.data.channelToken,
             channelId: _appointmentId,
             uid: _me.isCustomer ? _kAgoraCustomerId : _kAgoraDoctorId,
             options: const ChannelMediaOptions(
               channelProfile: ChannelProfileType.channelProfileCommunication,
               clientRoleType: ClientRoleType.clientRoleBroadcaster,
             ),
           );

Camera toggle for remote

  void toggleCameraForRemote() async {
    try {
      if (dataState.snapshot.isCameraOpened) {
        await rtcEngine.stopPreview();
        await _enableLocalVideo(false);
        state = dataState.copyWith(
            snapshot: dataState.snapshot.copyWith(
          isCameraOpened: false,
        ));
        return;
      }

      final permission = await Permission.camera.request();
      if (!permission.isGranted) {
        emitSideEffectEvent(const LiveSessionEvent.showPermissionsDeniedDialog(shouldExitSession: false));
        return;
      }

      await rtcEngine.enableVideo();
      await _enableLocalVideo(true);
      await rtcEngine.startPreview();
      state = dataState.copyWith(
          snapshot: dataState.snapshot.copyWith(
        isCameraOpened: true,
      ));
    } catch (exception, stackTrace) {
      Logger.error(
        'live-session-toggle-camera',
        exception: exception,
        stackTrace: stackTrace,
      );
    }
  }

Self preview widget

...
state.snapshot.isCameraOpened
                        ? Material(
                            elevation: 3.0,
                            shape: const CircleBorder(),
                            child: ClipRRect(
                              borderRadius: BorderRadius.circular(65.0),
                              child: AgoraVideoView(
                                controller: VideoViewController(
                                  rtcEngine: state.snapshot.rtcEngine,
                                  canvas: const VideoCanvas(uid: 0),
                                ),
                              ),
                            ),
                          )
                        : Ui.empty,
...

final LiveSessionDataState snapshot = dataState.snapshot;
        final int? remoteRtcUid = snapshot.remoteUserRtcUid;

        if (remoteRtcUid == null || remoteRtcUid == -1) {
          return Column(
            mainAxisSize: MainAxisSize.min,
            children: [
              ScaleTransition(
                scale: _scaleAnimation,
                child: Avatar(
                  radius: 64.0,
                  placeholder: const Icon(Icons.person),
                  loading: const CircularProgressIndicator(),
                  url: snapshot.remoteUser.photoUrl,
                ),
              ),
              Ui.vspace8,
              if (snapshot.remoteUserRtcUid == -1)
                Text('Αποσυνδέθηκε', style: context.theme.textTheme.titleLarge)
              else
                Text('Αναμένεται σύνδεση..', style: context.theme.textTheme.bodySmall),
            ],
          );
        }

        if (!kIsWeb && !snapshot.isRemoteCameraOpen) {
          return Avatar(
            radius: 64.0,
            placeholder: const Icon(Icons.person),
            loading: const CircularProgressIndicator(),
            url: snapshot.remoteUser.photoUrl,
          );
        }

        return AgoraVideoView(
          controller: VideoViewController.remote(
            rtcEngine: snapshot.rtcEngine,
            connection: RtcConnection(channelId: snapshot.appointment.id),
            canvas: VideoCanvas(uid: remoteRtcUid),
          ),
        );

[littleGnAl]

Forgot to mention the problem is mainly on mobile browsers

Hello @spideythewebhead, as you mentioned above, what mobile browsers(iOS Safari/chrome, Android Chrome?) are encountering the issue?
Does it happen in the debug environment or the production environment?

I think the logs are helpful for further investigation, is it possible to share the logs in the console? You can inspect the mobile browsers to get the logs
iOS: https://developer.apple.com/documentation/safari-developer-tools/inspecting-ios
Android: https://developer.chrome.com/docs/devtools/remote-debugging

[qiuyanli1990]

The code seems ok. Better we have sdk log to take a further look. If it is difficult to get the console log. You can also provide us the channel name so that we can take a rough check with the channel log for the first step on our end.
Thanks!

[spideythewebhead]

Hello, will do later the day, thanks

[spideythewebhead]

Sorry for the delay here are the logs from a session using an android and an iphone.. let me know if you need anything else

xiaomi_android13_chrome.txt

iphone14_ios16_safari.txt

[littleGnAl]

Based on your log https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/files/15488453/iphone14_ios16_safari.txt, it is most likely the remote user had not published the video, I think this is why you can receive the remote user's video.

[spideythewebhead]

Yes exactly but that's the problem even if I call these method that enable the camera after joining a channel the remotes don't receive the streams.. if I do it before it does.. but probably this is some big in my code as it seems to be working on the example app... Even tho I can't figure out why

[littleGnAl]

I think you should call updateChannelMediaOptions if you enable the camera after joining a channel

[spideythewebhead]

Hi I have added the call you suggested and now the remote can see the local user fine. I am just wondering why this works fine on the mobile and not on the web, is this a difference between the native implementations?

Also something else is that while I am updating the channel options the remote microphone stream is unsubscribed and the audio is lost for couple of seconds.. I will search it further and will come back with more info

[spideythewebhead]

Sadly whatever I do some other problem will pop on the web.. so sadly we will stay on 5.x. Thanks

[littleGnAl]

We're so sorry that this issue blocked your upgrade, the web support on 6.x is still in the alpha stage, and many thanks for your feedback.

I am just wondering why this works fine on the mobile and not on the web, is this a difference between the native implementations?

Also something else is that while I am updating the channel options the remote microphone stream is unsubscribed and the audio is lost for couple of seconds

I think these issues above should be bugs. We aim to keep the APIs behave the same across the platforms unless there are limitations on the platforms.

We will take a look at these issues later, I will update here if there's any progress.