Audio Issues with Video Merger

[jonathan-sjkim]

Hello I am implementing WebRTC conference app using video-merger.

App consists of One-To-Many Model and One Master Node get streams from each viewers and Master merge all streams and broadcast it.

I allocate merger to each Viewers and merge the each streams including video and audio. and i add video, audio on/off function by removing and adding with addStream() mute options.

The problem is, video works fine but "sometime" audio is not working. Although I set mute option always to false, the audio stream is not added to merger.(video works fine).

It seems the problem somtimes occurs in Chrome. Firefox works fine.

My Audio on/off function code is like this

 AudioOn() {
       var audioMute = false;
       merger.removeStream(stream);
       merger.addStream(stream, {
            x: 0,
            y: 0,
            width: merger.width,
            height: merger.height,
            mute: audioMute,
        });
    }
 AudioOff(){
        var audioMute = true;
       merger.removeStream(stream);
       merger.addStream(stream, {
            x: 0,
            y: 0,
            width: merger.width,
            height: merger.height,
            mute: audioMute,
        });
    }

Is there any problem coding like this? Or any dependencies with Browser?

[t-mullen]

Does the issue only occur after muting then unmuting, or does it sometimes happen the 1st time when adding the stream?

[jonathan-sjkim]

When i make merger, it is initialized as mute option.

It happens when i make over 3 mergers and it occurs in windows7/10 chrome browser. But it works fine in Firefox. Mac works fine in both chrome and firefox.

If i make over 3 mergers, first 2 mergers works fine, but the other merger did not work mute option change.

[jonathan-sjkim]

I assume because of the memory problem in windows chrome browser. The video merger takes a lots of memory and when i make over 3 mergers the memory usage increases. I thought this might effects windows chrome audio work. Just assumption.

[t-mullen]

So this only happens when you're creating multiple mergers?

[jonathan-sjkim]

Yes. in windows chrome, it occurs when i create over 3-4 mergers.

[chamamme]

@jonathan-sjkim i think muting an audio shouldn't be this intense. Just incase you have not tried this, get the audio tracks from the stream and then disable them.

 AudioOn() {     
      stream.getAudioTracks().map( track => track.enabled = true);
}

 AudioOff() {     
      stream.getAudioTracks().map( track => track.enabled = false);
    }

This is much efficient and does not cause the screen to black out when trying to remove and add streams.

[gsxipsarl]

I am actually having the same issue on Chrome, without creating multiple mergers or streams. It works fine on Firefox but it is always muted on Chrome. I send the merger.result to MediaRecorder and the output has no audio in Chrome, but it works fine on Firefox.