Clarify unmute event must fire on receiver.track AFTER sRD(offer) succeeds

[jan-ivar]

This invariant lets apps use getReceivers() to discover receivers added by sRD and attach event handlers in time. E.g:

await pc.setRemoteDescription(offer);
const [receiver] = pc.getReceivers();
receiver.track.onunmute = () => {
  console.log("unmute fired");
  video.srcObject = new MediaStream([receiver.track]);
}

The spec already guarantees this:

1. sLD's success task sets "transceiver.[[Receptive]] to true,"
2. and "Whenever an RTCRtpReceiver receives data on an RTP source whose corresponding MediaStreamTrack is muted, but not ended, and the [[Receptive]] slot of the RTCRtpTransceiver object the RTCRtpReceiver is a member of is true, it MUST queue a task to set the muted state of the corresponding MediaStreamTrack to false."

IOW, since step 2 queues a task to fire the event, it happens AFTER the sRD success callback.

Unfortunately, none of the examples show this, and crbug 1295295 undermines this (fiddle above doesn't work in Chrome).

Shall we add an example and a note?

[mickel8]

Maybe that's not directly related to this issue but does it mean that there is no support for Early Media mentioned in this blog post.

By Early Media I mean the ability of offerer to receive and display MediaStreamTrack before receiving remote SDP answer assuming that the offerer is willing to receive.

Both the blog post I linked and JSEP RFC (section 3.2-4) mentions that the receive must be prepared to demux incoming streams before receiving remote SDP answer.

I've created a simple jsfiddle that tries to accomplish this. The flow is as follows:

1. Setup two PCs with a dummy data channel to establish ICE connection (both sides must exchange their credentials to accept connectivity checks)
2. Offer to receive audio on PC1
3. Apply offer, add audio track, create and apply answer on PC2
4. Don't apply answer on PC1

As a result, looking at chrome://webrtc-internals, I can see that the PC2 sends its audio track, PC1 receives this data on ICE candidate pair but it is not demuxed and no events like ontrack, onunmute etc. are fired. Also, nothing plays in the audio element.

If we uncomment line 54 i.e. we do apply answer on PC1, PC1 successfully receives RTP stream.

Is this intentional? If yes, why do we always need to create a transceiver with a receiver with a track?

[alvestrand]

DTLS setup can't happen until offer/answer is completed, and before DTLS setup is complete, you don't have the keys to decrypt SRTP packets.
So early media can't happen on initial offer/answer.

It can in theory happen on subsequent offer/answer, however.

When the WG discussed this in 2015 or so, it was considered "not so important we have to have extra code to accomodate it".

[mickel8]

Thanks! That was my suspection. That's for another discussion but if there is no support for early media, perhaps we could get rid of the requirement to always create receiver with a track. I know this would mean changes in onmute, onunmute, onaddtrack and onremovetrack events but just mentioning

[jan-ivar]

Sorry I made a typo in the OP initially (now fixed):

1. sRD's success task sets "transceiver.[[Receptive]] to true,"

It should have said:

1. sLD's success task sets "transceiver.[[Receptive]] to true,"

IOW, the offerer is receptive in sLD(offer), allowing media to arrive from the answerer ahead of sRD(answer) on renegotiation (only), e.g. if the currentDirection is renegotiated from "inactive" to "recvonly" or "sendonly" to "sendrecv".

if there is no support for early media, perhaps we could get rid of the requirement to always create receiver with a track. I know this would mean changes in onmute, onunmute, onaddtrack and onremovetrack events but just mentioning

The spec supports early media, so the model is not likely to change again, for that and other reasons, like the transceiver.receiver.track's lifespan being tied to the transceiver. I'm also unaware of any hardship from this invariant to web-developers / requirement on vendors.