Skip to content

Commit

Permalink
Add captureTimestamp and senderCaptureTimeOffset to frame metadata
Browse files Browse the repository at this point in the history 
Fixes #225
  • Loading branch information
@Orphis
Orphis committed May 2, 2024
1 parent 4b61373 commit 71a2372
Showing 1 changed file with 103 additions and 0 deletions.
103 changes: 103 additions & 0 deletions index.bs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ spec:webidl; type:dfn; text:resolve
"CloneArrayBuffer": {
"href": "https://tc39.es/ecma262/#sec-clonearraybuffer",
"title": "CloneArrayBuffer"
},
"RTP-EXT-CAPTURE-TIME": {
"href": "https://webrtc.googlesource.com/src/+/refs/heads/main/docs/native-code/rtp-hdrext/abs-capture-time",
"title": "RTP Header Extension for Absolute Capture Time",
"publisher": "WebRTC Project"
}
}
</pre>
Expand Down Expand Up @@ -293,6 +298,10 @@ The <dfn method for="SFrameTransform">setEncryptionKey(|key|, |keyID|)</dfn> met

# RTCRtpScriptTransform # {#scriptTransform}

In this section, the capture system refers to the system where media is sourced from and the sender system
refers to the system that is sending RTP and RTCP packets to the receiver system where {{RTCEncodedVideoFrameMetadata}} data
or {{RTCEncodedAudioFrameMetadata}} data is populated.

## <dfn enum>RTCEncodedVideoFrameType</dfn> dictionary ## {#RTCEncodedVideoFrameType}
<pre class="idl">
// New enum for video frame types. Will eventually re-use the equivalent defined
Expand Down Expand Up @@ -358,6 +367,8 @@ dictionary RTCEncodedVideoFrameMetadata {
sequence&lt;unsigned long&gt; contributingSources;
long long timestamp; // microseconds
unsigned long rtpTimestamp;
DOMHighResTimeStamp captureTimestamp;
DOMHighResTimeStamp senderCaptureTimeOffset;
DOMString mimeType;
};
</pre>
Expand Down Expand Up @@ -430,6 +441,51 @@ dictionary RTCEncodedVideoFrameMetadata {
The RTP timestamp identifier is an unsigned integer value per [[RFC3550]]
that reflects the sampling instant of the first octet in the RTP data packet.
</p>
</dd>
<dt>
<dfn dict-member>captureTimestamp</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
</dt>
<dd>
<p>
The {{captureTimestamp}} is the timestamp that, the most recent frame (from an RTP packet
originating from this source) delivered to the {{RTCRtpReceiver}}'s {{MediaStreamTrack}}, was
originally captured. Its reference clock is the capture system's NTP clock (same clock used
to generate NTP timestamps for RTCP sender reports on that system).

On populating this member, the user agent MUST run the following steps:
1. If the relevant RTP packet contains the [[RTP-EXT-CAPTURE-TIME|RTP Header Extension for
Absolute Capture Time]], return the value of the
[[RTP-EXT-CAPTURE-TIME#absolute-capture-timestamp|absolute capture timestamp]] field and
abort these steps.
1. Otherwise, if the relevant RTP packet does not contain the RTP Header Extension for
Absolute Capture Time but a previous RTP packet did, return the result of calculating the
absolute capture timestamp according to [[RTP-EXT-CAPTURE-TIME#timestamp-interpolation|
timestamp interpolation]] and abort these steps.
1. Otherwise, return undefined.

</p>
</dd>
<dt>
<dfn dict-member>senderCaptureTimeOffset</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
</dt>
<dd>
<p>
The {{senderCaptureTimeOffset}} is the sender system's estimate of the offset between its own
NTP clock and the capture system's NTP clock, for the same frame that the {{captureTimestamp}} was
originated from.

On populating this member, the user agent MUST run the following steps:
1. If the relevant RTP packet contains the [[RTP-EXT-CAPTURE-TIME|RTP Header Extension for Absolute
Capture Time]] and the estimated [[RTP-EXT-CAPTURE-TIME#estimated-capture-clock-offset|capture
clock offset field]] is present, return the value of the estimated capture clock offset field
and abort these steps.
1. Otherwise, if the relevant RTP packet does not contain the [[RTP-EXT-CAPTURE-TIME|RTP Header
Extension for Absolute Capture Time]]'s [[RTP-EXT-CAPTURE-TIME#estimated-capture-clock-offset|
estimated capture clock offset]] field, but a previous RTP packet did, return the most recent
value that was present and abort these steps.
1. Otherwise, return undefined.

</p>
</dd>
<dt>
<dfn dict-member>mimeType</dfn> <span class="idlMemberType">DOMString</span>
Expand Down Expand Up @@ -611,6 +667,8 @@ dictionary RTCEncodedAudioFrameMetadata {
sequence&lt;unsigned long&gt; contributingSources;
short sequenceNumber;
unsigned long rtpTimestamp;
DOMHighResTimeStamp captureTimestamp;
DOMHighResTimeStamp senderCaptureTimeOffset;
DOMString mimeType;
};
</pre>
Expand Down Expand Up @@ -664,6 +722,51 @@ dictionary RTCEncodedAudioFrameMetadata {
that reflects the sampling instant of the first octet in the RTP data packet.
</p>
</dd>
<dt>
<dfn dict-member>captureTimestamp</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
</dt>
<dd>
<p>
The {{captureTimestamp}} is the timestamp that, the most recent frame (from an RTP packet
originating from this source) delivered to the {{RTCRtpReceiver}}'s {{MediaStreamTrack}}, was
originally captured. Its reference clock is the capture system's NTP clock (same clock used
to generate NTP timestamps for RTCP sender reports on that system).

On populating this member, the user agent MUST run the following steps:
1. If the relevant RTP packet contains the [[RTP-EXT-CAPTURE-TIME|RTP Header Extension for
Absolute Capture Time]], return the value of the
[[RTP-EXT-CAPTURE-TIME#absolute-capture-timestamp|absolute capture timestamp]] field and
abort these steps.
1. Otherwise, if the relevant RTP packet does not contain the RTP Header Extension for
Absolute Capture Time but a previous RTP packet did, return the result of calculating the
absolute capture timestamp according to [[RTP-EXT-CAPTURE-TIME#timestamp-interpolation|
timestamp interpolation]] and abort these steps.
1. Otherwise, return undefined.

</p>
</dd>
<dt>
<dfn dict-member>senderCaptureTimeOffset</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
</dt>
<dd>
<p>
The {{senderCaptureTimeOffset}} is the sender system's estimate of the offset between its own
NTP clock and the capture system's NTP clock, for the same frame that the {{captureTimestamp}} was
originated from.

On populating this member, the user agent MUST run the following steps:
1. If the relevant RTP packet contains the [[RTP-EXT-CAPTURE-TIME|RTP Header Extension for Absolute
Capture Time]] and the estimated [[RTP-EXT-CAPTURE-TIME#estimated-capture-clock-offset|capture
clock offset field]] is present, return the value of the estimated capture clock offset field
and abort these steps.
1. Otherwise, if the relevant RTP packet does not contain the [[RTP-EXT-CAPTURE-TIME|RTP Header
Extension for Absolute Capture Time]]'s [[RTP-EXT-CAPTURE-TIME#estimated-capture-clock-offset|
estimated capture clock offset]] field, but a previous RTP packet did, return the most recent
value that was present and abort these steps.
1. Otherwise, return undefined.

</p>
</dd>
<dt>
<dfn dict-member>mimeType</dfn> <span class="idlMemberType">DOMString</span>
</dt>
Expand Down

0 comments on commit 71a2372

Please sign in to comment.