You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The **`getMetadata()`** method of the {{domxref("RTCEncodedAudioFrame")}} interface returns an object containing the metadata associated with the frame.
12
12
13
-
This includes information about the frame, including the audio encoding used, the synchronization source and contributing sources, and the sequence number (for incoming frames).
13
+
This includes information about the frame, such as the audio encoding used, the synchronization source and contributing sources, and the sequence number (for incoming frames).
14
14
15
15
## Syntax
16
16
@@ -27,27 +27,40 @@ None.
27
27
An object with the following properties:
28
28
29
29
-`audioLevel`
30
-
- : A number representing the audio level of this frame. The value is between 0 and 1 inclusive (linear), where 1.0 represents 0 dBov ([decibels relative to full scale (DBFS)](https://en.wikipedia.org/wiki/DBFS)), 0 represents silence, and 0.5 represents approximately 6 dB SPL change in the [sound pressure level](https://en.wikipedia.org/wiki/Sound_pressure#Sound_pressure_level) from 0 dBov. The value is converted from the -127 to 0 range specified in [RFC6464](https://www.rfc-editor.org/rfc/rfc6464) via the equation `10^(-rfc_level/20)`. If the RFC6464 header extension is not present in the received packets of the frame, `audioLevel` will be `undefined`.
31
-
-`synchronizationSource`
32
-
- : A positive integer value indicating synchronization source ("ssrc") of the stream of RTP packets that are described by this frame.
33
-
A source might be something like a microphone, or a mixer application that combines multiple sources.
34
-
All packets from the same source share the same time source and sequence space, and so can be ordered relative to each other.
35
-
Note that two frames with the same value refer to the same source.
36
-
-`payloadType`
37
-
- : A positive integer value in the range from 0 to 127 that describes the format of the RTP payload.
38
-
The mappings of values to formats is defined in RFC3550, and more specifically [Section 6: Payload Type Definitions](https://www.rfc-editor.org/rfc/rfc3551#section-6) of RFC3551.
30
+
- : A number representing the audio level of this frame.
31
+
The value is between 0 and 1 inclusive (linear), where 1.0 represents 0 dBov ([decibels relative to full scale (DBFS)](https://en.wikipedia.org/wiki/DBFS)), 0 represents silence, and 0.5 represents approximately 6 dB SPL change in the [sound pressure level](https://en.wikipedia.org/wiki/Sound_pressure#Sound_pressure_level) from 0 dBov.
32
+
The value is converted from the -127 to 0 range specified in [RFC6464](https://www.rfc-editor.org/rfc/rfc6464) via the equation `10^(-rfc_level/20)`.
33
+
If the RFC6464 header extension is not present in the received packets of the frame, `audioLevel` will be `undefined`.
34
+
-`captureTime`
35
+
- : A {{domxref("DOMHighResTimeStamp")}} indicating the capture time of the frame relative to {{domxref("Performance.timeOrigin")}}.
39
36
-`contributingSources`
40
37
- : An {{jsxref("Array")}} of sources (ssrc) that have contributed to the frame.
41
38
Consider the case of a conferencing application that combines audio from multiple users.
42
39
The `synchronizationSource` would include the ssrc of the application, while `contributingSources` would include the ssrc values of all the individual audio sources.
40
+
-`mimeType`
41
+
- : A string containing the {{glossary("MIME type")}} of the codec used, such as "audio/opus".
42
+
-`payloadType`
43
+
- : A positive integer value in the range from 0 to 127 that describes the format of the RTP payload.
44
+
The mappings of values to formats is defined in {{rfc("3550")}}, and more specifically [Section 6: Payload Type Definitions](https://www.rfc-editor.org/rfc/rfc3551#section-6) of {{rfc("3551")}}.
45
+
-`receiveTime`
46
+
- : A {{domxref("DOMHighResTimeStamp")}} indicating the timestamp of the last received packet of an incoming frame (from an {{domxref("RTCRtpReceiver")}}) used to produce this media frame, relative to {{domxref("Performance.timeOrigin")}}.
47
+
-`rtpTimestamp`
48
+
- : A positive integer that reflects the sampling instant of the first octet in the RTP data packet (see {{rfc("3550")}}).
43
49
-`sequenceNumber`
44
50
- : The sequence number of an incoming audio frame (not used for outgoing frames) that can be used for reconstructing the original send-order of frames.
45
51
This is number between 0 and 32767.
46
52
Note that while numbers are allocated sequentially when sent, they will overflow at 32767 and restart back at 0.
47
53
Therefore to compare two frame sequence numbers, in order to determine whether one is assumed to be after another, you must use [serial number arithmetic](https://en.wikipedia.org/wiki/Serial_number_arithmetic). <!-- [RFC1982] -->
54
+
-`synchronizationSource`
55
+
- : A positive integer value indicating the synchronization source ("ssrc") of the stream of RTP packets that are described by this frame.
56
+
A source might be something like a microphone, or a mixer application that combines multiple sources.
57
+
All packets from the same source share the same time source and sequence space, and so can be ordered relative to each other.
58
+
Note that two frames with the same value refer to the same source.
48
59
49
60
## Examples
50
61
62
+
### Getting frame metadata
63
+
51
64
This example [WebRTC encoded transform](/en-US/docs/Web/API/WebRTC_API/Using_Encoded_Transforms) implementation shows how you might get the frame metadata in a `transform()` function and log it.
52
65
53
66
```js
@@ -73,8 +86,13 @@ Note that there are no contributing sources because there is just one source, an
Copy file name to clipboardExpand all lines: files/en-us/web/api/rtcencodedaudioframe/index.md
+7Lines changed: 7 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,6 +12,11 @@ The **`RTCEncodedAudioFrame`** of the [WebRTC API](/en-US/docs/Web/API/WebRTC_AP
12
12
The interface provides methods and properties to get metadata about the frame, allowing its format and order in the sequence of frames to be determined.
13
13
The `data` property gives access to the encoded frame data as a buffer, which might be encrypted, or otherwise modified by a transform.
@@ -26,6 +31,8 @@ The `data` property gives access to the encoded frame data as a buffer, which mi
26
31
27
32
## Examples
28
33
34
+
### Transforming an encoded audio frame
35
+
29
36
This code snippet shows a handler for the `rtctransform` event in a {{domxref("Worker")}} that implements a {{domxref("TransformStream")}}, and pipes encoded frames through it from the `event.transformer.readable` to `event.transformer.writable` (`event.transformer` is a {{domxref("RTCRtpScriptTransformer")}}, the worker-side counterpart of {{domxref("RTCRtpScriptTransform")}}).
30
37
31
38
If the transformer is inserted into an audio stream, the `transform()` method is called with a `RTCEncodedAudioFrame` whenever a new frame is enqueued on `event.transformer.readable`.
The **`RTCEncodedAudioFrame()`** constructor creates a new and fully independent {{domxref("RTCEncodedAudioFrame")}} object.
12
+
13
+
The new object is a deep clone of the original object data and metadata, with any metadata specified in the options parameter overwriting the copied values.
14
+
15
+
## Syntax
16
+
17
+
```js-nolint
18
+
19
+
new RTCEncodedAudioFrame(originalFrame);
20
+
new RTCEncodedAudioFrame(originalFrame, options);
21
+
```
22
+
23
+
### Parameters
24
+
25
+
-`originalFrame`
26
+
- : The frame to be copied.
27
+
-`options` {{optional_inline}}
28
+
- : This is an object with the following property:
29
+
-`metadata` {{optional_inline}}
30
+
- : An object setting the frame metadata.
31
+
This is an object with the same properties as the object returned by {{DOMxRef("RTCEncodedAudioFrame.getMetadata()")}}.
32
+
33
+
### Exceptions
34
+
35
+
- {{jsxref("TypeError")}}
36
+
- The source buffer is detached.
37
+
- {{jsxref("RangeError")}}
38
+
- The allocation is too large.
39
+
40
+
## Examples
41
+
42
+
### Cloning a frame with modified metadata
43
+
44
+
This snippet shows how you might copy a frame and modify its metadata.
45
+
In this case we just update the capture time.
46
+
47
+
```js
48
+
// Frame is an incoming RTCEncodedAudioFrame
49
+
frame.getMetadata();
50
+
51
+
constnewFrame=newRTCEncodedAudioFrame(frame, {
52
+
metadata: {
53
+
captureTime:frame.metadata.captureTime+3,
54
+
},
55
+
});
56
+
```
57
+
58
+
This kind of modification might be useful if you need to create multiple outgoing frames from a single incoming frame; for example, in order to relay media to another peer on the network.
59
+
Generally you will not need to modify the metadata for a frame.
The **`getMetadata()`** method of the {{domxref("RTCEncodedVideoFrame")}} interface returns an object containing the metadata associated with the frame.
12
12
13
-
This includes information about the frame, including its size, video encoding, other frames needed to construct a full image, timestamp, and other information.
13
+
This includes information about the frame, such as its size, video encoding, other frames needed to construct a full image, timestamp, and other information.
14
14
15
15
## Syntax
16
16
@@ -26,42 +26,45 @@ None.
26
26
27
27
An object with the following properties:
28
28
29
-
-`frameId`
30
-
- : A positive integer value indicating the id of this frame.
29
+
-`contributingSources`
30
+
- : An {{jsxref("Array")}} of sources (ssrc) that have contributed to the frame.
31
+
Consider the case of a conferencing application that combines the audio and video from multiple users.
32
+
The `synchronizationSource` would include the ssrc of the application, while `contributingSources` would include the ssrc values of all the individual video and audio sources.
31
33
-`dependencies`
32
34
- : An {{jsxref("Array")}} of positive integers indicating the frameIds of frames on which this frame depends.
33
35
For a key frame this will be empty, as a key frame contains all the information it needs to construct the image.
34
36
For a delta frame this will list all the frames needed to render this frame.
35
37
The type of frame can be determined using {{domxref("RTCEncodedVideoFrame.type")}}.
36
-
-`width`
37
-
- : A positive integer indicating the width of the frame.
38
-
The maximum value is 65535.
38
+
-`frameId`
39
+
- : A positive integer value indicating the id of this frame.
39
40
-`height`
40
41
- : A positive integer indicating the height of the frame.
41
42
The maximum value is 65535.
43
+
-`mimeType`
44
+
- : A string containing the {{glossary("MIME type")}} of the codec used, such as "video/VP8".
45
+
-`payloadType`
46
+
- : A positive integer value in the range from 0 to 127 that describes the format of the RTP payload.
47
+
The mappings of values to formats is defined in RFC3550.
48
+
-`receiveTime`
49
+
- : A {{domxref("DOMHighResTimeStamp")}} indicating the timestamp of the last received packet of an incoming frame (from an {{domxref("RTCRtpReceiver")}}) used to produce this media frame, relative to {{domxref("Performance.timeOrigin")}}.
50
+
-`rtpTimestamp`
51
+
- : A positive integer that reflects the sampling instant of the first octet in the RTP data packet (see {{rfc("3550")}}).
42
52
-`spatialIndex`
43
53
- : A positive integer indicating the spatial index of the frame.
44
54
Some codecs allow generation of layers of frames with different layers of resolutions.
45
55
Frames in higher layers can be selectively dropped in order to reduce bit rate when needed, while maintaining acceptable video quality.
46
-
-`temporalIndex`
47
-
- : A positive integer indicating the temporal index of the frame.
48
-
Some codecs group frames in layers, based on whether dropping the a frame will prevent others from being decoded.
49
-
Frames in higher layers can be selectively dropped in order to reduce bit rate when needed, while maintaining acceptable video quality.
50
56
-`synchronizationSource`
51
57
- : A positive integer value indicating synchronization source ("ssrc") of the stream of RTP packets that are described by this encoded video frame.
52
58
A source might be something like a camera or microphone, or some kind of mixer app that combines multiple sources.
53
59
All packets from the same source share the same time source and sequence space, and so can be ordered relative to each other.
54
60
Note two frames with the same value refer to the same source (for more information see [`RTCInboundRtpStreamStats.ssrc`](/en-US/docs/Web/API/RTCInboundRtpStreamStats/ssrc)).
55
-
-`payloadType`
56
-
- : A positive integer value in the range from 0 to 127 that describes the format of the RTP payload.
57
-
The mappings of values to formats is defined in RFC3550.
58
-
-`contributingSources`
59
-
- : An {{jsxref("Array")}} of sources (ssrc) that have contributed to the frame.
60
-
Consider the case of a conferencing application that combines the audio and video from multiple users.
61
-
The `synchronizationSource` would include the ssrc of the application, while `contributingSources` would include the ssrc values of all the individual video and audio sources.
62
-
-`timestamp`
63
-
- : The [media presentation timestamp (PTS)](https://en.wikipedia.org/wiki/Presentation_timestamp) in microseconds of raw frame, matching the timestamp for raw frames which correspond to this frame.
64
-
This is used to synchronize separate video, audio, subtitle and other streams belonging to the same presentation.
61
+
-`temporalIndex`
62
+
- : A positive integer indicating the temporal index of the frame.
63
+
Some codecs group frames in layers, based on whether dropping the a frame will prevent others from being decoded.
64
+
Frames in higher layers can be selectively dropped in order to reduce bit rate when needed, while maintaining acceptable video quality.
65
+
-`width`
66
+
- : A positive integer indicating the width of the frame.
67
+
The maximum value is 65535.
65
68
66
69
## Examples
67
70
@@ -91,14 +94,16 @@ Note that there are no contributing sources because there is just one source.
The **`RTCEncodedVideoFrame`** of the [WebRTC API](/en-US/docs/Web/API/WebRTC_API) represents an encoded video frame in the WebRTC receiver or sender pipeline, which may be modified using a [WebRTC Encoded Transform](/en-US/docs/Web/API/WebRTC_API/Using_Encoded_Transforms).
- : Copy constructor. Creates a new and independent `RTCEncodedVideoFrame` object from another frame, optionally overwriting some of the copied metadata.
@@ -37,6 +42,8 @@ The {{domxref("RTCEncodedVideoFrame.data", "data")}} property provides access to
37
42
38
43
## Examples
39
44
45
+
### Transforming an encoded video frame
46
+
40
47
This code snippet shows a handler for the `rtctransform` event in a {{domxref("Worker")}} that implements a {{domxref("TransformStream")}}, and pipes encoded frames through it from the `event.transformer.readable` to `event.transformer.writable` (`event.transformer` is a {{domxref("RTCRtpScriptTransformer")}}, the worker-side counterpart of {{domxref("RTCRtpScriptTransform")}}).
41
48
42
49
If the transformer is inserted into a video stream, the `transform()` method is called with a `RTCEncodedVideoFrame` whenever a new frame is enqueued on `event.transformer.readable`.
The **`RTCEncodedVideoFrame()`** constructor creates a new and fully independent {{domxref("RTCEncodedVideoFrame")}} object.
12
+
13
+
The new object is a deep clone of the original object data and metadata, with any metadata specified in the options parameter overwriting the copied values.
14
+
15
+
## Syntax
16
+
17
+
```js-nolint
18
+
19
+
new RTCEncodedVideoFrame(originalFrame);
20
+
new RTCEncodedVideoFrame(originalFrame, options);
21
+
```
22
+
23
+
### Parameters
24
+
25
+
-`originalFrame`
26
+
- : The frame to be copied.
27
+
-`options` {{optional_inline}}
28
+
- : This is an object with the following property:
29
+
-`metadata` {{optional_inline}}
30
+
- : An object setting the frame metadata.
31
+
This is an object with the same properties as the object returned by {{DOMxRef("RTCEncodedVideoFrame.getMetadata()")}}.
32
+
33
+
### Exceptions
34
+
35
+
- {{jsxref("TypeError")}}
36
+
- The source buffer is detached.
37
+
- {{jsxref("RangeError")}}
38
+
- The allocation is too large.
39
+
40
+
## Examples
41
+
42
+
### Cloning a frame with modified metadata
43
+
44
+
This snippet shows how you might copy a frame and modify its metadata.
45
+
In this case we just update the capture time.
46
+
47
+
```js
48
+
// Frame is an incoming RTCEncodedVideoFrame
49
+
frame.getMetadata();
50
+
51
+
constnewFrame=newRTCEncodedVideoFrame(frame, {
52
+
metadata: {
53
+
captureTime:frame.metadata.captureTime+3,
54
+
},
55
+
});
56
+
```
57
+
58
+
This kind of modification might be useful if you need to create multiple outgoing frames from a single incoming frame; for example, in order to relay media to another peer on the network.
59
+
Generally you will not need to modify the metadata for a frame.
0 commit comments