forked from w3c/picture-in-picture
-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.bs
513 lines (403 loc) · 20.8 KB
/
index.bs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
<pre class='metadata'>
Title: Picture-in-Picture
Shortname: picture-in-picture
Level: 1
Status: CG-DRAFT
ED: https://wicg.github.io/picture-in-picture
Favicon: https://raw.githubusercontent.com/google/material-design-icons/master/action/2x_web/ic_picture_in_picture_alt_black_48dp.png
Group: WICG
Markup Shorthands: markdown yes
Repository: wicg/picture-in-picture
!Web Platform Tests: <a href="https://github.com/web-platform-tests/wpt/tree/master/feature-policy">feature-policy/</a><br/><a href="https://github.com/web-platform-tests/wpt/tree/master/picture-in-picture">picture-in-picture/</a>
Editor: François Beaufort, w3cid 81174, Google LLC https://www.google.com, [email protected]
Editor: Mounir Lamouri, w3cid 45389, Google LLC https://www.google.com, [email protected]
Abstract: This specification intends to provide APIs to allow websites to
Abstract: create a floating video window always on top of other windows so that
Abstract: users may continue consuming media while they interact with other
Abstract: content sites, or applications on their device.
</pre>
<pre class="anchors">
spec: Feature Policy; urlPrefix: https://wicg.github.io/feature-policy/#
type: dfn
text: default allowlist
text: feature name
text: policy-controlled feature
spec: HTML; urlPrefix: https://html.spec.whatwg.org/multipage/
type: dfn
urlPrefix: infrastructure.html
text: in parallel
text: reflect
urlPrefix: interaction.html
text: triggered by user activation
urlPrefix: media.html
text: media element event task source
spec: Remote-Playback; urlPrefix: https://w3c.github.io/remote-playback/#dfn-
type: dfn
text: local playback device
text: local playback state
spec: Page-Visibility; urlPrefix: https://www.w3.org/TR/page-visibility/
type: attribute; text: visibilityState; for: Document; url: dom-visibilitystate
</pre>
<pre class="link-defaults">
spec:dom; type:attribute; text:bubbles
spec:dom; type:dfn; for:NamedNodeMap; text:element
spec:dom; type:interface; text:Document
spec:html; type:attribute; for:HTMLMediaElement; text:readyState
</pre>
# Introduction # {#intro}
<em>This section is non-normative.</em>
Many users want to continue consuming media while they interact with other
content, sites, or applications on their device. A common UI affordance for
this type of activity is Picture-in-Picture (PiP), where the video is contained
in a separate miniature window that is always on top of other windows. This
window stays visible even when user agent is not visible.
Picture-in-Picture is a common platform-level feature among desktop and mobile
OSs.
This specification aims to allow websites to initiate and control this behavior
by exposing the following sets of properties to the API:
* Notify the website when it enters and leaves Picture-in-Picture mode.
* Allow the website to trigger Picture-in-Picture mode via a user gesture on a
video element.
* Allow the website to know the size of the Picture-in-Picture window and notify
the website when it changes.
* Allow the website to exit Picture-in-Picture mode.
* Allow the website to check if Picture-in-Picture mode can be triggered.
The proposed Picture-in-Picture API is very similar to [[Fullscreen]] as they
have similar properties. The API only applies to
{{HTMLVideoElement}} at the moment but is meant to be
extensible.
# Examples # {#examples}
## Add a custom Picture-in-Picture button ## {#example-add-custom-pip-button}
```html
<video id="video" src="https://example.com/file.mp4"></video>
<button id="togglePipButton"></button>
<script>
const video = document.getElementById('video');
const togglePipButton = document.getElementById('togglePipButton');
// Hide button if Picture-in-Picture is not supported or disabled.
togglePipButton.hidden = !document.pictureInPictureEnabled ||
video.disablePictureInPicture;
togglePipButton.addEventListener('click', function() {
// If there is no element in Picture-in-Picture yet, let's request
// Picture-in-Picture for the video, otherwise leave it.
if (!document.pictureInPictureElement) {
video.requestPictureInPicture()
.catch(error => {
// Video failed to enter Picture-in-Picture mode.
});
} else {
document.exitPictureInPicture()
.catch(error => {
// Video failed to leave Picture-in-Picture mode.
});
}
});
</script>
```
## Monitor video Picture-in-Picture changes ## {#example-monitor-video-pip-changes}
```html
<video id="video" src="https://example.com/file.mp4"></video>
<script>
const video = document.getElementById('video');
video.addEventListener('enterpictureinpicture', function(event) {
// Video entered Picture-in-Picture mode.
const pipWindow = event.pictureInPictureWindow;
console.log('Picture-in-Picture window width: ' + pipWindow.width);
console.log('Picture-in-Picture window height: ' + pipWindow.height);
});
video.addEventListener('leavepictureinpicture', function() {
// Video left Picture-in-Picture mode.
});
</script>
```
## Update video size based on Picture-in-Picture window size changes ## {#example-update-video-on-window-size-changes}
```html
<video id="video" src="https://example.com/file.mp4"></video>
<button id="pipButton"></button>
<script>
let pipWindow;
const video = document.getElementById('video');
const pipButton = document.getElementById('pipButton');
pipButton.addEventListener('click', function() {
video.requestPictureInPicture()
.catch(error => {
// Video failed to enter Picture-in-Picture mode.
});
});
video.addEventListener('enterpictureinpicture', function(event) {
// Video entered Picture-in-Picture mode.
pipWindow = event.pictureInPictureWindow;
updateVideoSize(pipWindow.width, pipWindow.height);
pipWindow.addEventListener('resize', onPipWindowResize);
});
video.addEventListener('leavepictureinpicture', function() {
// Video left Picture-in-Picture mode.
pipWindow.removeEventListener('resize', onPipWindowResize);
});
function onPipWindowResize(event) {
// Picture-in-Picture window has been resized.
updateVideoSize(event.target.width, event.target.height);
}
function updateVideoSize(width, height) {
// TODO: Update video size based on pip window width and height.
}
</script>
```
# Concepts # {#concepts}
## Request Picture-in-Picture ## {#request-pip}
When the <dfn>request Picture-in-Picture algorithm</dfn> with |video|,
|userActivationRequired| and |playingRequired| is invoked, the user agent MUST
run the following steps:
1. If <a>Picture-in-Picture support</a> is `false`, throw a
{{NotSupportedError}} and abort these steps.
2. If the document is not allowed to use the <a>policy-controlled feature</a>
named `"picture-in-picture"`, throw a {{SecurityError}} and abort these
steps.
3. If |video|'s {{readyState}} attribute is {{HAVE_NOTHING}}, throw a
{{InvalidStateError}} and abort these steps.
4. If |video| has no video track, throw a {{InvalidStateError}} and abort
these steps.
5. OPTIONALLY, if the {{disablePictureInPicture}} attribute is present on
|video|, throw an {{InvalidStateError}} and abort these steps.
6. If |userActivationRequired| is `true` and the algorithm is not
<a>triggered by user activation</a>, throw a {{NotAllowedError}} and
abort these steps.
7. If |video| is {{pictureInPictureElement}}, abort these steps.
8. If |playingRequired| is `true` and |video| is {{paused}}, abort these steps.
9. Set {{pictureInPictureElement}} to |video|.
10. Let <dfn>Picture-in-Picture window</dfn> be a new instance of
{{PictureInPictureWindow}} associated with {{pictureInPictureElement}}.
11. <a>Queue a task</a> to <a>fire an event</a> with the name
{{enterpictureinpicture}} at the |video| with its {{bubbles}} attribute
initialized to `true` and its {{pictureInPictureWindow}} attribute
initialized to <a>Picture-in-Picture window</a>.
It is RECOMMENDED that video frames are not rendered in the page and in the
Picture-in-Picture window at the same time but if they are, they MUST be kept
in sync.
When a video is played in Picture-in-Picture mode, the states SHOULD transition
as if it was played inline. That means that the events SHOULD fire at the same
time, calling methods SHOULD have the same behaviour, etc. However, the user
agent MAY transition out of Picture-in-Picture when the video element enters a
state that is considered not compatible with Picture-in-Picture.
Styles applied to |video| (such as opacity, visibility, transform, etc.) MUST
NOT apply in the Picture-in-Picture window. Its aspect ratio is based on the
video size.
It is also RECOMMENDED that the Picture-in-Picture window has a maximum and
minimum size. For example, it could be restricted to be between a quarter and
a half of one dimension of the screen.
## Exit Picture-in-Picture ## {#exit-pip}
When the <dfn>exit Picture-in-Picture algorithm</dfn> is invoked,
the user agent MUST run the following steps:
1. If {{pictureInPictureElement}} is `null`, throw a {{InvalidStateError}} and
abort these steps.
2. Run the <a>close window algorithm</a> with the <a>Picture-in-Picture
window</a> associated with {{pictureInPictureElement}}.
3. Unset {{pictureInPictureElement}}.
4. <a>Queue a task</a> to <a>fire an event</a> with the name
{{leavepictureinpicture}} at the |video| with its {{bubbles}} attribute
initialized to `true`.
It is NOT RECOMMENDED that the video playback state changes when the <a>exit
Picture-in-Picture algorithm</a> is invoked. The website SHOULD be in control
of the experience if it is website initiated. However, the user agent MAY expose
Picture-in-Picture window controls that change video playback state (e.g.,
pause).
As one of the <a>unloading document cleanup steps</a>, run the <a>exit
Picture-in-Picture algorithm</a>.
## Disable Picture-in-Picture ## {#disable-pip}
Some pages may want to disable Picture-in-Picture mode for a video element; for
example, they may want to prevent the user agent from suggesting a
Picture-in-Picture context menu or to request Picture-in-Picture automatically
in some cases. To support these use cases, a new {{disablePictureInPicture}}
attribute is added to the list of content attributes for video elements.
The {{disablePictureInPicture}} IDL attribute MUST <a>reflect</a> the content
attribute of the same name.
If the {{disablePictureInPicture}} attribute is present on the video element,
the user agent SHOULD NOT play the video element in Picture-in-Picture or
present any UI to do so.
When the {{disablePictureInPicture}} attribute is added to a |video| element,
the user agent SHOULD run these steps:
1. Reject any pending promises returned by the {{requestPictureInPicture()}}
method with {{InvalidStateError}}.
2. If |video| is {{pictureInPictureElement}}, run the <a>exit
Picture-in-Picture algorithm</a>.
## Auto Picture-in-Picture ## {#auto-pip}
Some pages may want to automatically enter and leave Picture-in-Picture for
a video element; for example, video meeting web apps would benefit from
some automatic Picture-in-Picture behavior when the user switches back and forth
between the web app and other applications/tabs.
To support these use cases, a new {{autoPictureInPicture}} attribute is
added to the list of content attributes for video elements.
The {{autoPictureInPicture}} IDL attribute MUST <a>reflect</a> the content
attribute of the same name.
The <dfn>autoPictureInPictureElement</dfn> is the video element, among all
video elements with the {{autoPictureInPicture}} attribute currently set,
whose {{autoPictureInPicture}} attribute was set most recently. Note that the
<a>autoPictureInPictureElement</a> can have the {{disablePictureInPicture}}
attribute set. In that case, as expected, it won't be allowed to enter
Picture-in-Picture mode in the <a>request Picture-in-Picture algorithm</a>.
When the visibility state of a <a>top-level browsing context</a> value
switches from "visible" to "hidden", the user agent MAY run these steps:
1. If <a>autoPictureInPictureElement</a> is `null`, abort these steps.
2. If {{pictureInPictureElement}} is set, abort these steps.
3. Let |video| be <a>autoPictureInPictureElement</a>.
4. Let |userActivationRequired| be `false`.
5. Let |playingRequired| be `true`.
6. Run the <a>request Picture-in-Picture algorithm</a> with |video|,
|userActivationRequired|, and |playingRequired|.
When the visibility state of a <a>top-level browsing context</a> value
switches from "hidden" to "visible", the user agent MAY run these steps:
1. If <a>autoPictureInPictureElement</a> is `null`, abort these steps.
2. If {{pictureInPictureElement}} is <a>autoPictureInPictureElement</a>, run
the <a>exit Picture-in-Picture algorithm</a>.
## Interaction with Remote Playback ## {#remote-playback}
The [[Remote-Playback]] specification defines a <a>local playback device</a>
and a <a>local playback state</a>. For the purpose of Picture-in-Picture, the
playback is local and regardless of whether it is played in page or in
Picture-in-Picture.
## Interaction with Media Session ## {#media-session}
The API will have to be used with the [[MediaSession]] API for customizing the
available controls on the Picture-in-Picture window.
## Interaction with Page Visibility ## {#page-visibility}
When {{pictureInPictureElement}} is set, the Picture-in-Picture window MUST
be visible, even when the <a>Document</a> is not in focus or hidden. The user
agent SHOULD provide a way for users to manually close the Picture-in-Picture
window.
The [[Page-Visibility]] specification defines a {{Document/visibilityState}}
attribute used to determine the visibility state of a <a>top-level browsing
context</a>. The Picture-in-Picture window visibility MUST NOT be taken into
account when determining the value returned by {{Document/visibilityState}}.
## One Picture-in-Picture window ## {#one-pip-window}
Operating systems with a Picture-in-Picture API usually restrict
Picture-in-Picture mode to only one window. Whether only one window is allowed
in Picture-in-Picture mode will be left to the implementation and the platform.
However, because of the one Picture-in-Picture window limitation, the
specification assumes that a given {{Document}} can only have one
Picture-in-Picture window.
What happens when there is a Picture-in-Picture request while a window is
already in Picture-in-Picture will be left as an implementation detail: the
current Picture-in-Picture window could be closed, the Picture-in-Picture
request could be rejected or even two Picture-in-Picture windows could be
created. Regardless, the User Agent will have to fire the appropriate events
in order to notify the website of the Picture-in-Picture status changes.
# API # {#api}
## Extensions to <code>HTMLVideoElement</code> ## {#htmlvideoelement-extensions}
<xmp class="idl">
partial interface HTMLVideoElement {
[NewObject] Promise<PictureInPictureWindow> requestPictureInPicture();
attribute EventHandler onenterpictureinpicture;
attribute EventHandler onleavepictureinpicture;
[CEReactions] attribute boolean autoPictureInPicture;
[CEReactions] attribute boolean disablePictureInPicture;
};
</xmp>
The {{requestPictureInPicture()}} method, when invoked, MUST
return <a>a new promise</a> |promise| and run the following steps <a>in
parallel</a>:
1. Let |video| be the video element on which the method was invoked.
2. Let |userActivationRequired| be `true` if {{pictureInPictureElement}} is
`null`. It is `false` otherwise.
3. Let |playingRequired| be `false`.
4. Run the <a>request Picture-in-Picture algorithm</a> with |video|,
|userActivationRequired|, and |playingRequired|.
5. If the previous step threw an exception, reject |promise| with that
exception and abort these steps.
6. Return |promise| with the <a>Picture-in-Picture window</a> associated with
{{pictureInPictureElement}}.
## Extensions to <code>Document</code> ## {#document-extensions}
<xmp class="idl">
partial interface Document {
readonly attribute boolean pictureInPictureEnabled;
[NewObject] Promise<void> exitPictureInPicture();
};
</xmp>
The {{pictureInPictureEnabled}} attribute's getter must return `true` if
<a>Picture-in-Picture support</a> is `true` and the <a>context object</a> is
<a>allowed to use</a> the feature indicated by attribute name
`picture-in-picture`, and `false` otherwise.
<dfn>Picture-in-Picture support</dfn> is `false` if there's a user preference
that disables it or a platform limitation. It is `true` otherwise.
The {{exitPictureInPicture()}} method, when invoked, MUST
return <a>a new promise</a> |promise| and run the following steps <a>in
parallel</a>:
1. Run the <a>exit Picture-in-Picture algorithm</a>.
2. If the previous step threw an exception, reject |promise| with that
exception and abort these steps.
3. Return |promise|.
## Extension to <code>DocumentOrShadowRoot</code> ## {#documentorshadowroot-extension}
<xmp class="idl">
partial interface mixin DocumentOrShadowRoot {
readonly attribute Element? pictureInPictureElement;
};
</xmp>
The {{pictureInPictureElement}} attribute's getter must run these steps:
1. If the <a>context object</a> is not <a>connected</a>, return `null` and abort
these steps.
2. Let |candidate| be the result of <a>retargeting</a> Picture-in-Picture
element against the <a>context object</a>.
3. If |candidate| and the <a>context object</a> are in the same <a>tree</a>,
return |candidate| and abort these steps.
4. Return `null`.
## Interface <code>PictureInPictureWindow</code> ## {#interface-picture-in-picture-window}
<xmp class="idl">
interface PictureInPictureWindow : EventTarget {
readonly attribute long width;
readonly attribute long height;
attribute EventHandler onresize;
};
</xmp>
A {{PictureInPictureWindow}} instance represents a <a>Picture-in-Picture
window</a> associated with an {{HTMLVideoElement}}. When instantiated, an
instance of {{PictureInPictureWindow}} has its |state| set to |opened|.
When the <dfn>close window algorithm</dfn> with an instance of
{{PictureInPictureWindow}} is invoked, its |state| is set to |closed|.
The {{width}} attribute MUST return the width in <a lt=px value>CSS pixels</a> of the
<a>Picture-in-Picture window</a> associated with {{pictureInPictureElement}} if
the |state| is |opened|. Otherwise, it MUST return 0.
The {{height}} attribute MUST return the height in <a lt=px value>CSS pixels</a> of the
<a>Picture-in-Picture window</a> associated with {{pictureInPictureElement}} if
the |state| is |opened|. Otherwise, it MUST return 0.
When the size of the <a>Picture-in-Picture window</a> associated with
{{pictureInPictureElement}} changes, the user agent MUST <a>queue a task</a> to
<a>fire an event</a> with the name {{resize}} at {{pictureInPictureElement}}.
## Event types ## {#event-types}
: <dfn event for="HTMLVideoElement">`enterpictureinpicture`</dfn>
:: Fired on a {{HTMLVideoElement}} when it enters Picture-in-Picture. It uses
the {{EnterPictureInPictureEvent}} interface.
<xmp class="idl">
[
Constructor(DOMString type, EnterPictureInPictureEventInit eventInitDict),
Exposed=Window
]
interface EnterPictureInPictureEvent : Event {
[SameObject] readonly attribute PictureInPictureWindow pictureInPictureWindow;
};
dictionary EnterPictureInPictureEventInit : EventInit {
required PictureInPictureWindow pictureInPictureWindow;
};
</xmp>
: <dfn event for="HTMLVideoElement">`leavepictureinpicture`</dfn>
:: Fired on a {{HTMLVideoElement}} when it leaves Picture-in-Picture mode.
: <dfn event for="PictureInPictureWindow">`resize`</dfn>
:: Fired on a {{PictureInPictureWindow}} when it changes size.
## Task source ## {#task-source}
The <a>task source</a> for all the tasks queued in this specification is the
<a>media element event task source</a> of the video element in question.
## CSS pseudo-class ## {#css-pseudo-class}
The `:picture-in-picture` <a>pseudo-class</a> MUST match the Picture-in-Picture
element. It is different from the {{pictureInPictureElement}} as it does NOT
apply to the shadow host chain.
# Security considerations # {#security-considerations}
<em>This section is non-normative.</em>
The API applies only to {{HTMLVideoElement}} in order to start on a minimal
viable product that has limited security issues. Later versions of this
specification may allow PIP-ing arbitrary HTML content.
## Feature Policy ## {#feature-policy}
This specification defines a <a>policy-controlled feature</a> that controls
whether the <a>request Picture-in-Picture algorithm</a> may return a
{{SecurityError}} and whether {{pictureInPictureEnabled}} is `true` or `false`.
The <a>feature name</a> for this feature is `"picture-in-picture"`.
The <a>default allowlist</a> for this feature is `*`.
# Acknowledgments # {#acknowledgments}
Thanks to Jennifer Apacible, Zouhir Chahoud, Marcos Cáceres, Philip Jägenstedt,
Jeremy Jones, Chris Needham, Jer Noble, Justin Uberti, Yoav Weiss, and Eckhart
Wörner for their contributions to this document.