-
Notifications
You must be signed in to change notification settings - Fork 22
/
Copy pathapplication.bs
2350 lines (1835 loc) · 100 KB
/
application.bs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<pre class='metadata'>
Title: Open Screen Application Protocol
Shortname: openscreenprotocol-application
Level: None
Status: w3c/ED
ED: https://w3c.github.io/openscreenprotocol/application.html
Canonical URL: ED
Editor: Mark Foltz, Google, https://github.com/mfoltzgoogle, w3cid 68454
Repository: w3c/openscreenprotocol
Abstract: The Open Screen Application Protocol allows user agents to implement the
[[PRESENTATION-API|Presentation API]] and the
[[REMOTE-PLAYBACK|Remote Playback API]] in an interoperable fashion.
Group: secondscreenwg
Markup Shorthands: markdown yes, dfn yes, idl yes, markup yes
</pre>
<pre class="anchors">
url: https://html.spec.whatwg.org/multipage/media.html#concept-mediaerror-code; type: dfn; spec: html
text: media error code
urlPrefix: https://html.spec.whatwg.org/multipage/media.html#; type: dfn; spec: html
text: official playback position
text: poster frame
text: timeline offset
text: media resource
text: media timeline
urlPrefix: https://www.w3.org/TR/presentation-api/#dfn-; type: dfn; spec: PRESENTATION-API
text: available presentation display
text: controlling user agent
text: presentation; url: receiving-browsing-context
text: presentation identifier
text: presentation request url
text: receiving user agent
urlPrefix: https://w3c.github.io/remote-playback/#dfn-; type: dfn; spec: REMOTE-PLAYBACK
text: availability sources set
text: compatible remote playback device
text: initiate remote playback
text: media element state
text: remote playback devices
text: remote playback source
url: https://datatracker.ietf.org/doc/html/rfc9000#name-variable-length-integer-enc; type: dfn; spec: RFC9000; text: Variable-Length Integer Encoding
url: https://datatracker.ietf.org/doc/html/rfc9000#name-variable-length-integer-enc; type: dfn; spec: RFC9000; text: variable-length integer
url: https://tools.ietf.org/html/rfc5646#section-2; type: dfn; spec: RFC5646; text: language tag
url: https://tools.ietf.org/html/rfc4122#section-4.4; type: dfn; spec: RFC4122; text: UUID
url: https://tools.ietf.org/html/rfc6381#section-3; type: dfn; spec: RFC6381; text: codecs parameter
url: https://tools.ietf.org/html/rfc8610#section-3; type: dfn; spec: RFC8610; text: concise data definition language
</pre>
Introduction {#introduction}
============================
The Open Screen Application Protocol connects browsers to devices capable of
rendering Web content for a shared audience. Typically, these are devices like
Internet-connected TVs, HDMI dongles, and smart speakers.
The Open Screen Application Protocol is intended to be supported by a variety of
connection technologies between browsers and devices. There are specific
requirements to allow browsers and devices to discover and authenticate to one
another, but these requirements may be met by multiple possible implementations.
To maximize interoperability, browsers and devices should support the Open
Screen Network Protocol, which provides a way for browsers and devices to
discover, connect, and authenticate each other on a local area network.
The Open Screen Application Protocol allows a browser to present a URL, initiate
remote playback of an HTML <l spec=html>[=media element=]</l>, and stream media
data to another device.
The Open Screen Application Protocol is intended to be extensible, so that
additional capabilities can be added over time. This may include additions to
existing Web APIs or new Web APIs.
The accompanying [explainer](https://w3c.github.io/openscreenprotocol/explainer.html)
provides more background on the protocol.
Terminology {#terminology}
--------------------------
An <dfn lt="open screen protocol agent|osp agent">Open Screen Protocol
agent</dfn> (or OSP agent) is any implementation of this protocol (browser,
display, speaker, or other software).
Some OSP agents support the [[PRESENTATION-API|Presentation API]]. The
API allows a [=controlling user agent=] to initiate presentation of Web content
on another device. We call this agent a <dfn>controller</dfn> for short. A
[=receiving user agent=] is responsible for rendering the Web content, which we
will call a <dfn>receiver</dfn> for short. The the Web content itself is called
a [=presentation=].
Some OSP agents also support the [[REMOTE-PLAYBACK|Remote Playback API]]. That
API allows an agent to render a <l spec="html">[=media element=]</l> on a
[=remote playback device=]. In this document, we refer to it as a [=receiver=]
because it is shorter and keeps terminology consistent between presentations and
remote playbacks. Similarly, we use the term [=controller=] to refer to the
agent that starts, terminates, and controls the remote playback.
For media streaming, we refer to an OSP agent that sends a media stream
as a <dfn>media sender</dfn> and an agent that receives a media stream as a
<dfn>media receiver</dfn>. Note that an agent can be both a media sender and a
media receiver.
For additional terms and idioms specific to the
[[PRESENTATION-API|Presentation API]] or [[REMOTE-PLAYBACK|Remote Playback API]],
please consult the respective specifications.
Requirements {#requirements}
============================
Agent Discovery Requirements {#requirements-discovery}
------------------------------------------------------
Issue(342): Define Discovery Requirements
Transport Layer Requirements {#requirements-transport}
------------------------------------------------------
Issue(342): Define Transport Layer Requirements
Presentation API Requirements {#requirements-presentation-api}
--------------------------------------------------------------
1. A [=controller=] must be able to determine if a [=receiver=] is reasonably
capable of rendering a specific [=presentation request URL=].
2. A controller must be able to start a new [=presentation=] on a
receiver given a [=presentation request URL=] and [=presentation
identifier=].
3. A controller must be able to create a new {{PresentationConnection}} to an
existing presentation on the receiver, given its [=presentation request
URL=] and [=presentation identifier=].
4. It must be possible to close a {{PresentationConnection}} between a
controller and a presentation, and signal both parties with the reason why
the connection was closed.
5. Multiple controllers must be able to connect to a single presentation
simultaneously.
6. Messages sent by the controller must be delivered to the presentation (or
vice versa) in a reliable and in-order fashion.
7. If a message cannot be delivered, then the controller must be
able to signal the receiver (or vice versa) that the connection should be
closed with reason `error`.
8. The controller and presentation must be able to send and receive `DOMString`
messages (represented as `string` type in ECMAScript).
9. The controller and presentation must be able to send and receive binary
messages (represented as `Blob` objects in HTML5, or `ArrayBuffer` or
`ArrayBufferView` types in ECMAScript).
10. The controller must be able to signal to the receiver to
terminate a presentation, given its [=presentation request URL=] and
[=presentation identifier=].
11. The receiver must be able to signal all connected controllers
when a presentation is terminated.
Remote Playback API Requirements {#requirements-remote-playback}
----------------------------------------------------------------
1. A [=controller=] must be able to find out whether there is at least one
compatible [=receiver=] for a given {{HTMLMediaElement}}, both
instantaneously and continuously.
2. A controller must be able to to [=initiate remote playback=] of
an {{HTMLMediaElement}} to a compatible receiver.
3. The controller must be able send media sources as URLs and text
tracks from an {{HTMLMediaElement}} to a compatible receiver.
4. The controller must be able send media data from an {{HTMLMediaElement}} to
a compatible receiver.
5. During remote playback, the controller and the remote playback
device must be able to synchronize the [=media element state=] of the
{{HTMLMediaElement}}.
6. During remote playback, either the controller or the receiver must be able
to disconnect from the other party.
7. The controller should be able to pass locale and text direction information
to the receiver to assist in rendering text during remote playback.
Non-Functional Requirements {#requirements-non-functional}
----------------------------------------------------------
1. It should be possible to implement an OSP agent using modest
hardware requirements, similar to what is found in a low end smartphone,
smart TV or streaming device. See the [Device
Specifications](https://w3c.github.io/openscreenprotocol/device_specs.html)
document for agent hardware specifications.
2. Agents should present sensible information to the user when a protocol
operation fails. For example, if a controller is unable to start a
presentation, it should be possible to report in the controller interface if
it was a network error, authentication error, or the presentation content
failed to load.
3. Message latency between agents should be minimized to permit interactive
use. For example, it should be comfortable to type in a form in one agent
and have the text appear in the presentation in real time. Real-time
latency for gaming or mouse use is ideal, but not a requirement.
4. The controller initiating a presentation or remote playback should
communicate its preferred locale to the receiver, so it can render the
content in that locale.
5. It should be possible to extend the application protocol with optional
features not defined explicitly by the specification, to facilitate
experimentation and enhancement of the base APIs.
Metadata Discovery {#metadata}
==============================
To learn further metadata, an agent may send an [=agent-info-request=] message
and receive back an [=agent-info-response=] message. Any agent may send this
request at any time to learn about the state and capabilities of another device,
which are described by the [=agent-info=] message in the
[=agent-info-response=].
If an agent changes any information in its [=agent-info=] message, it should
send an [=agent-info-event=] message to all other connected agents with the new
[=agent-info=] (without waiting for an [=agent-info-request=]).
The [=agent-info=] message contains the following fields:
: display-name (required)
:: The display name of the agent, intended to be displayed to a user by the
requester. The requester should indicate through the UI if the responder
is not authenticated or if the display name changes.
: model-name (optional)
:: If the agent is a hardware device, the model name of
the device. This is used mainly for debugging purposes, but may be
displayed to the user of the requesting agent.
: capabilities (required)
:: The control protocols, roles, and media types the agent supports.
Presence indicates a capability and absence indicates lack of a
capability. Capabilities should should affect how an agent is
presented to a user, such as drawing a different icon depending on
the whether it receives audio, video or both.
: state-token (required)
:: A random alphanumeric value consisting of 8 characters in the range
[0-9A-Za-z]. This value is set before the agent makes its first connection
and must be set to a new value when the agent is reset or otherwise lost all
of its state related to this protocol.
: locales (required)
:: The agent's preferred locales for display of localized content, in the order
of user preference. Each entry is an RFC5646 [=language tag=].
The various capabilities have the following meanings:
: receive-audio
:: The agent can render audio via the other protocols it supports. Those other
protocols may report more specific capabilities, such as support for
certain audio codecs in the streaming protocol.
: receive-video
:: The agent can receive video via the other protocols it supports. Those other
protocols may report more specific capabilities, such as support for
certain video codecs in the streaming protocol.
: receive-presentation
:: The agent can receive presentations using the presentation protocol.
: control-presentation
:: The agent can control presentations using the presentation protocol.
: receive-remote-playback
:: The agent can receive remote playback using the remote playback
protocol.
: control-remote-playback
:: The agent can control remote playback using the remote playback
protocol.
: receive-streaming
:: The agent can receiving streaming using the streaming protocol.
: send-streaming
:: The agent can send streaming using the streaming protocol.
NOTE: See the [Capabilities Registry](https://github.com/w3c/openscreenprotocol/blob/main/capabilities.md)
for a list of all known capabilities (both defined by this specification, and
through [[#protocol-extensions]]).
Issue(343): Rewrite to not depend on QUIC, or move agent-status messges to the network spec
If a listening agent wishes to receive messages from an advertising agent or an
advertising agent wishes to send messages to a listening agent, it may wish to
keep the QUIC connection alive. Once neither side needs to keep the connection
alive for the purposes of sending or receiving messages, the connection should
be closed with an error code of 5139. In order to keep a QUIC connection alive, an
agent may send an [=agent-status-request=] message, and any agent that receives an
[=agent-status-request=] message should send an [=agent-status-response=] message. Such
messages should be sent more frequently than the QUIC idle_timeout transport
parameter (see Transport Parameter Encoding in QUIC) and QUIC PING
frames should not be used. An idle_timeout transport parameter of 25 seconds is
recommended. The agent should behave as though a timer less than the
idle_timeout were reset every time a message is sent on a QUIC stream. If the
timer expires, a [=agent-status-request=] message should be sent.
If a listening agent wishes to send messages to an advertising agent, the
listening agent can connect to the advertising agent "on demand"; it does not
need to keep the connection alive.
If an OSP agent suspends its network connectivity (e.g. for power saving
reasons), it should attempt to resume QUIC connections to the OSP agents to
which it was previously connected once network connectivity is restored. Once
reconnected, it should send `agent-status-request` messages to those agents.
The [=agent-info=] and [=agent-status-response=] messages may be extended to
include additional information not defined in this spec, as described in
[[#protocol-extension-fields]].
Message delivery using CBOR {#messages}
=======================================
Issue(343): Rewrite to not depend on QUIC
Messages are serialized using [[!RFC8949|CBOR]]. To send a group of messages in
order, that group of messages must be sent in one QUIC stream. Independent
groups of messages (with no ordering dependency across groups) should be sent in
different QUIC streams. In order to put multiple CBOR-serialized messages into
the the same QUIC stream, the following is used.
For each message, the [=OSP agent=] must write into a unidirectional QUIC stream
the following:
1. A type key representing the type of the message, encoded as a [=variable-length
integer=] (see [[#appendix-a]] for type keys)
2. The message encoded as CBOR.
If an agent receives a message for which it does not recognize a type key, it
must close the QUIC connection with an application error code of 404 and should
include the unknown type key in the reason phrase of the CONNECTION_CLOSE
frame.
Variable-length integers are encoded in the [=Variable-Length Integer Encoding=]
used by [[!RFC9000|QUIC]].
Many messages are requests and responses, so a common format is defined for
those. A request and a response includes a request ID which is an unsigned
integer chosen by the requester. Responses must include the request ID of the
request they are associated with.
Type Key Backwards Compatibility {#message-compatibility}
--------------------------------
As messages are modified or extended over time, certain rules must be followed
to maintain backwards compatibiilty with agents that understand older versions
of messages.
1. If a required field is added to or removed from a message (either to/from the
message directly or indirectly through the field of a field), a new type key
must be assigned to the message. Is is effectively a new message and must not
be sent unless the receiving agent is known to understand the new type key.
1. If an optional field is added to a message (either to the message directly
or indirectly through the field of a field), the type key may remain unchanged
if the behavior of older receiving agents that do not understand the added field
is compatible with newer sending agents that include the field.
Otherwise, a new type key must be assigned.
1. If an optional field is removed from a message (either from the message
directly or indirectly through the field of a field), the type key may remain
unchanged if the behavior of newer receiving agents that do not understand the
removed field is compatible with older sending agents that include the field.
Otherwise, a new type key must be assigned.
1. Required fields may not be added or removed from array-based messages, such
as audio-frame.
Presentation Protocol {#presentation-protocol}
=====================
This section defines the use of the Open Screen Protocol for starting,
terminating, and controlling presentations as defined by
[[PRESENTATION-API|Presentation API]]. [[#presentation-api]]
defines how APIs in [[PRESENTATION-API|Presentation API]] map to the
protocol messages defined in this section.
To learn which receivers are [=available presentation displays=] for a
particular [=presentation request URL=] or set of URLs, the controller may send
a [=presentation-url-availability-request=] message with the following values:
: urls
:: A list of presentation URLs. Must not be empty.
: watch-duration
:: The period of time that the controller is interested in receiving updates
about their URLs, should the availability change.
: watch-id
:: An identifier the receiver must use when sending updates about URL
availability so that the controller knows which URLs the receiver is referring
to.
In response, the receiver should send one [=presentation-url-availability-response=]
message with the following values:
: url-availabilities
:: A list of URL availability states. Each state must correspond to the matching URL
from the request by list index.
While the watch is valid (the watch-duration has not expired), the receivers
should send [=presentation-url-availability-event=] messages when URL
availabilities change. Such events contain the following values:
: watch-id
:: The watch-id given in the [=presentation-url-availability-response=],
used to refer to the presentation URLs whose availability has changed.
: url-availabilities
:: A list of URL availability states. Each state must correspond to the URLs from the
request referred to by the watch-id.
Note that these messages are not broadcasted to all controllers. They are sent
individually to controllers that have requested availability for the URLs that
have changed in availability state within the watch duration of the original
availability request.
Issue(342): Add power saving as a transport requirement and remove the following.
To save power, the controller may disconnect the QUIC connection and
later reconnect to send availability requests and receive availability
responses and updates. The QUIC connection ID may or may not be the same
when reconnecting.
To start a presentation, the controller may send a
[=presentation-start-request=] message to the receiver with the following
values:
: presentation-id
:: The [=presentation identifier=]
: url
:: The selected presentation URL
: headers
:: headers that the receiver should use to fetch the presentation URL. For example,
[[PRESENTATION-API#creating-a-receiving-browsing-context|section 6.6.1]] of
the Presentation API says that the HTTP `Accept-Language` header should be
provided.
The [=presentation identifier=] must follow the restrictions defined by
[[PRESENTATION-API#common-idioms|section 6.1]] of the Presentation API, in that
it must consist of at least 16 ASCII characters.
When the receiver receives the [=presentation-start-request=], it should send back a
[=presentation-start-response=] message after either the presentation URL has been
fetched and loaded, or the receiver has failed to do so. If it has failed, it
must respond with the appropriate result (such as invalid-url or timeout). If
it has succeeded, it must reply with a success result.
Additionally, the response must include the following:
: connection-id
:: An ID that both agents can use to send connection messages
to each other. It is chosen by the receiver for ease of implementation: if
the message receiver chooses the connection-id, it may keep the ID unique
across connections, thus making message demuxing/routing easier.
The response should include the following:
: http-response-code
:: The numeric HTTP response code that was returned from fetching the
presentation URL (after redirects).
To send a presentation message, the controller or receiver may send a
[=presentation-connection-message=] with the following values:
: connection-id
:: The ID from the [=presentation-start-response=] or
[=presentation-connection-open-response=] messages.
: message
:: The presentation message data.
Issue(342): Rewrite the following NOTE as a transport requirement for any
message transport.
NOTE: An OSP agent should minimize buffering and processing of messages sent or
received beyond what is strictly necessary (i.e., CBOR serialization). Message
payloads should be treated as real-time data, as they may be used to synchronize
playback of media streams between agents or other low latency use cases. The
synchronization thresholds recommended in [[ITU-R-BT.1359-1]] imply that the
total agent-to-agent processing latency (including serialization, buffering, and
network latency) must be no greater than 45 ms to permit effective lip sync
during media playback.
To terminate a presentation, the controller may send a
[=presentation-termination-request=] message with the following values:
: presentation-id
:: The ID of the presentation to terminate.
: reason
:: Set to `application-request` if the application requested termination,
or `user-request` if the user requested termination. (These are the only
valid values for `reason` in a [=presentation-termination-request=].)
When a [=receiver=] receives a [=presentation-termination-request=], it should
send back a [=presentation-termination-response=] message to the requesting
controller.
It should also notify other controllers about the termination by sending
a [=presentation-termination-event=] message. And it can send the same message if
it terminates a presentation without a request from a controller to do so. This
message contains the following values:
: presentation-id
:: The ID of the presentation that was terminated.
: source
:: Set to `controller` when the termination was in response to a
[=presentation-termination-request=], or `receiver` otherwise.
: reason
:: The detailed reason why the presentation was terminated.
To accept incoming connection requests from controller, a receiver must receive
and process the [=presentation-connection-open-request=] message which contains the
following values:
: presentation-id
:: The ID of the presentation to connect to.
: url
:: The URL of the presentation to connect to.
The receiver should, upon receipt of a
[=presentation-connection-open-request=] message, send back a
[=presentation-connection-open-response=] message which contains the
following values:
: result
:: a code indicating success or failure, and the reason for the failure
: connection-id
:: An ID that both agents can use to send connection messages
to each other. It is chosen by the receiver for ease of implementation (if
the message receiver chooses the connection-id, it may keep the ID unique
across connections, thus making message demuxing/routing easier).
: connection-count
:: The new number of open connections to the presentation that received
the incoming connection request.
If the [=presentation-connection-open-response=] message indicates success, the
receiver should also send a [=presentation-change-event=] to all other endpoints
that have an active presentation connection to that presentation with the
values:
: presentation-id
:: The ID of the presentation that just received a new presentation connection.
: connection-count
:: The new total number of open connections to that presentation.
A controller may close a connection without terminating the presentation by
sending a [=presentation-connection-close-event=] message to the receiver with the
following values:
: connection-id
:: The ID of the connection that was closed.
: reason
:: Set to `close-method-called` or `connection-object-discarded`.
The receiver may also close a connection without terminating a presentation. If
it does so, it should send a [=presentation-connection-close-event=] message to the
controller with the following values:
: connection-id
:: The ID of the connection that was closed.
: reason
:: Set to `close-method-called` or `connection-object-discarded`.
: connection-count
:: The number of open presentation connections that remain.
If a receiver closes a presentation connection (for any reason), it should send
a [=presentation-change-event=] to all other controllers with an open connection
to that presentation with the values:
: presentation-id
:: The ID of the presentation that just closed a connection.
: connection-count
:: The number of open presentation connections that remain.
Note: When an agent closes a presentation connection, it is always successful,
so request and response messages are not needed. A request to terminate a
presentation may succeed or fail, so a response message is required.
Presentation API {#presentation-api}
---------------------------------------------
An [=Open Screen Protocol agent=] that is a [=controlling user agent=] for the
[[PRESENTATION-API|Presentation API]] must support the `control-presentation` capability.
An [=OSP agent=] that is a [=receiving user agent=] for the
[[PRESENTATION-API|Presentation API]] must support the `receive-presentation` capability.
The same OSP agent may be a [=controlling user agent=] and a [=receiving user agent=].
This is how the [[PRESENTATION-API|Presentation API]] uses the
[[#presentation-protocol]]:
When [[PRESENTATION-API#the-list-of-available-presentation-displays|section
6.4.2]] says "This list of presentation displays ... is populated based on an
implementation specific discovery mechanism", the [=controller=] may
use the mDNS, QUIC, [=agent-info-request=], and
[=presentation-url-availability-request=] messages defined previously in this spec
to discover receivers.
When [[PRESENTATION-API#the-list-of-available-presentation-displays|section
6.4.2]] says "To further save power, ... implementation specific discovery of
presentation displays can be resumed or suspended.", the agent may use the
power saving mechanism defined in the previous section.
When [[PRESENTATION-API#starting-a-presentation-connection|section 6.3.4]] says
"Using an implementation specific mechanism, tell U to create a receiving
browsing context with D, presentationUrl, and I as parameters.", U (the
controller) may send a [=presentation-start-request=] message to D
(the receiver), with I for the [=presentation identifier=] and presentationUrl
for the selected presentation URL.
When [[PRESENTATION-API#reconnecting-to-a-presentation|section 6.3.5]] says to
"establish a presentation connection with newConnection," let U be the
presentationURL of `newConnection` and I the presentation identifier of
`newConnection.` The agent should send a
[=presentation-connection-open-request=] message with U for the `url` and I for
the `presentation-id`.
When [[PRESENTATION-API#sending-a-message-through-presentationconnection|section
6.5.2]] says "Using an implementation specific mechanism, transmit the contents
of messageOrData as the presentation message data and messageType as the
presentation message type to the destination browsing context", the
controller may send a [=presentation-connection-message=] with
messageOrData for the presentation message data. Note that the messageType is
embedded in the encoded CBOR type and does not need an additional value in the
message.
When [[PRESENTATION-API#closing-a-presentationconnection|section 6.5.5]] says
"Start to signal to the destination browsing context the intention to close the
corresponding PresentationConnection", the agent may send a
[=presentation-connection-close-event=] message to the other agent with the
destination browsing context and a [=presentation-change-event=] when required.
When
[[PRESENTATION-API#terminating-a-presentation-in-a-controlling-browsing-context|section
6.5.6]] says "Send a termination request for the presentation to its receiving
user agent using an implementation specific mechanism", the controller may send
a [=presentation-termination-request=] message to the receiver with a `reason`
of `application-request`.
When [[PRESENTATION-API#monitoring-incoming-presentation-connections|section
6.7.1]]
says "it MUST listen to and accept incoming connection requests from a
controlling browsing context using an implementation specific
mechanism", the receiver must receive and process the
[=presentation-connection-open-request=].
When [[PRESENTATION-API#monitoring-incoming-presentation-connections|section
6.7.1]] says "Establish the connection between the controlling and receiving
browsing contexts using an implementation specific mechanism.", the receiver
must send a [=presentation-connection-open-response=] message and
[=presentation-change-event=] messages when required.
Representation Of Time {#time-representation}
======================
The [[#remote-playback-protocol]] and the [[#streaming-protocol]] represent
points of time and durations in terms of a [=time scale=]. A <dfn>time
scale</dfn> is a common denominator for time values that allows values to be
expressed as rational numbers without loss of precision. The [=time scale=] is
represented in hertz, such as 90000 for 90000 Hz, a common time scale for
video.
Remote Playback Protocol {#remote-playback-protocol}
========================
This section defines the use of the Open Screen Protocol for starting, terminating,
and controlling remote playback of media as defined by the
[[REMOTE-PLAYBACK|Remote Playback API]]. [[#remote-playback-api]] defines how
APIs in [[REMOTE-PLAYBACK|Remote Playback API]] map to the protocol messages
defined in this section.
For all messages defined in this section, see [[#appendix-a]] for the full CDDL
definitions.
To learn which receivers are [=compatible remote playback devices=] for a
particular URL or set of URLs, the controller may send a
[=remote-playback-availability-request=] message with the following values:
: sources
:: A list of [=media resources=], the same as specified in the
[=remote-playback-start-request=] message. Must not be empty.
: headers
:: headers that the receiver should use to fetch the
urls. For example,
[[REMOTE-PLAYBACK#establishing-a-connection-with-a-remote-playback-device|section 6.2.4 of
the Remote Playback API]] says that the Accept-Language header should be
provided.
: watch-duration
:: The period of time that the controller is interested in receiving updates
about their URLs, should the availability change.
: watch-id
:: An identifier the receiver must use when sending updates about URL
availability so that the controller knows which URLs the receiver is referring
to.
In response, the receiver should send a [=remote-playback-availability-response=]
message with the following values:
: url-availabilities
:: A list of URL availability states. Each state must correspond to the matching URL
from the request by list index.
The receivers should later (up to the current time plus request
watch-duration) send [=remote-playback-availability-event=] messages if
URL availabilities change. Such events contain the following values:
: watch-id
:: The watch-id given in the [=remote-playback-availability-response=]
used to refer to the remote playback URLs whose availability has changed.
: url-availabilities
:: A list of URL availability states. Each state must correspond to the URLs from the
request referred to by the watch-id.
Note that these messages are not broadcasted to all controllers. They are sent
individually to controllers that have requested availability for the URLs that
have changed in availability state within the watch duration of the original
availability request.
Issue(342): Add power saving as a transport requirement and remove the following.
To save power, the controller may disconnect the QUIC connection and
later reconnect to send availability requests and receive availability
responses and updates. The QUIC connection ID may or may not be the same
when reconnecting.
To start remote playback, the controller may send a
[=remote-playback-start-request=] message to the receiver with the following
values:
: remote-playback-id
:: An identifier for this remote playback. It should be universally unique
among all remote playbacks.
Note: A version 4 (pseudorandom) [=UUID=] is recommended as it meets the
requirements for a remote-playback-id.
: sources (optional)
:: The [=media resources=] that the controller has selected for playback
on the receiver. Each source must include a <{source/src|source URL}>
and should include an <{source/type|extended MIME type}> when available
for the [=media resource=]. If `sources` is missing or empty, the
`remoting` field must be populated, as the controller will use a
streaming session to send encoded media.
: text-track-urls
:: URLs of text tracks associated with the [=media resources=].
: controls
:: Initial controls for modifying the initial state of the remote playback, as
defined in [[#remote-playback-state-and-controls]]. The controller may send
controls that are optional for the receiver to support before it knows the
receiver supports them. If the receiver does not support them, it will
ignore them and the controller will learn that it does not support them from
the [=remote-playback-start-response=] message.
: remoting (optional)
:: Parameters for starting a streaming session associated with this
remote playback. If not included, no streaming session is started.
Required when `sources` is missing or empty.
When the receiver receives a [=remote-playback-start-request=] message, it should
send back a [=remote-playback-start-response=] message. It should do so quickly,
usually before the [=media resource=] has been loaded and instead give updates
of the progress of loading with [=remote-playback-state-event=] messages, unless
the receiver decides to not attempt to load the resource at all. If it chooses
not to, it must respond with the appropriate failure result (such as timeout or
invalid-url). Additionally, the response must include the following:
: state
:: The initial state of the remote playback, as defined in
[[#remote-playback-state-and-controls]].
: remoting (optional)
:: A response to the started streaming session associated with this remote playback.
If not included, no streaming session is started.
If a streaming session is started, streaming messages such a
[=streaming-session-modify-request=] and [=video-frame=] can be used for the
streaming session as if the streaming session had been started with
[=streaming-session-start-request=] and
[=streaming-session-start-response=]. The streaming session may be terminated
before the remote playback is terminated, but if the remote playback is
terminated first, the streaming session associated with it is automatically
terminated.
If the controller wishes to modify the state of the remote playback (for
example, to pause, resume, skip, etc), it may send a
[=remote-playback-modify-request=] message with the following values:
: remote-playback-id
:: The ID of the remote playback to be modified.
: controls
:: Updated controls as defined in [[#remote-playback-state-and-controls]].
When a receiver receives a [=remote-playback-modify-request=] it should send a
[=remote-playback-modify-response=] message in reply with the following values:
: state
:: The updated state of the remote playback as defined in
[[#remote-playback-state-and-controls]].
When the state of remote playback changes without request for modification from
the controller (such as when the skips or pauses due to user user interaction on
the receiver), the receiver may send a [=remote-playback-state-event=] to the
controller.
The receiver should send a [=remote-playback-state-event=] message whenever:
Any of the following methods are called:
* {{HTMLMediaElement/fastSeek()|HTMLMediaElement.fastSeek()}}
* {{HTMLMediaElement/pause()|HTMLMediaElement.pause()}}
* {{HTMLMediaElement/play()|HTMLMediaElement.play()}}
Any of the following attributes observably change since the last sent [=remote-playback-state-event=] message:
* {{HTMLMediaElement/currentSrc|HTMLMediaElement.currentSrc}}
* {{HTMLMediaElement/networkState|HTMLMediaElement.networkState}}
* {{HTMLMediaElement/readyState|HTMLMediaElement.readyState}}
* {{HTMLMediaElement/error!!attribute|HTMLMediaElement.error}}
* {{HTMLMediaElement/duration|HTMLMediaElement.duration}}
* {{HTMLMediaElement/buffered|HTMLMediaElement.buffered}}
* {{HTMLMediaElement/seekable|HTMLMediaElement.seekable}}
* {{HTMLMediaElement/playbackRate|HTMLMediaElement.playbackRate}}
* {{HTMLMediaElement/paused|HTMLMediaElement.paused}}
* {{HTMLMediaElement/seeking!!attribute|HTMLMediaElement.seeking}}
* {{HTMLMediaElement/ended!!attribute|HTMLMediaElement.ended}}
* {{HTMLMediaElement/volume|HTMLMediaElement.volume}}
* {{HTMLMediaElement/muted|HTMLMediaElement.muted}}
* {{HTMLMediaElement/audioTracks|HTMLMediaElement.audioTracks}}
* {{HTMLMediaElement/videoTracks|HTMLMediaElement.videoTracks}}
* {{HTMLMediaElement/textTracks|HTMLMediaElement.textTracks}}
* {{HTMLVideoElement/videoWidth|HTMLVideoElement.videoWidth}}
* {{HTMLVideoElement/videoHeight|HTMLVideoElement.videoHeight}}
The [=timeline offset=] associated with the playback changes since the last sent [=remote-playback-state-event=] message:
The {{stalled}} event needs to fire at the associated {{HTMLMediaElement}} instance.
More than 250ms pass since the last [=remote-playback-state-event=] message
and any of the following attributes observably change since the last
[=remote-playback-state-event=] message. Any new continuously changing
attributes fall under this rule.
* {{HTMLMediaElement/played|HTMLMediaElement.played}}
* {{HTMLMediaElement/currentTime|HTMLMediaElement.currentTime}}
NOTE: A media element is required to fire a {{HTMLMediaElement/timeupdate}}
event every 250ms or sooner.
: remote-playback-id
:: The ID of the remote playback whose state has changed.
: state
:: The updated state of the remote playback, as defined in
[[#remote-playback-state-and-controls]].
To terminate the remote playback, the controller may send a
[=remote-playback-termination-request=] message with the following values:
: remote-playback-id
:: The ID of the remote playback to terminate.
: reason
:: The reason the remote playback is being terminated.
When a receiver receives a [=remote-playback-termination-request=], it should
send back a [=remote-playback-termination-response=] message to the controller.
If a receiver terminates a remote playback without a request from the controller
to do so, it must send a [=remote-playback-termination-event=] message to the
controller with the following values:
: remote-playback-id
:: The ID of the remote playback that was terminated.
: reason
:: The reason the remote playback was terminated.
As mentioned in
[[REMOTE-PLAYBACK#disconnecting-from-a-remote-playback-device|Remote Playback
API section 6.2.7]], terminating the remote playback means the controller is no
longer controlling the remote playback and does not necessarily stop media from
rendering on the receiver. Whether or not the receiver stops rendering media depends
upon the implementation of the receiver.
Remote Playback State and Controls {#remote-playback-state-and-controls}
------------------------------------------------------------------------
In order for the controller and receiver to stay in sync with regards to the
state of the remote playback, the controller may send controls to modify the state
(for example, via the [=remote-playback-modify-request=] message) and the receiver
may send updates about state changes (for example, via the
[=remote-playback-state-event=] message).
The controls sent by the controller include the following individual control
values, each of which is optional. This allows the controller to change one
control value or many control values at once without having to specify all
control values every time. A non-present control value indicates no change. A
present control value indicates the change defined below. These controls
intentionally mirror settable attributes and methods of the
{{HTMLMediaElement}}.
: source
:: Change the [=media resource=]. See
{{HTMLMediaElement/src|HTMLMediaElement.src}}
for more details. Must not be used in the initial controls of the
[=remote-playback-start-request=] message (which already contains a
[=media resource=]).
: preload
:: Set how aggressively to preload media. See
{{HTMLMediaElement/preload|HTMLMediaElement.preload}}
for more details. Should only be used in the initial controls of the
[=remote-playback-start-request=] message or when the source is changed. If not
set in the initial controls, it is left to the receiver to decide. This is
optional for the receiver to support and if not supported, the receiver will
behave as though it were never set.
: loop
:: Set whether or not to loop media. See
{{HTMLMediaElement/loop|HTMLMediaElement.loop}}
for more details. Should only be used in the initial control of the
[=remote-playback-start-request=]. If not set in the initial controls, it is
assumed to be false.
: paused
:: If true, pause; if false, resume. See
{{HTMLMediaElement/pause()|HTMLMediaElement.pause()}}
and
{{HTMLMediaElement/play()|HTMLMediaElement.play()}}
for more details. If not set in the initial controls, it is left to the
receiver to decide.
: muted
:: If true, mute; if false, unmute. See
{{HTMLMediaElement/muted|HTMLMediaElement.muted}}
for more details. If not set in the initial controls, it is left to the
receiver to decide.
: volume
:: Set the audio volume in the range from 0.0 to 1.0 inclusive. See
{{HTMLMediaElement/volume|HTMLMediaElement.volume}}
for more details. If not set in the initial controls, it is left to the
receiver to decide.
: seek
:: Seek to a precise time. See
{{HTMLMediaElement/currentTime|HTMLMediaElement.currentTime}}
for more details.
: fast-seek
:: Seek to an approximate time as fast as possible. See
{{HTMLMediaElement/fastSeek()|HTMLMediaElement.fastSeek()}}
for more details.
: playback-rate
:: Set the rate a which the media plays. See
{{HTMLMediaElement/playbackRate|HTMLMediaElement.playbackRate}}
for more details. If not set in the initial controls, it is left to the
receiver to decide. This is optional for the receiver to support and if not
supported, the receiver will behave as though it were never set.