-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.rst
196 lines (133 loc) · 9.21 KB
/
index.rst
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
###################
Teleport Unreal SDK
###################
The Teleport Unreal SDK provides everything necessary to run a Teleport server as an Unreal Engine 5 application. In typical usage, only the server need run Unreal Engine - the client will be a standard Teleport client app such as the Teleport reference client.
Getting Started
===============
These instructions are for Windows platforms, Linux and MacOS support is to follow.
Install the plugin to an existing project
-----------------------------------------
Get the Teleport Unreal Engine plugin from its Git repo at `[email protected]:simul/teleport-unreal.git <https://github.com/simul/teleport-unreal>`_.
You can install it as a Git submodule or copy the code directly, it should go in a subfolder of your Unreal Engine project called Teleport:
[GAME FOLDER]/Plugins/Teleport
Regenerate your game project (e.g by right-clicking on the [GAME].uproject file and selecting "Generate Visual Studio Project Files").
Then build the [GAME].sln solution. You can run Unreal as normal, or debug the C++ code by setting TeleportServer as the active C++ project in the solution, and launching it for debugging.
Configuration
=============
First, activate the Teleport render pipeline. This adds functionality for video streaming and lighting generation. In the *Edit* menu, under *Project Settings...*, select *Graphics*. Set the Scriptable Render Pipeline Settings
object to point to the Default Teleport Render Pipeline, found in Teleport/Resources.
For global settings for the plugin, in the Project Settings panel, select *Teleport VR*.
.. image:: /images/unreal/ProjectSettings.png
:width: 600
:alt: Teleport has a page on the Project Settings panel.
**Teleport Project Settings panel**
Resource Management
===================
Resources in a Unreal project can be streamed to clients if they have been *extracted* - i.e. if they have been converted to and stored in, the standard formats that the Teleport protocol uses.
.. image:: /images/Unreal/TeleportMenu.png
:width: 600
:alt: Selecting the Resource Manager option in the Teleport Menu.
Select the "Teleport VR" menu from Unreal Editor's menu bar, and click on "Resource Manager". The Resource Manager window will open.
.. image:: /images/Unreal/ResourceManager.png
:width: 600
:alt: Teleport Resource Manager in the Unreal Editor.
To extract all streamable resources in a scene, click "Scene Geometry: Extract". This is all you will usually need to do.
To extract the resources of a specific object, select it and click the "Selected Geometry: Extract" button.
Separately, you can also extract the Global Illumination textures (generated by Unreal from static lights), or the Dynamic Object Lighting Textures: these are the cubemaps for specular and diffuse lighting
generated by the Monitor.
There are two main Ways to setup objects for streaming:
1. By Tag
2. By Root
Usage
=====
You can create a GameObject in Unreal that has the Monitor component. If not, one will be created when you run the project.
When running, the server awaits connections via the Teleport protocol. When a client connects, the server creates a player instance.
A client, when connected, will maintain a list of Nodes (i.e. GameObjects in Unreal) that are relevant to it. The space around the client's
own position is called its Area of Interest (AOI). The server will send it updates for all Nodes within its AOI.
Inputs
======
To show the Inputs Panel, select Inputs from the Teleport VR menu on the main menu bar.
Here, you can specify the inputs you want to receive from connected clients.
.. image:: /images/Unreal/InputsPanel.png
:width: 600
:alt: Teleport has a page on the Project Settings panel.
**Teleport Inputs Panel in Unreal Editor**
There are three elements to each input. The Name is arbitrary, but should be unique in the application. The Type specifies what kind of input this is. A Float input is floating-point, while a boolean is either on or off, true or false. A state input is updated continuously, whereas an Event input is only updated when it changes. Finally, the input's *Path* is how the client knows what to map it to.
Teleport VR clients use `OpenXR interaction profiles <https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#semantic-path-interaction-profiles>`_ to know what inputs the XR hardware provides. Each input (buttons, triggers, poses etc) has a *path* that is mapped by the OpenXR runtime on the client device.
When a Teleport client connects, it tries to match the path of each server-specified Teleport input to the OpenXR paths of its hardware. If any part of the OpenXR path matches the Teleport path, the mapping is made.
One or more Teleport inputs can be mapped to a single OpenXR input if the paths match.
Teleport input paths use `Regular Expression <https://en.wikipedia.org/wiki/Regular_expression>`_ syntax to match OpenXR paths.
For example, if the client recognizes your hardware as supporting the `Oculus Touch Controller Profile <https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#_oculus_touch_controller_profile>`_:
/interaction_profiles/oculus/touch_controller
It will have the path:
user/hand/left/input/x/click
to represent the "x" button on the left-hand controller. We might specify a control:
+-------------------------+-------------------+---------------------------+
| Name | Type | Path |
| | | |
+=========================+===================+===========================+
| Toggle Onscreen Display | Boolean Event | left/input/[a|x]/click |
+-------------------------+-------------------+---------------------------+
The regex syntax [a|x] means "either 'a' or 'x'", so the client will recognize this as a match, and map the "x" button on the client-side controller to the "Toggle Onscreen Display" boolean event that the server will receive.
Mapping types
-------------
Different control types can be mapped to each other. If a path match is found, it is possible for a boolean ("click") action, which is only on or off, to be mapped to an Analogue input.
+-------------------------+--------------------------+---------------------+-----------------+
| Teleport / OpenXR | OpenXR Boolean | OpenXR float | OpenXR pose |
+=========================+==========================+=====================+=================+
| Teleport Boolean | Yes | Yes, client decides | No |
+-------------------------+--------------------------+---------------------+-----------------+
| Teleport Analogue | Yes: false=0.0, true=1.0 | Yes | No |
+-------------------------+--------------------------+---------------------+-----------------+
The mapping of an OpenXR floating-point input to a Teleport Boolean input is determined by the client application. The usual method is by hysteresis, so when the control goes above a certain threshold, it will be considered to be "true", and when it goes below a lower threshold, it will revert to "false".
Teleport cannot map OpenXR Pose actions to inputs, these are handled differently.
Pose Mapping
------------
XR devices such as headsets, handsets and trackers report their state as a "pose", containing position and orientation in space. These poses can be mapped to spatial nodes using the Teleport Controller component in Unreal.
.. image:: /images/Unreal/TeleportController.png
:width: 600
:alt: Inspector properties of a teleport Controller component.
==Teleport controller component in Unreal Inspector==
The "Pose Regex Path" for a controller is matched client-side to an OpenXR path representing a pose state. When this mapping occurs, the object will be controlled directly by the tracked controller on the client. The Teleport_Controller component can be added to any child of a Teleport_SessionComponent Game Object.
Player Session Hierarchy
========================
Typically, the Unreal Game Objects would be arranged as follows in Unreal:
.. image:: /images/Unreal/PlayerHierarchy.png
:alt: Example player prefab hierarchy.
:width: 400
==Typical session and player hierarchy in Unreal==
i.e. at the root, an object containing a Teleport_Session Component, which tracks client-specific session data. Below this, a Player object which may move in space. Below that, two controllers and a head tracking object.
* TeleportXR : Teleport_Session Component
* Player
* Left Hand Controller: Teleport_Controller Component
* Right Hand Controller: Teleport_Controller Component
* Head: Teleport_Head Component
SDK Reference
===============
Global
~~~~~~
doxygenclass:: teleport::TeleportSettings
project: TeleportUnreal
members:
Singleton Components
~~~~~~~~~~~~~~~~~~~~
.. doxygenclass:: teleport::Monitor
:project: TeleportUnreal
:members:
.. doxygenclass:: teleport::TagHandler
:project: TeleportUnreal
:members:
.. doxygenclass:: teleport::Teleport_SessionComponent
:project: TeleportUnreal
:members:
Components
~~~~~~~~~~
.. doxygenclass:: teleport::StreamableRoot
:project: TeleportUnreal
:members:
.. doxygenclass:: teleport::StreamableNode
:project: TeleportUnreal
:members:
.. doxygenclass:: teleport::StreamableProperties
:project: TeleportUnreal
:members: