Skip to content

Commit

Permalink
Merge branch 'l7mp:main' into staging
Browse files Browse the repository at this point in the history
  • Loading branch information
tyhu05 authored Jul 24, 2023
2 parents b506d42 + 8ec906e commit 58186cb
Show file tree
Hide file tree
Showing 5 changed files with 69 additions and 187 deletions.
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ With a minimal understanding of WebRTC and Kubernetes, deploying STUNner should
minutes.

* [Customize STUNner and deploy it](#installation) into your Kubernetes cluster.
* Optionally [deploy a WebRTC media server](examples/kurento-one2one-call).
* Optionally [deploy a WebRTC media server](docs/examples/kurento-one2one-call).
* [Set STUNner as the ICE server](#configuring-webrtc-clients) in your WebRTC clients.
* ...
* Profit!!
Expand Down Expand Up @@ -551,45 +551,45 @@ applications into Kubernetes.

### Basics

* [Opening a UDP tunnel via STUNner](/examples/simple-tunnel/README.md): This introductory tutorial
* [Opening a UDP tunnel via STUNner](/docs/examples/simple-tunnel/README.md): This introductory tutorial
shows how to tunnel an external connection via STUNner to a UDP service deployed into
Kubernetes. The demo can be used to quickly check and benchmark a STUNner installation.

### Headless deployment mode

* [Direct one to one video call via STUNner](/examples/direct-one2one-call/README.md): This
* [Direct one to one video call via STUNner](/docs/examples/direct-one2one-call/README.md): This
tutorial showcases STUNner acting as a TURN server for two WebRTC clients to establish
connections between themselves, without the mediation of a media server.

### Media-plane deployment model

* [One to one video call with Kurento](/examples/kurento-one2one-call/README.md): This tutorial
* [One to one video call with Kurento](/docs/examples/kurento-one2one-call/README.md): This tutorial
shows how to use STUNner to connect WebRTC clients to a media server deployed into Kubernetes
behind STUNner in the [media-plane deployment model](/docs/DEPLOYMENT.md). All this happens
*without* modifying the media server code in any way, just by adding 5-10 lines of
straightforward JavaScript to configure clients to use STUNner as the TURN server.
* [Magic mirror with Kurento](/examples/kurento-magic-mirror/README.md): This tutorial has been
* [Magic mirror with Kurento](/docs/examples/kurento-magic-mirror/README.md): This tutorial has been
adopted from the [Kurento](https://www.kurento.org) [magic
mirror](https://doc-kurento.readthedocs.io/en/stable/tutorials/node/tutorial-magicmirror.html)
demo, deploying a basic WebRTC loopback server behind STUNner with some media processing
added. In particular, the application uses computer vision and augmented reality techniques to
add a funny hat on top of faces.
* [Video-conferencing with LiveKit](/examples/livekit/README.md): This tutorial helps you deploy
* [Video-conferencing with LiveKit](/docs/examples/livekit/README.md): This tutorial helps you deploy
the [LiveKit](https://livekit.io) WebRTC media server behind STUNner. The docs also show how to
obtain a valid TLS certificate to secure your signaling connections, courtesy of the
[cert-manager](https://cert-manager.io) project, [nip.io](https://nip.io) and [Let's
Encrypt](https://letsencrypt.org).
* [Video-conferencing with Jitsi](/examples/jitsi/README.md): This tutorial helps you deploy a
* [Video-conferencing with Jitsi](/docs/examples/jitsi/README.md): This tutorial helps you deploy a
fully fledged [Jitsi](https://jitsi.org) video-conferencing service into Kubernetes behind
STUNner. The docs also show how to obtain a valid TLS certificate to secure your signaling
connections, using [cert-manager](https://cert-manager.io), [nip.io](https://nip.io) and [Let's
Encrypt](https://letsencrypt.org).
* [Cloud-gaming with Cloudretro](/examples/cloudretro/README.md): This tutorial lets you play Super
* [Cloud-gaming with Cloudretro](/docs/examples/cloudretro/README.md): This tutorial lets you play Super
Mario or Street Fighter in your browser, courtesy of the amazing
[CloudRetro](https://cloudretro.io) project and, of course, STUNner. The demo also presents a
simple multi-cluster setup, where clients can reach the game-servers in their geographical
locality to minimize latency.
* [Remote desktop access with Neko](/examples/neko/README.md): This demo showcases STUNner
* [Remote desktop access with Neko](/docs/examples/neko/README.md): This demo showcases STUNner
providing an ingress gateway service to a remote desktop application. We use
[neko.io](https://neko.m1k1o.net) to run a browser in a secure container inside the Kubernetes
cluster, and stream the desktop to clients via STUNner.
Expand All @@ -609,7 +609,7 @@ notable limitations at this point are as follows.
deviates from the prescribed behavior in some cases, all in the name of simplifying the
configuration process. The [STUNner Kubernetes gateway
operator](https://github.com/l7mp/stunner-gateway-operator) docs contain a [detailed
list](https://github.com/l7mp/stunner-gateway-operator/README.md#caveats) on the differences.
list](https://github.com/l7mp/stunner-gateway-operator#caveats) on the differences.
* STUNner supports *multiple parallel GatewayClass hierarchies*, each deployed into a separate
namespace with a separate GatewayClass an a separate dataplane. This mode can be useful for
testing new STUNner versions or canary-upgrades and A/B testing of a new media server version. At
Expand All @@ -625,8 +625,8 @@ notable limitations at this point are as follows.
* v0.13: Observability: Prometheus + Grafana dashboard.
* v0.15: Performance: per-allocation CPU load-balancing for UDP
* v0.16: Management: managed STUNner dataplane.
* v0.17: Performance: eBPF TURN acceleration.
* v1.0: GA
* v1.1: Performance: eBPF TURN acceleration.
* v2.0: Service mesh: adaptive scaling & resiliency

## Help
Expand Down
63 changes: 8 additions & 55 deletions docs/CONCEPTS.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,27 @@
# Concepts

In this guide we describe STUNner's architecture and the most important components of an
operational STUNner installation.
In this guide we describe STUNner's architecture and the most important components of an operational STUNner installation.

## Architecture

A STUNner installation consists of two parts, a *control plane* and a *dataplane*. The control
plane consists of declarative policies specifying the way STUNner should route WebRTC media traffic
to the media servers, plus a gateway operator that renders the high-level policies into an actual
dataplane configuration. The dataplane in turn comprises one or more `stunnerd` pods, responsible
for actually ingesting media traffic into the cluster through a STUN/TURN server. Since the TURN
service underlying STUNner is agnostic to NATs, STUNner can inject clients' media traffic into the
private Kubernetes pod network, addressing all NAT traversals (client-side and server-side) in a
single go.
A STUNner installation consists of two parts, a *control plane* and a *dataplane*. The control plane consists of declarative policies specifying the way STUNner should route WebRTC media traffic to the media servers, plus a gateway operator that renders the high-level policies into an actual dataplane configuration. The dataplane in turn comprises one or more `stunnerd` pods, responsible for actually ingesting media traffic into the cluster through a STUN/TURN server. Since the TURN service underlying STUNner is agnostic to NATs, STUNner can inject clients' media traffic into the private Kubernetes pod network, addressing all NAT traversal steps (client-side and server-side) in a single go.

![STUNner architecture](img/stunner_arch_big.svg)

The unit of the STUNner configuration is a [designated Kubernetes
namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces) that holds
the control plane configuration and the dataplane pods. You can run multiple STUNner deployments
side-by-side by installing a separate dataplane into a each namespace and defining a distinct
gateway hierarchy to configure each dataplane separately. Currently there is no way to share
configuration (e.g., UDPRoutes) between different namespaces.
The unit of the STUNner configuration is a [designated Kubernetes namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces) that holds the control plane configuration and the dataplane pods. You can run multiple STUNner deployments side-by-side by installing a separate dataplane into a each namespace and defining a distinct gateway hierarchy to configure each dataplane separately.

### Control plane

The STUNner control plane consists of the following components:

* **Gateway hierarchy:** A gateway hierarchy is a collection of [Kubernetes Custom
Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources)
that together describe the way media traffic should enter the cluster, including public IP
addresses and ports clients can use to reach STUNner, TURN credentials, forwarding rules, etc. The
anchor of the gateway hierarchy is the GatewayClass object, and the rest of the resources form a
complete hierarchy underneath it: the GatewayConfig describes general STUNner configuration,
Gateways define the port and transport protocol for each TURN server listener, and UDPRoutes point
to the backend services client traffic should be forwarded to. See [here](GATEWAY.md) for a full
reference.
* **Gateway hierarchy:** A gateway hierarchy is a collection of [Kubernetes Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources) that together describe the way media traffic should enter the cluster, including public IP addresses and ports clients can use to reach STUNner, TURN credentials, forwarding rules, etc. The anchor of the gateway hierarchy is the GatewayClass object, and the rest of the resources form a complete hierarchy underneath it: the GatewayConfig describes general STUNner configuration, Gateways define the port and transport protocol for each TURN server listener, and UDPRoutes point to the backend services client traffic should be forwarded to. See [here](GATEWAY.md) for a full reference.

* **Gateway operator:** The main purpose of the gateway operator is to watch gateway hierarchies
for change and, once a custom resource is added or modified by the user, render a new dataplane
configuration. This configuration is then mapped into the filesystem of the `stunnerd` pods running
in the same namespace, so that each `stunnerd` instance will use the most recent configuration. The
STUNner Helm chart [automatically install](INSTALL.md) the gateway operator; more information can
be found [here](https://github.com/l7mp/stunner-gateway-operator).
* **Gateway operator:** The main purpose of the gateway operator is to watch gateway hierarchies for change and, once a custom resource is added or modified by the user, render a new dataplane configuration. This configuration is then mapped into the filesystem of the `stunnerd` pods running in the same namespace, so that each `stunnerd` instance will use the most recent configuration. The STUNner Helm chart [automatically installs](INSTALL.md) the gateway operator; more information can be found [here](https://github.com/l7mp/stunner-gateway-operator).

* **STUNner ConfigMap:** The STUNner ConfigMap contains the running dataplane configuration. Of
course, we could let the `stunnerd` pods themselves to watch the control plane for changes, but
this would run into scalability limitations for large deployments. Instead, we separate the control
plane and the dataplane, which brings lots of cool
[benefits](https://en.wikipedia.org/wiki/Software-defined_networking). The STUNner ConfigMap is
usually named as `stunnerd-config`, but you can override this from the GatewayConfig.
* **STUNner ConfigMap:** The STUNner ConfigMap contains the running dataplane configuration. Of course, we could let the `stunnerd` pods themselves to watch the control plane for changes, but this would run into scalability limitations for large deployments. Instead, we separate the control plane and the dataplane, which brings cool [benefits](https://en.wikipedia.org/wiki/Software-defined_networking). The STUNner ConfigMap is usually named as `stunnerd-config`, but you can override this from the GatewayConfig.

## Dataplane

The STUNner dataplane is comprised of a fleet of `stunnerd` pods. These pods actually implement the
TURN server, using the configuration available in the STUNner ConfigMap which is mapped into the
pods' filesystem dynamically. Then, `stunnerd` will watch for changes in the config file and, once
a change is detected, it [reconciles](https://kubernetes.io/docs/concepts/architecture/controller)
the dataplane to match the new user policies.
The STUNner dataplane is comprised of a fleet of `stunnerd` pods. These pods actually implement the TURN server, using the configuration available in the STUNner ConfigMap which is mapped into the pods' filesystem dynamically. Then, `stunnerd` will watch for changes in the config file and, once a change is detected, it [reconciles](https://kubernetes.io/docs/concepts/architecture/controller) the dataplane to match the new user policies.

The `stunnerd` daemon itself is essentially a simple TURN server on top of
[pion/turn](https://github.com/pion/turn) written in Go. The daemon will instantiate a separate
*TURN listener* for each Gateway listener in the gateway hierarchy to terminate clients' TURN
sessions, a *cluster* per each UDPRoute to forward packets to the backend services (e.g., to the
media servers), with some ancillary administrative and authentication mechanisms in place to check
client credentials before admitting traffic into the cluster, logging, etc. There is a one-to-one
mapping between the control-plane Gateway listeners and the `stunnerd` TURN listeners, as well as
between the UDPRoute resources and `stunnerd`'s clusters. Whenever you modify a Gateway (UDPRoute),
the gateway operator renders a new dataplane configuration with the modified listener (cluster,
respectively) specs and the `stunnerd` pods reconcile their internal state to the new
configuration. You are free to scale the dataplane to as many `stunnerd` pods as you like:
Kubernetes will make sure that new client connections are distributed evenly over the scaled-out
STUNner dataplane.
The `stunnerd` daemon itself is essentially a simple TURN server on top of [pion/turn](https://github.com/pion/turn) written in Go. The daemon will instantiate a separate *TURN listener* for each Gateway listener in the gateway hierarchy to terminate clients' TURN sessions, a *cluster* per each UDPRoute to forward packets to the backend services (e.g., to the media servers), with some ancillary administrative and authentication mechanisms in place to check client credentials before admitting traffic into the cluster, logging, etc. There is a one-to-one mapping between the control-plane Gateway listeners and the `stunnerd` TURN listeners, as well as between the UDPRoute resources and `stunnerd`'s clusters. Whenever you modify a Gateway (UDPRoute), the gateway operator renders a new dataplane configuration with the modified listener (cluster, respectively) specs and the `stunnerd` pods reconcile their internal state to the new configuration. You are free to scale the dataplane to as many `stunnerd` pods as you like: Kubernetes will make sure that new client connections are distributed evenly over the scaled-out STUNner dataplane.
Loading

0 comments on commit 58186cb

Please sign in to comment.