Noob guide to making this work

[Jimmy-SafeCash]

Hey guys,

Trying to get this to work but not having much luck. I can reach the page, but when I click "Join Meeting" nothing happens. I suspect this is due to the StatefulSet stuck at "In Progress".

To be clear, I'm very new to k8s and helm, and the instructions aren't meant for someone as newbie as me. What I've tried:

• Created a folder called jitsi
• Copied the values.yaml from the root folder and customised to my liking
• Run the helm install command (no errors).

I would like to use prosody (which I believe is used for auth?) but there is no pod and the statefulset is stuck pendning. I can see there is a prosody/values.yaml folder but I've no idea how to use/incorporate that. I'm guessing it needs to be persistent but I'm not surer how or where to include.

I do notice this error on prosody StatefulSet: persistentvolumeclaims "prosody-data-jitsi-prosody-0" is forbidden: Internal error occurred: 2 default StorageClasses were found. I do have longhorn installed and would like to use that.

Any help would be greatly appreciated (I've been using jitsi in docker for years and looking to expand my k8s knowledge). @sapkra

[Jimmy-SafeCash]

p.s., I've no idea what should go in "server: " in the prosody section. I'm just running a k3s cluster locally with metallb.

[Jimmy-SafeCash]

Update: I manually copied:
persistence: enabled: true size: 3G storageClassName: longhorn
into values.yaml and it has now deployed.

Now trying to figure out how to create a user as there's no prosody config file. Previously I specified to use internal_auth (for docker) but not sure where to put the variable here.

[Jimmy-SafeCash]

Even with Auth=false set I still cannot create a conference without being prompted for username/password

[Jimmy-SafeCash]

Update, I've now successfully created a user and can log in but I receive a websocket error on second person joining. My vaules.yaml is below - thanks in advance.

[spijet]

Hello @Jimmy-SafeCash!

Can you please share your values.yaml (minus the sensitive data, like keys and passwords) here? I have been using it without authentication and persistence enabled in Prosody for a couple of years now, without problems on that part.

[Jimmy-SafeCash]

Hi @spijet , thank you for your response. Glad to know it's just my problem and hopefully it can be resolved. Here's the values.yaml

# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

global:
  # Set your cluster's DNS domain here.
  # "cluster.local" should work for most environments.
  # Set to "" to disable the use of FQDNs (default in older chart versions).
  clusterDomain: cluster.local
  podLabels: {}
  podAnnotations: {}
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

enableAuth: true
enableGuests: true
publicURL: "redacted"

tz: Europe/London

image:
  pullPolicy: IfNotPresent

## WebSocket configuration:
#
#  Both Colibri and XMPP WebSockets are disabled by default,
#  since some LoadBalancer / Reverse Proxy setups can't pass
#  WebSocket connections properly, which might result in breakage
#  for some clients.
#
#  Enable both Colibri and XMPP WebSockets to replicate the current
#  upstream `meet.jit.si` setup. Keep both disabled to replicate
#  older setups which might be more compatible in some cases.
websockets:
  ## Colibri (JVB signalling):
  colibri:
    enabled: true
  ## XMPP (Prosody signalling):
  xmpp:
    enabled: true

web:
  replicaCount: 1
  image:
    repository: jitsi/web

  extraEnvs: {}
  service:
    type: ClusterIP
    port: 80
    externalIPs: []

  ingress:
    enabled: false
    # ingressClassName: "nginx-ingress-0"
    annotations: {}
      # kubernetes.io/tls-acme: "true"
    hosts:
    - host: jitsi.local
      paths: ['/']
    tls: []
    #  - secretName: jitsi-web-certificate
    #    hosts:
    #      - jitsi.local

  # Useful for ingresses that don't support http-to-https redirect by themself, (namely: GKE),
  httpRedirect: false

  # When tls-termination by the ingress is not wanted, enable this and set web.service.type=Loadbalancer
  httpsEnabled: false

  ## Resolver IP for nginx.
  #
  #  Starting with version `stable-8044`, the web container can
  #  auto-detect the nameserver from /etc/resolv.conf.
  #  Use this option if you want to override the nameserver IP.
  #
  # resolverIP: 10.43.0.10

  livenessProbe:
    httpGet:
      path: /
      port: 80
  readinessProbe:
    httpGet:
      path: /
      port: 80

  podLabels: {}
  podAnnotations: {}
  podSecurityContext: {}
    # fsGroup: 2000

  securityContext: {}
    # capabilities:
    #   drop:
    #   - ALL
    # readOnlyRootFilesystem: true
    # runAsNonRoot: true
    # runAsUser: 1000

  resources: {}
    # We usually recommend not to specify default resources and to leave this as a conscious
    # choice for the user. This also increases chances charts run on environments with little
    # resources, such as Minikube. If you do want to specify resources, uncomment the following
    # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
    # limits:
    #   cpu: 100m
    #   memory: 128Mi
    # requests:
    #   cpu: 100m
    #   memory: 128Mi

  nodeSelector: {}

  tolerations: []

  affinity: {}

jicofo:
  replicaCount: 1
  image:
    repository: jitsi/jicofo

  xmpp:
    user: focus
    password:
    componentSecret:

  livenessProbe:
    tcpSocket:
      port: 8888
  readinessProbe:
    tcpSocket:
      port: 8888

  podLabels: {}
  podAnnotations: {}
  podSecurityContext: {}
  securityContext: {}
  resources: {}
  nodeSelector: {}
  tolerations: []
  affinity: {}
  extraEnvs: {}

jvb:
  replicaCount: 1
  image:
    repository: jitsi/jvb

  xmpp:
    user: jvb
    password: password

  ## Set public IP addresses to be advertised by JVB.
  #  You can specify your nodes' IP addresses,
  #  or IP addresses of proxies/LoadBalancers used for your
  #  Jitsi Meet installation. Or both!
  #
  #  Note that only the first IP address will be used for legacy
  #  `DOCKER_HOST_ADDRESS` environment variable.
  #
  publicIPs:
    - 192.168.200.15
  #   - 5.6.7.8
  ## Use a STUN server to help some users punch through some
  #  especially nasty NAT setups. Usually makes sense for P2P calls.
  stunServers: 'meet-jit-si-turnrelay.jitsi.net:443'
  ## Try to use the hostPort feature:
  #  (might not be supported by some clouds or CNI engines)
  useHostPort: false
  ## Use host's network namespace:
  #  (not recommended, but might help for some cases)
  useHostNetwork: false
  ## UDP transport port:
  UDPPort: 10000
  ## Use a pre-defined external port for NodePort or LoadBalancer service,
  #  if needed. Will allocate a random port from allowed range if unset.
  #  (Default NodePort range for K8s is 30000-32767)
  # nodePort: 10000
  service:
    enabled:
    type: ClusterIP
    externalIPs: []
    ## Annotations to be added to the service (if LoadBalancer is used)
    ##
    annotations: {}

  breweryMuc: jvbbrewery

  livenessProbe:
    httpGet:
      path: /about/health
      port: 8080
  readinessProbe:
    httpGet:
      path: /about/health
      port: 8080

  podLabels: {}
  podAnnotations: {}
  podSecurityContext: {}
  securityContext: {}
  resources: {}
  nodeSelector: {}
  tolerations: []
  affinity: {}
  extraEnvs: {}

  metrics:
    enabled: false
    prometheusAnnotations: false
    image:
      repository: docker.io/systemli/prometheus-jitsi-meet-exporter
      tag: 1.2.1
      pullPolicy: IfNotPresent
    serviceMonitor:
      enabled: true
      selector:
        release: prometheus-operator
      interval: 10s
      # honorLabels: false
    resources:
      requests:
        cpu: 10m
        memory: 16Mi
      limits:
        cpu: 20m
        memory: 32Mi

octo:
  enabled: false


jibri:
  ## Enabling Jibri will allow users to record
  ## and/or stream their meetings (e.g. to YouTube).
  enabled: false

  ## Enable multiple Jibri instances.
  ## If enabled (i.e. set to 2 or more), each Jibri instance
  ## will get an ID assigned to it, based on pod name.
  replicaCount: 1

  ## Enable persistent storage for local recordings.
  ## If disabled, jibri pod will use a transient
  ## emptyDir-backed storage instead.
  persistence:
    enabled: false
    size: 4Gi
    ## Set this to existing PVC name if you have one.
    existingClaim:
    storageClassName:

  shm:
    ## Set to true to enable "/dev/shm" mount.
    ## May be required by built-in Chromium.
    enabled: false
    ## If "true", will use host's shared memory dir,
    ## and if "false" — an emptyDir mount.
    # useHost: false
    # size: 256Mi

  image:
    repository: jitsi/jibri

  podLabels: {}
  podAnnotations: {}

  breweryMuc: jibribrewery
  timeout: 90

  ## jibri XMPP user credentials:
  xmpp:
    user: jibri
    password: password

  ## recorder XMPP user credentials:
  recorder:
    user: recorder
    password: password

  livenessProbe:
    exec:
      command:
        - /bin/bash
        - "-c"
        - >-
          curl -sq localhost:2222/jibri/api/v1.0/health
          | jq '.status.health.healthStatus'
          | grep -qF HEALTHY

  readinessProbe:
    exec:
      command:
        - /bin/bash
        - "-c"
        - >-
          curl -sq localhost:2222/jibri/api/v1.0/health
          | jq '.status.health.healthStatus'
          | grep -qF HEALTHY

  extraEnvs: {}

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: jimmy

xmpp:
  domain: meet.jitsi
  authDomain: auth
  mucDomain: muc
  internalMucDomain: internal-muc
  guestDomain: guest

extraCommonEnvs: {}

prosody:
  enabled: true
  server: xmpp.meet.jitsi
  extraEnvFrom:
  - secretRef:
      name: '{{ include "prosody.fullname" . }}-jicofo'
  - secretRef:
      name: '{{ include "prosody.fullname" . }}-jvb'
  - configMapRef:
      name: '{{ include "prosody.fullname" . }}-common'
  ## Uncomment this if you want to use jibri:
  # - secretRef:
  #     name: '{{ include "prosody.fullname" . }}-jibri'
  image:
    repository: jitsi/prosody
    tag: 'stable-8044'
  persistence:
    enabled: true
    size: 3G
    storageClassName: longhorn

Things I've altered from default that are probably wrong:

• domains for xmpp: section
• adding the persistence to prosody
• enabling auth, colibri, and xmpp

[spijet]

I see that you've set your public IP here to 192.168.200.15 — is this intentional? JVB won't be able to communicate with clients connecting from the internet when using this address.

As for other values, you can try this minimal values.yaml and see if it works for you (it will be applied "on top" of chart's defaults):

websockets:
  colibri:
    enabled: true
  xmpp:
    enabled: true

jvb:
  # This will expose JVB to the world using hostPort feature,
  # so you won't need to come up with weird UDP proxying schemes:
  useHostPort: true

  # Use a custom UDP port (in case if your k8s doesn't allow to bind
  # to hostPorts < 32000
  UDPPort: 32768
  publicIPs:
    - XXX.YYY.ZZZ.AAA # <- your server's (preferably public) IP
  stunServers: "meet-jit-si-turnrelay.jitsi.net:443,stun1.l.google.com:19302,stun2.l.google.com:19302,stun3.l.google.com:19302,stun4.l.google.com:19302"

You don't really need to change the default Jitsi domains here unless you know what you're doing, as these domains are only used by Jitsi for its' internal communications (e.g. JVB<->Prosody) and not for users/clients.

[Jimmy-SafeCash]

Thanks @spijet, I will try this later.

The 192.168.200.15 is the IP of my loadbalancer where traefik sits. I have an ingress route from that to the service.

[Jimmy-SafeCash]

@spijet I was able to have the server run first time with the above (I did need to add the storageclass as longhorn for prosody as there's no default, but that's fine).

Now I have the common problem of wss failure when 3 people join which I know is a UDP routing problem. I'm currently trying to route it through traefik (deployed via helm). If that fails I can always revert to a loadbalancer setup and do it that way.

Don't suppose you have a handy template for jvb 10000 udp on traefik with this setup?

[spijet]

Unfortunately I don't, as I used Jitsi Meet in two setups as of yet: one with NGINX Ingress Controller acting as a stand-in UDP proxy (turned out to be not so good, had plenty of packet loss) and one with JVB's UDP port published as a hostPort right there on the node, which is working flawlessly for me so far.

It used to be a bit easier back when JVB supported TCP transport, but it turned out to be flawed in terms of performance and security, so it was removed in favour of TURN server setups. I'm currently waiting until Jitsi devs come up with a somewhat standard Docker setup of Jitsi Meet with coturn server before I start implementing it as part of this Helm chart. :)

[Jimmy-SafeCash]

I've now figured out how to successfully create the UDP ingress route in Traefik. Everything is working fine now, thanks for the help.

[psic4t]

Thanks for sharing! DId you use UDP port 32768 without UDP proxying or 10000 like you wrote above?

I can get both to work, but only in bad quality and without websockets enabled (no video if I enable colibri).

[spijet]

@mdebus:

I can get both to work, but only in bad quality and without websockets enabled (no video if I enable colibri).

Are you sure that your setup uses properly-setup HTTPS and a WSS-compatible proxy? If not, you'd get better results by disabling WS support for both Colibri and XMPP.

[spijet]

@Jimmy-SafeCash: with useHostPort: true you don't have to use Traefik for UDP proxying in the first place, since JVB port is already exposed directly off your k8s nodes. Unless these nodes are not directly reachable from the Internet, using hostPort would likely yield much better results.

From my experience it's better to leave as few intermediate points between users and JVB as possible, to reduce the chance of weird problems with jitter and packet loss. I tried using NGINX Ingress controller to proxy UDP traffic between clients and JVB and the result was… underwhelming, to say the least: almost every single client experienced a 30% and higher packet loss and had to watch a lo-fi Minecraft livestream instead of proper video. :)

[psic4t]

@spijet I'm using Traefik. HTTPS works. Do I need additional configuration for WS?

[spijet]

@mdebus, you can check if WS actually works by opening your browser's (or Electron App's) dev console and checking if there are any successful WS requests with HTTP 101 response code and any active WS sessions with messages in them as well. Also make sure to check if it's the same when you have 2 people in a room (P2P mode) and when you have 3+ people in a room (server mode).

Successful WSS requests in Jitsi Meet app

Here you can see two requets: one for XMPP and (later) one for Colibri.

Messages in XMPP WebSocket

Messages in Colibri WebSocket