Build hubs community edition on AWS using helm and terraform

This repository manages the AWS resources needed to build the community version as code in terraform and deploys them using helm.

By writing the AWS resources required for Hubs Community edition as IaC in terraform, resources can be built in one shot with terraform apply.

This is useful for preventing simple mistakes in configuring manually in the AWS console and for deploying to multiple environments.

This repository depends on mozilla-hubs-ce-chart.

This repository is based on the following document. These documents may be updated.

• Hubs Community Edition Is Here!
• Deploying Mozilla Hubs CE on AWS with Ease: A Guide to the Scale Edition Helm Chart
• Deploying Community Edition on AWS with Alex Griggs' Helm Chart

My Local Environment

• MacBook Pro
• CPU: Apple M3 Pro
• macOS: Sonoma 14.2.1

This is only my confirmed environment, so other environments can be deployed.

Prerequirements

• Kubectl
• Awscli
• Helm
• Terraform

Getting Started

1. Configure DNS on Route53

Configure your DNS according to Step 1: Configuring your DNS on AWS's Route53

2. Setup AWS Infrastructure

1. Create S3 bucket for infrastructure state management files (tfstate)

   1. Execute the following commands in a terminal {environment}: AWS Environment Name （ex. develop）

      aws s3api create-bucket --bucket ov-hubs-ce-tfstate-{environment} --region us-east-1

      💡 Only when region is not us-east-1, specify
      --create-bucket-configuration LocationConstraint={region}

2. Create tfbackend file

   1. Create {env_name}.tfbackend

      touch {env_name}.tfbackend

   2. Enter and update values in the {env_name}.tfbackend file, referring to the sample configuration file

   3. Create{env_name}.tfvars

      touch {env_name}.tfvars

   4. Enter and update values in the {env_name}.tfvars file, referring to the sample configuration file. The ENV_NAME_TAG should match the {env_name}.

      💡 When adding environment variables, in addition to editing the {env_name}.tfvars file, it is necessary to define the variables in variables.tf

3. Build environment on AWS

1. Terraform Initialize

   sh terraform.sh {env_name} init

2. Format

   sh terraform.sh {env_name} fmt

3. Validation

   sh terraform.sh {env_name} validate

4. Build Environment

   sh terraform.sh {env_name} apply

   A list of resources to be created will be output. If there are no problems, enter "yes".

   When the process is completed, resources such as VPCs and EKSs whose definitions are created in the AWS environment.

4. Configure SMTP on SES

Configure your SMTP according to Step 2: Configuring your SMTP on AWS's Simple Email Service (SES)

5. Setup Helm Chart

💡 The helm chart configuration basically follows the document below. **[Deploying Mozilla Hubs CE on AWS with Ease: A Guide to the Scale Edition Helm Chart](https://hubs.mozilla.com/labs/deploying-mozilla-hubs-ce-on-aws-with-ease-a-guide-to-the-scale-edition-helm-chart/)**

• Setup Helm

  1. Create a namespace named hcce in the EKS cluster

     kubectl create ns hcce

  2. Create a namespace named security in the EKS cluster

     kubectl create ns security
     

  3. Add the jetstack repository to helm and install cert-manager in the namespace security.

     helm repo add jetstack https://charts.jetstack.io
     helm repo update
     helm install cert-manager jetstack/cert-manager \
     --namespace security \
     --set ingressShim.defaultIssuerName=letsencrypt-issuer \
     --set ingressShim.defaultIssuerKind=ClusterIssuer \
     --set installCRDs=true

  4. Create {env_name}-cluster-issuer.yaml

     touch {env_name}-cluster-issuer.yaml

  5. Update the email to the administrator's email address in {env_name}-cluster-issuer.yaml

  6. Apply Issuer to EKS

     kubectl apply -f '{env_name}-cluster-issuer.yaml'

  7. Get the helm chart resources for hubs ce by git clone from the repository for Mozilla Hubs CE Chart

     git clone https://github.com/hubs-community/mozilla-hubs-ce-chart.git

  8. Copy the event file with the following command

     cp mozilla-hubs-ce-chart/values.scale.yaml {env_name}-values-event.yaml

  9. update render_helm.sh in mozilla-hubs-ce-chart folder

     1. put random strings in the following three variables in the render_helm.sh file.

        NODE_COOKIE="node-{YOUR_NODE_COOKIE_ID}"
        GUARDIAN_KEY="{YOUR_GUARDIAN_KEY}"
        PHX_KEY="{YOUR_PHX_KEY}"

     2. Update DB authentication information in render_helm.sh file

        DB_USER="postgres"
        DB_PASS="123456"
        EXT_DB_HOST="pgsql"

        • DB_USER: DB_MASTER_USERNAME specified in {env_name}.tfvars
        • DB_PASS: DB_MASTER_PASSWORD specified in {env_name}.tfvars
        • EXT_DB_HOST: Value output when the following command is executed

          sh terraform.sh {env_name} output -raw rds_writer_endpoint
          

     3. Update SMTP information in render_helm.sh file
        Update the settings based on the information configured in "4. Configure SMTP on SES”

        SMTP_SERVER: SMTP endpoint
        SMTP_USER: SMTP username as listed in the csv you downloaded your credentials (note that this is not an IAM user)
        SMTP_PASS: SMTP password from the csv downloaded with the SMTP credentials
        

  10. Run the edited render_helm.sh

      ./mozilla-hubs-ce-chart/render_helm.sh {domain} {mail_address}

  11. Check the contents of the generated . Check the contents of the /config.yaml file.

  12. Update configs > data in the {env_name}-values-event.yaml file

      1. In the {env_name}-values-event.yaml file, near line 118 replace the part below where it says # Get the following from render_helm.sh
      💡 If you have difficulty understanding the changes, check out

      Deploying Community Edition on AWS with Alex Griggs' Helm Chart
      17 minutes:around 35 seconds

  13. Update defaultCert > data in the {env_name}-values-event.yaml file

  14. Copy tls.crt, tls.key in the /config.yaml file.

  15. Replace tls.crt, tls.key in defaultCert > data near line 165 in file {env_name}-values-event.yaml

  16. Also, change enabled under defaultCert from false to true

  17. In the file {env_name}-values-event.yaml, replace the following value near line 6

      global:
        domain: &HUBS_DOMAIN "{YOUR_HUBS_DOMAIN}"
        adminEmail: &ADMINEMAIL "{ADMIN_EMAIL_ADDRESS}"
      

  18. Change certificate settings

      1. Open mozilla-hubs-ce-chart/charts/haproxy/templates/deployment.yaml

      2. Change near line 39.

         1. before

            - --default-ssl-certificate={{ .Release.Namespace }}/cert-**hcce**

         2. after

            - --default-ssl-certificate={{ .Release.Namespace }}/cert-**{{ .Values.global.domain }}**

         💡 Failure to follow this procedure will result in a warning to the user that the communication is not protected.

• EFS mount settings

  💡 If you do not use efs, you can run the application without following the steps below, leaving enabled: false. However, in that case, assets such as scenes and logos will be deleted when the node (EC2 instance) of the EKS cluster is deleted.

  1. Update the efs setting near line 17 in the file {env_name}-values-event.yaml

     enabled: true

     fileSystemId: Value output when executing sh ./terraform.sh {env_name} output -raw efs_id

     aws:
         efs:
           enabled: false
           isDynamicProvisioning: false
           fileSystemId: fs-000000000000

  2. Changed to not use pgsql

     1. Add the following to the last line in the file {env_name}-values-event.yaml

        # Add
        pgsql:
          enabled: false

6. Deploy Hubs CE

1. helm install

   helm install moz -f {env_name}-values-event.yaml ./mozilla-hubs-ce-chart --namespace=hcce

2. Check the external IP of the resource to be created in EKS

   kubectl get --namespace hcce svc -w haproxy-lb

3. Update A record for the Hubs application in Route53

   1. open the Route53 console

   2. select "Host Zone" on the left side and select the domain you specified when deploying

   3. update the values of the 4 A records created in "1. Configure DNS on Route53" as follows:

      1. select the target A record and click "Edit Record" displayed in the upper right corner
      2. update as follows and save
         1. record type: A
         2. alias: on
         3. traffic routing destination: alias to Application Load Balancer and Classic Load Balancer
         4. region: us-east-1
         5. load balancer: dualstack.{external IP of EKS}

      Perform the above steps for all four A records ({domain}, assets.{domain}, cors.{domain}, stream.{domain})

7. Check Hubs Application

• Check the status of EKS Pod

  kubectl get pods -n hcce

  💡 If some pods do not become Running after 10 minutes or more, there may be a problem. Please check the information of each pod and debug it by referring to the following.

  • Check logs for each pod

    💡 `{pod_name}` can be found in the output of `kubectl get pods -n hcce`

    kubectl logs pod {pod_name} -n hcce

  • Check the information for each pod (e.g., if the pod is Pending, check this)

    kubectl describe pod {pod_name} -n hcce

8. Confirmation of operation

Confirm the following operations

• Access to the domain After accessing the domain, the sign-in screen appears. Can sign in by entering the administrator's e-mail address
• Can create a room
• Can listen to other users' voices in the room

Troubleshooting

• Certificate verification does not complete successfully and domain cannot be accessed
  • → Temporarily turn off http → https redirects In the file /mozilla-hubs-ce-chart/charts/haproxy/values.yamlchange ssl-redirect from true to false at line 204

    ssl-redirect: "false" # true -> false