Skip to content

Latest commit

 

History

History
102 lines (75 loc) · 5.71 KB

platform-set-up.md

File metadata and controls

102 lines (75 loc) · 5.71 KB
Raw

Platform set-up

Ingress DNS

There are two DNS zones for ingress DNS:

  • teacherservices.cloud (prod zone)
  • development.teacherservices.cloud (dev zone)

Zone Build

make {dev/prod}-domain domains-infra-{plan/apply}

There is also an NS record for delegation from teacherservices.cloud to development.teacherservices.cloud, which is created if delegation_name and delegation_ns are set in tscp.tfvars.json

If the development zone NS records are changed for any reason, then these variables must be updated manually, and the prod zone updated.

Register new domain

The Infrastructure Operations team registered the teacherservices.cloud domain in route53. It is valid for 1 year and renews automatically. A senior civil servant is recorded as contact and receives renewal notices. The domains is configured with the nameservers for the teacherservices.cloud zone, built above.

Ingress DNS record

We use a wildcard DNS record for the ingress domain, which is the default domain of any application deployed to the cluster.

For the development clusters, on cluster build this record will be automatically created in the dev DNS zone.

Ingress TLS

We use a wildcard certificate for the default domain of any application deployed to the cluster. Certs are created from Azure Keyvault (manually), and then loaded into the cluster using terraform on cluster build.

Initial set up requires manual steps. Then renewals are automated.

Add Globalsign domain

The teacherservices.cloud domain is created in route53 and owned by the Infrastructure Operations team. The following steps were required for allowing teacherservices.cloud top domain. They won't be required for new clusters under the same top domain.

Generate value for DNS record

  1. Login to GlobalSign and select Managed SSL, then Add Domain under O: Department for Education
  2. Enter "teacherservices.cloud"

Follow instructions after Domain renewal.

Domain renewal

The domain expires every year and must be renewed manually.

  1. Login to GlobalSign, select Managed SSL, then Manage Domains under O: Department for Education
  2. Click on the Renew button against the teacherservices.cloud domain
  3. Verify the details and click Continue

Follow instructions in the next section.

Continuation

  1. Add point of contact (a senior Civil Servant)
  2. Select DNS Verification (on next page)
  3. Confirm the details and click Complete. The feedback should look something like: 
    Thank you for submitting your application. Your order number is XXX.
Domain: teacherservices.cloud
The DNS value for this domain is: _globalsign-domain-verification=XXXXXXXXXXXXXXXXXX
  4. Update the value of _globalsign-domain-verification in custom_domains/terraform/infrastructure/config/tscp.tfvars.json

Verify Domain

  1. Login to GlobalSign, select Managed SSL, then Manage Domains under O: Department for Education
  2. Search for teacherservices.cloud and select the green check mark
  3. Select verify domain
  4. You should receive a feedback: "Your domain has been successfully verified."

Create Certificate in Azure

Follow the technical guidance on certificates.

  1. Ensure CAA record (for teacherservices.cloud) allows GlobalSign. See terraform configuration or GlobalSign documentation.
  2. Navigate to Key Vaults, select the applicable Key vault
  3. Create a new certificate using the defaults from the above documentation and the following properties ([config] refers to the value defined in variables.tf):
    • Certificate Name: [config]-teacherservices-cloud (for development: cluster[N]-development-teacherservices-cloud)
    • Subject CN: *.[config].teacherservices.cloud (for development: *.cluster[N].development.teacherservices.cloud)
    • DNS Names: 0
    • Validity: 12 months

Add the certificate name to the terraform configuration with the ingress_cert_name variable. On cluster build, terraform will load the cert into a kubernetes secret, and this will be set as the default-ssl-certificate in the nginx ingress.

Create new cluster configuration

When creating a brand new cluster with its own configuration, follow these steps:

  • Create the config files in:
    • cluster/config
    • cluster/terraform_aks_cluster/config
    • cluster/terraform_kubernetes/config
  • Create the new config entry in the Makefile (e.g. test:)
  • Create low-level terraform resources: make <config> validate-azure-resources and make <config> deploy-azure-resources
  • Request the Cloud Engineering Team to assign role "Network Contributor" to the new managed identity on the new resource group
  • Create the admin AD group following the AD groups documentation
  • Use the group object id in the admin_group_id variable
  • Use PIM for groups to activate membership of the admin group
  • Run: make <environment> terraform-apply
  • Configure a domain pointing at the new ingress IP following Cluster DNS zone configuration
  • Create or update the user AD groups as per the AD groups documentation